UI-TARS-desktop

Most automation tools break the moment an interface changes. Macro recorders depend on fixed coordinates, scripts depend on stable DOM selectors, and neither can read a screen the way a person does. UI-TARS-desktop, from ByteDance, takes a different route. It is a native desktop app for Windows and macOS, with a browser build alongside it, that turns a natural-language instruction into real mouse and keyboard actions. You type what you want done, and the agent looks at the actual screen, decides where to click, and carries out the steps. It belongs to the emerging computer-use category, where an AI operates a computer the same way a human would rather than through brittle, hand-coded integrations that need constant maintenance.

The loop is conceptually simple. The app takes a screenshot, a vision-language model reads the interface and locates the relevant elements, and the agent drives the cursor and keyboard to act, then repeats with a fresh screenshot to check progress. What is distinctive is that the perception model, UI-TARS, is open-weight: the base UI-TARS-1.5-7B is published under Apache-2.0 and can run locally, so the part that actually reads your screen does not have to be a closed cloud service. The project ships alongside Agent TARS, an MCP-based CLI and web sibling that adds tool calling and works with any model provider. You can run fully local, point at ByteDance's cloud Seed and Doubao models, or mix the two depending on the task.

The natural audience is developers, QA engineers, and automation tinkerers who want to script flows that resist conventional automation. Concrete examples from the project include changing a setting buried in VS Code, checking GitHub issues, booking flights and hotels through ordinary web forms, and generating charts via connected tools. It suits legacy desktop apps with no API, repetitive multi-step UI chores, and exploratory research into how computer-use agents behave. Because the base model is genuinely Apache-2.0 and self-hostable, it is also attractive to anyone who wants to experiment without per-action cloud billing or who needs the screen to stay on their own machine for privacy or compliance reasons.

The honesty caveats are real. The open 7B model is free, but the strongest models, Doubao-1.5-UI-TARS and Seed-1.5-VL, are proprietary and billed through ByteDance's VolcEngine API, so top performance means cloud lock-in. The free remote operator was discontinued in August 2025. Computer use is also inherently risky: an agent with full mouse, keyboard, and browser control is exposed to prompt injection from web content and to plain misclicks, so a sandbox or dedicated VM is strongly advised rather than optional. The software is pre-1.0, around v0.3.0, with a large open-issue backlog, and the local hardware requirements for the 7B model are not clearly documented, which makes capacity planning a matter of trial and error.

Against Claude Computer Use and OpenAI's Operator, UI-TARS-desktop trades polish and managed safety for openness and control. The commercial options are more reliable and better guarded, but they are closed and metered. Here you own the app and can self-host the perception model, which is the real reason to choose it. Adopt it now if you want an open, self-hostable computer-use agent for automation experiments and you are comfortable running it sandboxed. Hold off if you need production-grade reliability, want the top models without a paid VolcEngine plan, or cannot isolate the agent properly. As a research and tinkering platform it is genuinely useful; as a hands-off production worker it is still early.