UI-TARS: Pioneering Automated GUI Interaction with Native Agents
- Published
- Source
- arXiv
- Paper number
- 062
- Field
- GUI Agents
- arXiv ID
- 2501.12326
Key points
- Existing GUI agents rely heavily on text-based representations and modular frameworks, which limit scalability.
- There is a lack of large, high-quality data for training end-to-end GUI agents.
- Complex GUI scenarios require advanced perception, reasoning, and coordination capabilities that current approaches do not have.
- We developed UI-TARS, an end-to-end model that interacts with interfaces by processing screenshots directly.
- We built large-scale datasets for perception, grounding, and action traces through automatic collection and curation.
- We integrated System 2 reasoning through thought augmentation and reflection tuning.
Paper links
External research summaries. These are not HDATF publications or measured product results.