Pith. sign in

REVIEW 2 major objections 3 cited by

Xiaomi-GUI-0 reaches 72.0 percent success on real mobile tasks by training inside a real-device dominant loop.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-07-02 19:35 UTC pith:767OSZZB

load-bearing objection Xiaomi-GUI-0 gives competitive numbers on real-device mobile GUI tasks but the methods and supporting measurements are missing, so the claims stay hard to assess. the 2 major comments →

arxiv 2606.31410 v2 pith:767OSZZB submitted 2026-06-30 cs.AI

Xiaomi-GUI-0 Technical Report

Wanxia Cao , Chengzhen Duan , Pei Fu , Pengzhi Gao , Niu Lian , Fazhan Liu , Hui Liu , Heng Qu
show 24 more authors
Qinzhuo Wu Zhehao Yu Tongbo Chen Shiqi Cui Anan Du Shukai Jia Yuanfa Li Wei Liu Yike Liu Wenchao Lu Zhenbo Luo Haoyuan Sun Jiatong Sun Cheng Tan Yajie Wang Changqiao Wu Tao Xiong Jiahui Yang Yuxuan Yuan Ruoceng Zhang Shaojie Zhang Jian Zhu Jian Luan Cong Zou
This is my paper
classification cs.AI
keywords GUI agentmobile devicereal-device trainingmultimodal modelreinforcement learningerror-driven data flywheelAndroidWorldexecution stability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents Xiaomi-GUI-0 as a multimodal GUI agent built to act directly on physical mobile devices instead of simulated or offline trajectories. It centers on a hybrid infrastructure that keeps physical devices as the main execution environment and uses sandboxes only for support, so that data collection, training, and evaluation stay close to actual deployment conditions. The approach adds an error-driven data flywheel that converts failure cases into corrected actions, explanations, and recovery examples, then applies this data in a three-stage training sequence of supervised fine-tuning, step-level reinforcement learning, and agentic reinforcement learning. Evaluation on public benchmarks and the in-house RealMobile set shows 72.0 percent success on RealMobile and 78.9 percent on AndroidWorld together with gains in handling abnormal states. The central goal is to shrink the mismatch between benchmark numbers and usable performance when account states, permissions, and risk controls keep changing the environment.

Core claim

Xiaomi-GUI-0 is a native multimodal GUI agent for real mobile environments, trained and evaluated inside a real-device closed loop. Its core is a real-device-dominant hybrid infrastructure in which physical devices serve as the primary execution environment while sandboxes provide auxiliary support. Training data come from multi-source collection covering head tasks, long-tail intents, and reflection and memory capabilities, processed through an error-driven data flywheel that turns failure trajectories into corrected actions, reflective explanations, and recovery demonstrations. The model follows a progressive three-stage pipeline of supervised fine-tuning, step-level reinforcement learning

What carries the argument

The real-device-dominant hybrid infrastructure, which places physical devices as the primary execution environment so that the distribution of training, rollout, and evaluation matches real deployment conditions.

Load-bearing premise

The hybrid setup with physical devices as primary produces an execution distribution close enough to real deployment that measured benchmark gains will appear as usable stability when account states, permissions, and risk controls keep changing.

What would settle it

Performance measured on a fresh set of real devices running apps with previously unseen permission dialogs or risk-control flows that were absent from the training trajectories would show whether the infrastructure actually aligns the distributions.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Execution stability rises because the training distribution already includes the abnormal states that appear in live use.
  • Abnormal-state recognition improves through the reflective and recovery demonstrations generated by the error-driven flywheel.
  • The three-stage pipeline produces progressive gains from basic action following to full agentic behavior with memory.
  • The same infrastructure can be reused for ongoing data collection without shifting away from real-device conditions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same real-device-dominant loop could be replicated on other device classes such as tablets or wearables to test whether the stability gains transfer.
  • If the flywheel keeps ingesting new failures at scale, the model could improve continuously without additional human-labeled data.
  • Wider deployment across more device models would further tighten the match between training distribution and live conditions over time.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The manuscript introduces Xiaomi-GUI-0, a native multimodal GUI agent for real mobile environments. It proposes a real-device-dominant hybrid infrastructure (physical devices primary, sandboxes auxiliary) to align data collection, training, rollout, and evaluation with real deployment distributions. Multi-source training data and an error-driven flywheel (converting failures into corrected actions and reflective explanations) are used within a three-stage pipeline of supervised fine-tuning, step-level reinforcement learning, and agentic reinforcement learning. The model is reported to achieve 72.0% success on the in-house RealMobile benchmark and 78.9% on AndroidWorld while improving execution stability and abnormal-state recognition.

Significance. If the hybrid infrastructure and three-stage pipeline demonstrably reduce the distribution gap to live deployments, the work could meaningfully advance GUI agent research by providing a practical route to higher real-world robustness. The error-driven flywheel and emphasis on abnormal states (permissions, risk controls, account states) target a documented limitation in offline/simulated training.

major comments (2)
  1. [Abstract] Abstract: The central claim that the real-device-dominant hybrid infrastructure produces an execution distribution close enough to real deployments for benchmark scores to predict usability is unsupported by any quantitative evidence, such as abnormal-state frequency statistics, KL-divergence or overlap metrics between training/eval trajectories and held-out real sessions, or ablations showing the three-stage pipeline reduces the gap.
  2. [Abstract] Abstract: The reported success rates (72.0% RealMobile, 78.9% AndroidWorld) and the assertion of 'substantially improving execution stability' are presented without details on evaluation protocol, number of tasks, error bars, baseline comparisons, task selection criteria, or how success is defined, making it impossible to verify whether the numbers reflect genuine gains or post-hoc selection.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed review and constructive suggestions. We address the major comments point-by-point below and plan to incorporate revisions to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim that the real-device-dominant hybrid infrastructure produces an execution distribution close enough to real deployments for benchmark scores to predict usability is unsupported by any quantitative evidence, such as abnormal-state frequency statistics, KL-divergence or overlap metrics between training/eval trajectories and held-out real sessions, or ablations showing the three-stage pipeline reduces the gap.

    Authors: We agree that additional quantitative evidence would strengthen the central claim in the abstract. While the manuscript emphasizes the design of the hybrid infrastructure and reports strong performance on real-device benchmarks, we will revise the paper to include abnormal-state frequency statistics and distribution overlap metrics from our internal logs. We will also provide an ablation analysis demonstrating the contribution of the three-stage pipeline to closing the distribution gap. revision: yes

  2. Referee: [Abstract] Abstract: The reported success rates (72.0% RealMobile, 78.9% AndroidWorld) and the assertion of 'substantially improving execution stability' are presented without details on evaluation protocol, number of tasks, error bars, baseline comparisons, task selection criteria, or how success is defined, making it impossible to verify whether the numbers reflect genuine gains or post-hoc selection.

    Authors: The full manuscript provides the evaluation protocol, task counts, success definitions, baseline comparisons, and error analysis in the dedicated experimental sections. The abstract summarizes the key results. We will revise the abstract to include a concise description of the evaluation setup and success criteria to improve verifiability. revision: yes

Circularity Check

0 steps flagged

No derivation chain or equations present; empirical system description only

full rationale

The paper is a technical report describing an empirical GUI agent system and its training/evaluation pipeline. No equations, mathematical derivations, fitted parameters, or predictive claims appear in the abstract or described content. Performance numbers are reported outcomes of running the system on benchmarks, not results derived from prior quantities within the paper. No self-citations, ansatzes, or uniqueness theorems are invoked in a load-bearing way. The design choice of a real-device-dominant hybrid infrastructure is presented as an engineering decision to align distributions, not as a derived result that reduces to its own inputs. This is a standard non-finding for an empirical systems paper with no formal derivation chain.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; all technical details are absent.

pith-pipeline@v0.9.1-grok · 5942 in / 1139 out tokens · 25999 ms · 2026-07-02T19:35:25.894688+00:00 · methodology

0 comments
read the original abstract

Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, text entry, and navigation. However, existing GUI agents are trained and evaluated largely on offline trajectories, simulated environments, and standardized benchmarks. These differ substantially from real applications in interface layout, interaction logic, and abnormal-state distribution, and cannot faithfully characterize execution stability in real-world use, where account states, permission dialogs, payment authentication, and risk control continually reshape the state distribution and open a persistent gap between benchmark scores and real usability. To close this gap, we propose Xiaomi-GUI-0, a native multimodal GUI agent for real mobile environments, trained and evaluated within a real-device closed loop. At its core is a real-device-dominant hybrid infrastructure, where physical devices are the primary execution environment and sandboxes provide auxiliary support, so that data collection, training, rollout, and evaluation share an execution distribution close to real deployment. We construct multi-source training data spanning high-frequency head tasks, high-generalization data for long-tail intents, and capability-enhancement data for reflection and memory, and introduce an error-driven data flywheel that turns failure trajectories into corrected actions, reflective explanations, and recovery demonstrations. The model is trained through a progressive three-stage pipeline of supervised fine-tuning, step-level reinforcement learning, and agentic reinforcement learning. Evaluated on public benchmarks and our in-house RealMobile, Xiaomi-GUI-0 achieves 72.0% success on RealMobile and 78.9% on AndroidWorld, while substantially improving execution stability and abnormal-state recognition in real-world tasks.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Scaling GUI Agents with Visual State Transitions

    cs.AI 2026-07 conditional novelty 6.0

    A joint inverse-forward pretraining stage on visual screen transitions improves GUI-agent fine-tuning by 0.6–6.2 percentage points across three benchmarks.

  2. UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning

    cs.CL 2026-07 conditional novelty 6.0

    Platform-conditioned multi-teacher on-policy distillation lets a shared 8B GUI policy adapt across desktop and mobile without collapsing platform-specific action conventions.

  3. Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents

    cs.AI 2026-07 conditional novelty 5.5

    A real-device-centric foundation GUI agent with hybrid GUI+CLI batched actions, AutoResearch data flywheel, online RL, and a proactive harness reaches SOTA mobile and competitive desktop/web scores.

Reference graph

Works this paper leans on

77 extracted references · 77 canonical work pages · cited by 3 Pith papers · 21 internal anchors

  1. [1]

    Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet

    Anthropic. Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet. Model card addendum, 2024. URL https://assets.anthropic.com/m/1cd9d098ac3e6467/original/ Claude-3-Model-Card-October-Addendum.pdf

    work page 2024

  2. [2]

    Introducing claude opus 4.6

    Anthropic. Introducing claude opus 4.6. Anthropic announcement, 2026. URL https://www.anthropic.com/news/ claude-opus-4-6

    work page 2026

  3. [3]

    Introducing claude opus 4.7

    Anthropic. Introducing claude opus 4.7. Anthropic announcement, 2026. URL https://www.anthropic.com/news/ claude-opus-4-7

    work page 2026

  4. [4]

    Digirl: Training in- the-wild device-control agents with autonomous reinforcement learning

    Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, and A viral Kumar. Digirl: Training in- the-wild device-control agents with autonomous reinforcement learning. Advances in Neural Information Processing Systems, 37:12461–12495, 2024

    work page 2024

  5. [5]

    Qwen3-VL Technical Report

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 , 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  6. [6]

    Seed2.0 model card: Towards intelligence frontier for real-world complexity

    ByteDance Seed. Seed2.0 model card: Towards intelligence frontier for real-world complexity. arXiv preprint arXiv:2603.11103, 2026. URL https://arxiv.org/abs/2603.11103

    work page arXiv 2026

  7. [7]

    KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation

    Tongbo Chen, Zhengxi Lu, Zhan Xu, Guocheng Shao, Shaohan Zhao, Fei Tang, Yong Du, Kaitao Song, Yizhou Liu, Yuchen Yan, et al. Knowu-bench: Towards interactive, proactive, and personalized mobile agent evaluation. arXiv preprint arXiv:2604.08455 , 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  8. [8]

    arXiv preprint arXiv:2511.13091 , year=

    Yuhan Chen, Yuxuan Liu, Long Zhang, Pengzhi Gao, Jian Luan, and Wei Liu. Step: Success-rate-aware trajectory- efficient policy optimization. arXiv preprint arXiv:2511.13091 , 2025

    work page arXiv 2025

  9. [9]

    UIShift: Enhancing VLM-based GUI Agents through Self- supervised Reinforcement Learning.arXiv preprint arXiv:2505.12493, 2025

    Longxi Gao, Li Zhang, Pengzhi Gao, Wei Liu, Jian Luan, and Mengwei Xu. Gui-shift: Enhancing vlm-based gui agents through self-supervised reinforcement learning. arXiv preprint arXiv:2505.12493 , 2025

    work page arXiv 2025

  10. [10]

    Gemini 3.1 pro model card

    Google DeepMind. Gemini 3.1 pro model card. Model card, 2026. URL https://deepmind.google/models/ model-cards/gemini-3-1-pro/

    work page 2026

  11. [11]

    Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training

    Jihao Gu, Qihang Ai, Yingyao Wang, Pi Bu, Jingxuan Xing, Zekun Zhu, Wei Jiang, Ziming Wang, Yingxiu Zhao, Ming-Liang Zhang, et al. Mobile-r1: Towards interactive reinforcement learning for vlm-based mobile agent via task-level rewards. arXiv preprint arXiv:2506.20332 , 2025. 25

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  12. [12]

    arXiv preprint arXiv:2507.01663 (2025)

    Zhenyu Han, Ansheng You, Haibo Wang, Kui Luo, Guang Yang, Wenqi Shi, Menglong Chen, Sicheng Zhang, Zeshun Lan, Chunshi Deng, Huazhong Ji, Wenjie Liu, Yu Huang, Yixiang Zhang, Chenyi Pan, Jing Wang, Xin Huang, Chunsheng Li, and Jianping Wu. Asyncflow: An asynchronous streaming rl framework for efficient llm post-training, 2025. URL https://arxiv.org/abs/2507.01663

    work page arXiv 2025

  13. [13]

    arXiv preprint arXiv:2505.12299 , year=

    Kun Huang, Weikai Xu, Yuxuan Liu, Quandong Wang, Pengzhi Gao, Wei Liu, Jian Luan, Bin Wang, and Bo An. Mo- bileipl: Enhancing mobile agents thinking process via iterative preference learning. arXiv preprint arXiv:2505.12299 , 2025

    work page arXiv 2025

  14. [14]

    GPT-4o System Card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 , 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  15. [15]

    Screenspot-pro: Gui grounding for professional high-resolution computer use

    Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. Screenspot-pro: Gui grounding for professional high-resolution computer use. In Proceedings of the 33rd ACM International Conference on Multimedia , pages 8778–8786, 2025

    work page 2025

  16. [16]

    From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents

    Niu Lian, Yuting Wang, Hanshu Yao, Jinpeng Wang, Bin Chen, Yaowei Wang, Min Zhang, and Shu-Tao Xia. From verbatim to gist: Distilling pyramidal multimodal memory via semantic information bottleneck for long-horizon video agents. arXiv preprint arXiv:2603.01455 , 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  17. [17]

    ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis

    Guohong Liu, Jialei Ye, Pengzhi Gao, Wei Liu, Jian Luan, Yunxin Liu, and Yuanchun Li. Simuwob: Simulating real-world mobile apps for fast and faithful gui agent benchmarking. arXiv preprint arXiv:2605.25160 , 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  18. [18]

    Pc-agent: A hierarchical multi-agent collaboration framework for complex task automation on pc.arXiv preprint arXiv:2502.14282, 2025

    Haowei Liu, Xi Zhang, Haiyang Xu, Yuyang Wanyan, Junyang Wang, Ming Yan, Ji Zhang, Chunfeng Yuan, Chang- sheng Xu, Weiming Hu, et al. Pc-agent: A hierarchical multi-agent collaboration framework for complex task automation on pc. arXiv preprint arXiv:2502.14282 , 2025

    work page arXiv 2025

  19. [19]

    AutoGLM: Autonomous Foundation Agents for GUIs.arXiv preprint arXiv:2411.00820, 2024

    Xiao Liu, Bo Qin, Dongzhu Liang, Guang Dong, Hanyu Lai, Hanchen Zhang, Hanlin Zhao, Iat Long Iong, Jiadai Sun, Jiaqi Wang, et al. Autoglm: Autonomous foundation agents for guis. arXiv preprint arXiv:2411.00820 , 2024

    work page arXiv 2024

  20. [20]

    Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning

    Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Pengxiang Zhao, Guangyi Liu, et al. Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 40, pages 17608–17616, 2026

    work page 2026

  21. [21]

    GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents

    Run Luo, Lu Wang, Wanwei He, Longze Chen, Jiaming Li, and Xiaobo Xia. Gui-r1: A generalist r1-style vision- language action model for gui agents. arXiv preprint arXiv:2504.10458 , 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  22. [22]

    Addendum to openai o3 and o4-mini system card: Openai o3 operator

    OpenAI. Addendum to openai o3 and o4-mini system card: Openai o3 operator. System card addendum, 2025. URL https://openai.com/index/o3-o4-mini-system-card-addendum-operator-o3/

    work page 2025

  23. [23]

    Toolllm: Facilitating large language models to master 16000+ real-world apis

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. In International Conference on Learning Representations, volume 2024, pages 9695–9717, 2024

    work page 2024

  24. [24]

    UI-TARS: Pioneering Automated GUI Interaction with Native Agents

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents, 2025. URL https://arxiv. org/abs/2501.12326, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  25. [25]

    Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation

    Heng Qu, Yike Liu, Renren Jin, Wenzong Zhang, Pengzhi Gao, Wei Liu, and Jian Luan. Scaling, benchmarking, and reasoning of vision-language agents for mobile gui navigation. arXiv preprint arXiv:2605.27134 , 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  26. [26]

    Androidworld: A dynamic benchmarking environment for au- tonomous agents

    Chris Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, et al. Androidworld: A dynamic benchmarking environment for au- tonomous agents. In International Conference on Learning Representations , volume 2025, pages 406–441, 2025

    work page 2025

  27. [27]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in neural information processing systems , 36:68539–68551, 2023

    work page 2023

  28. [28]

    Ui-tars-1.5

    Seed. Ui-tars-1.5. ByteDance Seed Blog, 2025. URL https://seed-tars.com/1.5/

    work page 2025

  29. [29]

    Bytedance Seed. Seed1. 8 model card: Towards generalized real-world agency. arXiv preprint arXiv:2603.20633 , 2026. 26

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  30. [30]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems , pages 1279–1297, 2025

    work page 2025

  31. [31]

    Gui knowledge bench: Revealing the knowledge gap behind vlm failures in gui tasks.arXiv preprint arXiv:2510.26098, 2025

    Chenrui Shi, Zedong Yu, Zhi Gao, Ruining Feng, Enqi Liu, Yuwei Wu, Yunde Jia, Liuyu Xiang, Zhaofeng He, and Qing Li. Gui knowledge bench: Revealing the knowledge gap behind vlm failures in gui tasks. arXiv preprint arXiv:2510.26098, 2025

    work page arXiv 2025

  32. [32]

    arXiv preprint arXiv:2507.05720 , year=

    Yucheng Shi, Wenhao Yu, Zaitang Li, Yonglin Wang, Hongming Zhang, Ninghao Liu, Haitao Mi, and Dong Yu. Mobilegui-rl: Advancing mobile gui agent through reinforcement learning in online environment. arXiv preprint arXiv:2507.05720, 2025

    work page arXiv 2025

  33. [33]

    Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1909

  34. [34]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 , 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  35. [35]

    arXiv preprint arXiv:2602.09082 , year=

    Venus Team, Changlong Gao, Zhangxuan Gu, Yulin Liu, Xinyu Qiu, Shuheng Shen, Yue Wen, Tianyu Xia, Zhenyu Xu, Zhengwen Zeng, et al. Ui-venus-1.5 technical report. arXiv preprint arXiv:2602.09082 , 2026

    work page arXiv 2026

  36. [36]

    CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

    Bowen Wang, Dunjie Lu, Junli Wang, Tianyi Bai, Shixuan Liu, Zhipeng Zhang, Haiquan Wang, Hao Hu, Tianbao Xie, Shuai Bai, et al. Cua-gym: Scaling verifiable training environments and tasks for computer-use agents. arXiv preprint arXiv:2605.25624 , 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  37. [37]

    UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning

    Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, Junjie Fang, Junting Lu, Longxiang Liu, Qinyu Luo, Shihao Liang, Shijue Huang, et al. Ui-tars-2 technical report: Advancing gui agent with multi-turn reinforcement learning. arXiv preprint arXiv:2509.02544 , 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  38. [38]

    Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration

    Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration. Advances in Neural Information Processing Systems , 37:2686–2710, 2024

    work page 2024

  39. [39]

    Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception

    Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. Mobile-agent: Autonomous multi-modal mobile device agent with visual perception. arXiv preprint arXiv:2401.16158 , 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  40. [40]

    Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 , 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    Opencua: Open foundations for computer-use agents

    Xinyuan Wang, Bowen Wang, Dunjie Lu, Junlin Yang, Tianbao Xie, Junli Wang, Jiaqi Deng, Xiaole Guo, Yiheng Xu, Chen Wu, et al. Opencua: Open foundations for computer-use agents. Advances in Neural Information Processing Systems, 38:139756–139806, 2026

    work page 2026

  42. [42]

    OpenClaw-RL: Train Any Agent Simply by Talking

    Yinjie Wang, Xuyang Chen, Xiaolong Jin, Mengdi Wang, and Ling Yang. Openclaw-rl: Train any agent simply by talking. arXiv preprint arXiv:2603.10165 , 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  43. [43]

    MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

    Dingbang Wu, Rui Hao, Haiyang Wang, Shuzhe Wu, Han Xiao, Zhenghong Li, Bojiang Zhou, Zheng Ju, Zichen Liu, Lue Fan, et al. Mobilegym: A verifiable and highly parallel simulation platform for mobile gui agent research. arXiv preprint arXiv:2605.26114 , 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  44. [44]

    Backtrackagent: Enhancing gui agent with error detection and backtracking mechanism

    Qinzhuo Wu, Pengzhi Gao, Wei Liu, and Jian Luan. Backtrackagent: Enhancing gui agent with error detection and backtracking mechanism. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 4250–4272, 2025

    work page 2025

  45. [45]

    Mobilebench-ol: A comprehensive chinese benchmark for evaluating mobile gui agents in real-world environment

    Qinzhuo Wu, Zhizhuo Yang, Hanhao Li, Pengzhi Gao, Wei Liu, and Jian Luan. Mobilebench-ol: A comprehensive chinese benchmark for evaluating mobile gui agents in real-world environment. arXiv preprint arXiv:2601.20335 , 2026

    work page arXiv 2026

  46. [46]

    arXiv preprint arXiv:2509.23263 (2025)

    Tao Xiong, Xavier Hu, Yurun Chen, Yuhang Liu, Changqiao Wu, Pengzhi Gao, Wei Liu, Jian Luan, and Shengyu Zhang. Gui-pra: Process reward agent for gui tasks. arXiv preprint arXiv:2509.23263 , 2025. 27

    work page arXiv 2025

  47. [47]

    arXiv preprint arXiv:2602.16855 , year=

    Haiyang Xu, Xi Zhang, Haowei Liu, Junyang Wang, Zhaozai Zhu, Shengjie Zhou, Xuhao Hu, Feiyu Gao, Junjie Cao, Zihua Wang, et al. Mobile-agent-v3. 5: Multi-platform fundamental gui agents. arXiv preprint arXiv:2602.16855 , 2026

    work page arXiv 2026

  48. [48]

    Sman-bench: A cross-system benchmark for mobile agents under single-and multi-path, ambiguous, and noisy tasks

    Weikai Xu, Zhizheng Jiang, Yuxuan Liu, Pengzhi Gao, Wei Liu, Jian Luan, Yunxin Liu, Yuanchun Li, Bin Wang, and Bo An. Sman-bench: A cross-system benchmark for mobile agents under single-and multi-path, ambiguous, and noisy tasks. In The Fourteenth International Conference on Learning Representations , 2026

    work page 2026

  49. [49]

    arXiv preprint arXiv:2512.15431 , year=

    Haolong Yan, Jia Wang, Xin Huang, Yeqing Shen, Ziyang Meng, Zhimin Fan, Kaijun Tan, Jin Gao, Lieyu Shi, Mi Yang, et al. Step-gui technical report. arXiv preprint arXiv:2512.15431 , 2025

    work page arXiv 2025

  50. [50]

    Mobile-Agent-v3: Fundamental Agents for GUI Automation

    Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, Junyang Wang, Zhaoqing Zhu, Ziwei Zheng, Feiyu Gao, Junjie Cao, Zhengxi Lu, et al. Mobile-agent-v3: Fundamental agents for gui automation, 2025. URL https://arxiv. org/abs/2508.15144, 4:21–27, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  51. [51]

    DAPO: An Open-Source LLM Reinforcement Learning System at Scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale, 2025. URL https://arxiv. org/abs/2503.14476, 1:2, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  52. [52]

    Group Sequence Policy Optimization

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprint arXiv:2507.18071 , 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  53. [53]

    Sglang: Efficient execution of structured language model programs

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody H Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs. Advances in neural information processing systems , 37:62557–62583, 2024

    work page 2024

  54. [54]

    arXiv preprint arXiv:2512.22047 , year=

    Hanzhang Zhou, Xu Zhang, Panrong Tong, Jianan Zhang, Liangyu Chen, Quyu Kong, Chenglin Cai, Chen Liu, Yue Wang, Jingren Zhou, et al. Mai-ui technical report: Real-world centric foundation gui agents. arXiv preprint arXiv:2512.22047, 2025. 28 Contributions and Acknowledgments All contributors are listed in alphabetical order by their last names. Core Contr...

    work page arXiv 2025

  55. [55]

    Current device type & foreground app

    work page

  56. [56]

    Output the corresponding JSON string inside `<tool_call>`.,→

    Current screenshot # Available Tools You MUST pick exactly one tool per step. Output the corresponding JSON string inside `<tool_call>`.,→

    work page

  57. [57]

    name": "Tap

    Tap: `{"name": "Tap", "position": [x, y], "times": 1}` (Tap at coordinate)

    work page

  58. [58]

    name": "LongPress

    LongPress: `{"name": "LongPress", "position": [x, y]}` (Trigger contextual menus)

    work page

  59. [59]

    name": "Swipe

    Swipe: `{"name": "Swipe", "start_position": [x1, y1], "end_position": [x2, y2]}` (Swipe to scroll/move. Swipe up to scroll down),→

    work page

  60. [60]

    name": "Type

    Type: `{"name": "Type", "position": [x, y], "text": "..."}` (Tap input box and type)

    work page

  61. [61]

    name": "Search

    Search: `{"name": "Search", "position": [x, y], "text": "..."}` (Macro: tap -> clear -> type -> submit),→

    work page

  62. [62]

    name": "Open

    Open: `{"name": "Open", "app": "..."}` (Launch app via system)

    work page

  63. [63]

    name": "Back

    Back: `{"name": "Back"}` (System-level back)

    work page

  64. [64]

    name": "Home

    Home: `{"name": "Home"}` (Go to home screen)

    work page

  65. [65]

    name": "Wait

    Wait: `{"name": "Wait"}` (Wait for page loading/rendering)

    work page

  66. [66]

    name": "Request

    Request: `{"name": "Request", "text": "..."}` (Ask user for clarification/confirmation)

    work page

  67. [67]

    name": "Fail

    Fail: `{"name": "Fail", "type": "...", "reason": "..."}` (Report failure. `<TYPE>` MUST be one of: LOGIN_REQUIRED, USE_GUIDANCE, CAPTCHA_VERIFICATION, RESULT_NOT_FOUND, BLUETOOTH_CONNECTION_REQUIRED, NETWORK_ERROR, PAYMENT_AUTHENTICATION, TASK_CANT_FULFILLED, REPEAT_OPERATION, PERMISSION_REQUEST, PASSWORD_REQUIRED, TAKEOVER_EXIT, TEMPORARY_TAKEOVER, MANUA...

    work page

  68. [68]

    name": "Complete

    Complete: `{"name": "Complete"}` (Confirm goal reached for non-Q&A tasks)

    work page

  69. [69]

    name": "Speak

    Speak: `{"name": "Speak", "text": "..."}` (Present final answer for Q&A tasks) # Operational Constraints

    work page

  70. [70]

    Top-left is (0, 0); bottom-right is (1, 1).,→

    Coordinate system: every `position` is a relative [x, y] in [0, 1] with 3-decimal precision. Top-left is (0, 0); bottom-right is (1, 1).,→

    work page

  71. [71]

    Dismiss unrelated pop-ups (ads, upgrade prompts, rating requests) by tapping their Close / Skip / X / "Later" button rather than calling Fail.,→

    work page

  72. [72]

    If self-correction fails, call Fail

    Loop breaker: if three consecutive steps cause no visible change, or the same action is repeating in a loop, self-correct (try Back or a different target). If self-correction fails, call Fail. ,→ ,→ # Reasoning Framework (inside <think>) Before emitting the action, reason inside `<think>...</think>` (omit steps if no new info):

    work page

  73. [73]

    [Observation]: Objectively describe the current App, page state, and key visible elements

    work page

  74. [74]

    Explain what was expected vs

    [Reflection]: (Optional) Include ONLY if the current screen deviates from the previous plan's expectation. Explain what was expected vs. what is actually seen.,→

    work page

  75. [75]

    Output a 2-4 step path in a single line separated by `|`

    [Plan] / [Plan Update] / [Replan]: (Choose one). Output a 2-4 step path in a single line separated by `|`. Mark completed steps with `[done]` and the current step with `->`. Use [Replan] if the previous plan failed. ,→ ,→

    work page

  76. [76]

    [Decision]: Deduce the exact action based on the Observation and the current `->` step in the Plan.,→

    work page

  77. [77]

    Tap the search bar

    [Memory]: Cache persistent info needed for future steps. # Output Format Your assistant message `content` MUST strictly follow this three-span XML-like shape, in this exact order, with no extra text before, between, or after:,→ <think> [your observation, analysis and reasoning] </think> <action> [a short natural-language description of the action, e.g., "...

    work page