Pith. sign in

REVIEW 3 major objections 5 minor 68 references

A shared GUI agent can learn desktop and mobile habits without averaging them away by routing each platform to its own teacher during on-policy distillation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 19:15 UTC pith:CGKLLUIE

load-bearing objection Solid multi-platform GUI training recipe with real Uni-GUI data and clean OSWorld/MobileWorld numbers; the MOPD attribution is under-isolated but the paper is still worth engaging. the 3 major comments →

arxiv 2607.04425 v1 pith:CGKLLUIE submitted 2026-07-05 cs.CL cs.AIcs.CVcs.LGcs.MM

UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning

Niu Lian , Alan Chen , Zhehao Yu , Chengzhen Duan , Fazhan Liu , Hui Liu , Pei Fu , Jian Luan
show 3 more authors
Yaowei Wang Shu-Tao Xia Jinpeng Wang
This is my paper
classification cs.CL cs.AIcs.CVcs.LGcs.MM
keywords GUI agentsmulti-platformon-policy distillationcontinual learningOSWorldMobileWorldUni-GUIplatform-conditioned routing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Cross-platform GUI agents fail when desktop and mobile signals are mixed: the distinct action conventions of each platform collapse into an averaged policy, and continual training forgets earlier platform skills. This paper argues that the fix is not more mixed data or static model merging, but platform-conditioned multi-teacher on-policy distillation. First the authors build Uni-GUI, a cleaned set of roughly ten thousand executable desktop and mobile trajectories. Then they fine-tune separate strong teachers on each platform and train a single smaller student that samples its own rollouts online; for every rollout the matching teacher supplies reverse-KL guidance so the student absorbs native interaction patterns only on the states it actually visits. The resulting shared policy reaches 38.2 percent success on OSWorld and 12.0 percent on MobileWorld, beating mixed supervised fine-tuning and weight-merging baselines while keeping both platforms alive in one model.

Core claim

UI-MOPD shows that multi-teacher on-policy distillation, routed by platform label, can transfer platform-specific behavioral priors into one shared GUI student: desktop rollouts are aligned only to the desktop teacher and mobile rollouts only to the mobile teacher, so the student improves task success on both environments without collapsing their interaction conventions or erasing earlier platform skills.

What carries the argument

Platform-conditioned multi-teacher on-policy distillation (MOPD): the student samples trajectories online, a platform router selects the matching frozen teacher, and a K3 reverse-KL term plus adaptive reward-gated mask pulls the student toward that teacher's token distribution only on the visited states.

Load-bearing premise

The paper assumes that reverse-KL alignment to frozen platform teachers on the student's own rollouts is a strong enough and non-destructive anchor to keep native interaction conventions from collapsing or being ignored.

What would settle it

Train the same 8B student with identical Uni-GUI data and rewards but without platform routing (or with a single mixed teacher); if OSWorld and MobileWorld success then fall to or below mixed-SFT and model-merge levels, or if one platform collapses while the other rises, the claim that platform-conditioned MOPD is what prevents convention mixing fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses continual multi-platform GUI agent learning by releasing Uni-GUI (~10–11.5K high-quality desktop/mobile trajectories from a unified collection harness) and proposing UI-MOPD: Stage-1 SFT yields frozen platform-specific 32B teachers; Stage-2 trains a shared 8B student with GRPO-style online RL plus platform-routed reverse-KL on-policy distillation (K3 estimator, adaptive group-level KL mask; Eqs. 1–7, 10–12) and a structured action-JSON reward (Eq. 8). On interactive benchmarks the student reaches 38.2% success on OSWorld (361 tasks) and 12.0% on MobileWorld (117 tasks), outperforming Mixed-SFT and weight/TIES merging (Table 1) and avoiding the cross-platform collapse of single-platform 8B SFT (Table 2), while largely preserving static GUI grounding (Table 3).

Significance. If the attribution holds, this is a useful and timely contribution to multi-platform GUI agents: it is, to the authors’ knowledge, the first use of multi-teacher on-policy distillation for continual GUI learning, pairs a carefully filtered cross-platform dataset with a concrete platform-conditioned routing design, and reports balanced gains on two standard interactive suites rather than only static grounding. The teacher–student analysis (Table 2) and the contrast with Mixed-SFT/model merge (Table 1) are the right experimental axes for the claimed problem of behavioral-convention mixing. Strengths that should be credited include the explicit Uni-GUI construction pipeline (Appendix A–B), the full training configuration (Appendix C), and the demonstration that interactive gains need not destroy ScreenSpot/OSWorld-G grounding (Table 3/7). The work is of clear interest to the GUI-agent and continual multimodal-agent communities even if the isolation of MOPD versus data/reward remains incomplete.

major comments (3)
  1. [§4.3–4.4, Tables 1–2, Eqs. 3, 7, 10–12] Central attribution of the headline OSWorld/MobileWorld gains to platform-conditioned MOPD is under-isolated. Teachers are SFT’d on Uni-GUI and frozen; the student is optimized under the same structured reward (Eq. 8) with Uni-GUI-derived mixed-platform rollouts. Table 1’s Mixed-SFT and merge baselines and Table 2’s single-platform 8B SFT do not include a matched control that runs the same GRPO/DAPO online RL + reward without the routed reverse-KL term (Eqs. 3, 10–12), nor a non-routed multi-teacher KL baseline, nor a same-size teacher ablation. Without those, gains could largely come from high-quality dual-platform data plus RL rather than routing/MOPD. A load-bearing revision is to add at least: (i) RL-only (no KL), (ii) single-teacher or mixed-teacher KL without platform routing, under matched data, reward, and compute.
  2. [§3.2, Eq. (6); §3.5, Eq. (10)] The adaptive KL mask (Eq. 6) zeros teacher supervision when the prompt-group mean reward exceeds τ_KL. The paper’s narrative treats routed teachers as stable behavioral anchors that prevent convention averaging precisely during successful optimization; selectively disabling the anchor on high-reward groups is therefore in tension with that claim and is not ablated (e.g., fixed-β KL vs adaptive mask, or sensitivity to τ_KL). Please report mask firing rates by platform and an ablation showing that dual-platform retention still holds when the mask is off or when β is held fixed.
  3. [Abstract; §1; §3.1; §4.4] Continual-learning framing is only weakly operationalized. The abstract and introduction emphasize continual adaptation and catastrophic forgetting, but Stage 2 is effectively joint multi-platform on-policy training with simultaneous desktop/mobile routing rather than a sequential platform-arrival protocol with measured forgetting curves (e.g., train mobile after desktop freeze, then re-evaluate OSWorld). Table 2 shows single-platform SFT collapse, which supports interference risk, but does not establish that UI-MOPD is a continual learner rather than a better joint multi-task regularizer. Either add a sequential continual protocol or temper the continual-learning claims to multi-platform joint adaptation with retention.
minor comments (5)
  1. [§2.1] Section 2.1 title and heading text use “plantform” twice (“Single-plantform”, “Multi-plantform”); correct to “platform”.
  2. [Abstract; §1; Appendix A, Table 4] Uni-GUI scale is stated inconsistently: abstract/intro “nearly 10K” trajectories vs Appendix Table 4 “~11.5K” trajectories / “~160K” steps. Align numbers across abstract, §1, and Appendix A.
  3. [Figure 1; §1] Figure 1 caption and panel labels are helpful, but the main text never quantifies “action convention collapse” (e.g., rate of mobile-style actions on desktop rollouts under Mixed-SFT vs UI-MOPD). A small diagnostic would strengthen the motivation figure.
  4. [§3.2; Appendix C, Table 6] Hyperparameters β=0.01 and τ_KL are listed in Appendix C / free parameters but τ_KL’s numerical value and selection procedure are not stated in the main method section; please specify.
  5. [§4.2–4.3, Table 1] Table 1 marks many MobileWorld entries as “–”; for fairness, note which baselines were not run vs inapplicable, and whether evaluation protocols (max steps, success criteria) match published numbers.

Circularity Check

0 steps flagged

No circularity: external-benchmark success rates are not forced by construction from Uni-GUI fits or self-cited uniqueness claims.

full rationale

UI-MOPD is an empirical methods paper. Stage-1 teachers are SFT’d on Uni-GUI and frozen; Stage-2 optimizes a shared student with a GRPO/DAPO-style policy gradient plus reverse-KL (K3) to platform-routed teachers and a structured action reward (Eqs. 1–12). Reported headline numbers (38.2% OSWorld, 12.0% MobileWorld) are interactive task success rates on external benchmarks, not quantities algebraically identical to fitted training parameters. There is no self-definitional loop (X defined via Y then “predicted”), no fitted constant renamed as a prediction, no load-bearing uniqueness theorem imported from overlapping authors, and no ansatz smuggled in via self-citation that forces the result. Using Uni-GUI for teacher SFT and as the data foundation for student training is ordinary supervised/RL practice, not circular derivation. Ablation gaps (whether routing/KL—not data or reward—drive the gains) are experimental-isolation concerns, not circularity under this pass’s criteria. Derivation chain is self-contained against external evaluation; score 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

Load-bearing content is empirical: a new dataset recipe, a platform-router MOPD objective, and hand-chosen RL/distillation hyperparameters. No deep mathematical axioms; domain assumptions are standard RLHF/agent training choices plus the claim that platform labels are known at training time.

free parameters (4)
  • OPD KL coefficient β = 0.01
    Set to 0.01 in Table 6; controls distillation strength vs policy-gradient term and is not derived.
  • Adaptive KL mask threshold τ_KL
    Group-level reward threshold that zeros teacher KL when average group reward exceeds τ_KL (Eq. 6); value not derived from first principles.
  • Structured action reward levels = 1.0 / -0.5 / -1.0
    R ∈ {1.0, −0.5, −1.0} for full match / partial / invalid (Eq. 8); discrete hand design that shapes advantages.
  • GRPO/DAPO clip ratios and rollout count = 0.2/0.28, n=8, lr=1e-6
    clip low/high 0.2/0.28, 8 rollouts per prompt, LR 1e-6; standard but free training knobs.
axioms (4)
  • domain assumption Platform label of each rollout is known and correctly routes to the matching teacher (Eq. 7).
    Routing is defined by data-source label recorded at collection; inference does not need teachers, but training assumes clean platform tags.
  • domain assumption Reverse KL from student to frozen platform teacher on on-policy states transfers useful behavioral priors without requiring full-vocabulary KL.
    Core MOPD modeling choice (Eqs. 1–5); justified by prior OPD literature but not proved for GUI action spaces.
  • ad hoc to paper Rule-based partial-match reward on action JSON is a valid proxy for long-horizon task success during RL.
    Reward design §3.4; intermediate −0.5 penalty is paper-specific and not environment-native success.
  • standard math Standard policy-gradient / GRPO math and nonnegativity of the K3 KL estimator.
    Used without re-proof; standard in RLHF toolkits (verl).
invented entities (3)
  • Uni-GUI dataset no independent evidence
    purpose: Provide ~10–11.5K high-quality executable cross-platform trajectories after unified collection and multi-stage cleaning.
    New resource constructed for this work; independent value depends on public release quality.
  • UI-MOPD / platform-conditioned multi-teacher on-policy distillation no independent evidence
    purpose: Integrate desktop and mobile expert behaviors into one shared student without convention collapse.
    Named method of the paper; evaluated only via the reported benchmarks.
  • Unified cross-platform data collection harness no independent evidence
    purpose: Generate, collect, clean, and normalize desktop/mobile trajectories under one pipeline.
    Engineering artifact enabling Uni-GUI; not a physical entity but a postulated reusable system.

pith-pipeline@v1.1.0-grok45 · 25341 in / 3404 out tokens · 36262 ms · 2026-07-11T19:15:39.925164+00:00 · methodology

0 comments
read the original abstract

Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction. However, building multi-platform GUI agents remains challenging. On one hand, high-quality and executable cross-platform interaction trajectories are still scarce, and existing data often suffer from limited platform coverage. On the other hand, different platforms exhibit distinct interaction conventions, making joint or continual training prone to behavioral pattern mixing, platform-specific capability degradation, and catastrophic forgetting. To address these challenges, we construct Uni-GUI, a high-quality cross-platform GUI interaction dataset, and propose UI-MOPD, the first method that incorporates multi-teacher on-policy distillation into continual learning for GUI agents. UI-MOPD dynamically selects a platform-specific teacher according to the current environment and transfers platform-specific behavioral priors to a shared policy through platform-conditioned distillation, enabling adaptation to new platforms while preserving capabilities on existing ones. Experiments on OSWorld and MobileWorld show that UI-MOPD achieves task success rates of 38.2% and 12.0%, respectively, demonstrating its effectiveness in balancing cross-platform capability retention and new-platform adaptation. Project page: https://elispectre.github.io/UI-MOPD/.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 23 linked inside Pith

  1. [1]

    Introducing claude opus 4.6

    Anthropic. Introducing claude opus 4.6. Anthropic announcement, 2026. URLhttps://www.anthropic.com/ news/claude-opus-4-6

    2026

  2. [2]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Windows agent arena: Evaluating multi-modal os agents at scale.arXiv preprint arXiv:2409.08264, 2024

    Rogerio Bonatti, Dan Zhao, Francesco Bonacci, Dillon Dupont, Sara Abdali, Yinheng Li, Yadong Lu, Justin Wagle, Kazuhito Koishida, Arthur Bucker, et al. Windows agent arena: Evaluating multi-modal os agents at scale.arXiv preprint arXiv:2409.08264, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    Seed2.0 model card: Towards intelligence frontier for real-world complexity.arXiv preprint arXiv:2603.11103, 2026

    ByteDance Seed. Seed2.0 model card: Towards intelligence frontier for real-world complexity.arXiv preprint arXiv:2603.11103, 2026. URLhttps://arxiv.org/abs/2603.11103

    arXiv 2026

  5. [5]

    Xiaomi-gui-0 technical report, 2026

    Wanxia Cao, Chengzhen Duan, Pei Fu, Pengzhi Gao, Niu Lian, Fazhan Liu, Hui Liu, Heng Qu, Qinzhuo Wu, Zhehao Yu, Tongbo Chen, Shiqi Cui, Anan Du, Shukai Jia, Yuanfa Li, Wei Liu, Yike Liu, Wenchao Lu, Zhenbo Luo, Haoyuan Sun, Jiatong Sun, Cheng Tan, Yajie Wang, Changqiao Wu, Tao Xiong, Jiahui Yang, Yuxuan Yuan, Ruoceng Zhang, Shaojie Zhang, Jian Zhu, Jian L...

    Pith/arXiv arXiv 2026

  6. [6]

    Knowu-bench: Towards interactive, proactive, and personalized mobile agent evaluation

    Tongbo Chen, Zhengxi Lu, Zhan Xu, Guocheng Shao, Shaohan Zhao, Fei Tang, Yong Du, Kaitao Song, Yizhou Liu, Yuchen Yan, et al. Knowu-bench: Towards interactive, proactive, and personalized mobile agent evaluation. arXiv preprint arXiv:2604.08455, 2026

    Pith/arXiv arXiv 2026

  7. [7]

    Openmobile: Building open mobile agents with task and trajectory synthesis.arXiv preprint arXiv:2604.15093, 2026

    Kanzhi Cheng, Zehao Li, Zheng Ma, Nuo Chen, Jialin Cao, Qiushi Sun, Zichen Ding, Fangzhi Xu, Hang Yan, Jiajun Chen, et al. Openmobile: Building open mobile agents with task and trajectory synthesis.arXiv preprint arXiv:2604.15093, 2026

    Pith/arXiv arXiv 2026

  8. [8]

    Audio-oscar: A multi-agent system for complex audio scene generation, orchestration, and refinement

    Yifan Duan, Qixiang Xu, Hengtao Wu, Zhanxun Liu, Wenhao Guan, Junxi Liu, Ziyang Ma, Kelu Xu, and Xie Chen. Audio-oscar: A multi-agent system for complex audio scene generation, orchestration, and refinement. arXiv preprint arXiv:2606.07397, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    Gemini 3.1 pro model card

    Google DeepMind. Gemini 3.1 pro model card. Model card, 2026. URLhttps://deepmind.google/models/ model-cards/gemini-3-1-pro/

    2026

  10. [10]

    Seed1.5-vl technical report.arXiv preprint arXiv:2505.07062, 2025

    Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1.5-vl technical report.arXiv preprint arXiv:2505.07062, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Uni-opd: Unifying on-policy distillation with a dual-perspective recipe.arXiv preprint arXiv:2605.03677, 2026

    Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, et al. Uni-opd: Unifying on-policy distillation with a dual-perspective recipe.arXiv preprint arXiv:2605.03677, 2026

    Pith/arXiv arXiv 2026

  12. [12]

    Visualwebarena: Evaluating multimodal agents on realistic visual web tasks

    Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 881–905, 2024

    2024

  13. [13]

    Mobileworld: Benchmarking autonomous mobile agents in agent-user interactive and mcp-augmented environments.arXiv preprint arXiv:2512.19432, 2025

    Quyu Kong, Xu Zhang, Zhenyu Yang, Nolan Gao, Chen Liu, Panrong Tong, Chenglin Cai, Hanzhang Zhou, Jianan Zhang, Liangyu Chen, et al. Mobileworld: Benchmarking autonomous mobile agents in agent-user interactive and mcp-augmented environments.arXiv preprint arXiv:2512.19432, 2025

    arXiv 2025

  14. [14]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pages 611–626, 2023

    2023

  15. [15]

    Computerrl: Scaling end-to-end online reinforcement learning for computer use agents.arXiv preprint arXiv:2508.14040, 2025

    Hanyu Lai, Xiao Liu, Yanxiao Zhao, Han Xu, Hanchen Zhang, Bohao Jing, Yanyu Ren, Shuntian Yao, Yuxiao Dong, and Jie Tang. Computerrl: Scaling end-to-end online reinforcement learning for computer use agents.arXiv preprint arXiv:2508.14040, 2025

    arXiv 2025

  16. [16]

    Screenspot-pro: Gui grounding for professional high-resolution computer use

    Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. Screenspot-pro: Gui grounding for professional high-resolution computer use. InProceedings of the 33rd ACM International Conference on Multimedia, pages 8778–8786, 2025. 12

    2025

  17. [17]

    On the effects of data scale on ui control agents.Advances in Neural Information Processing Systems, 37: 92130–92154, 2024

    Wei Li, William Bishop, Alice Li, Chris Rawles, Folawiyo Campbell-Ajala, Divya Tyamagundlu, and Oriana Riva. On the effects of data scale on ui control agents.Advances in Neural Information Processing Systems, 37: 92130–92154, 2024

    2024

  18. [18]

    From verbatim to gist: Distilling pyramidal multimodal memory via semantic information bottleneck for long-horizon video agents.arXiv preprint arXiv:2603.01455, 2026

    Niu Lian, Yuting Wang, Hanshu Yao, Jinpeng Wang, Bin Chen, Yaowei Wang, Min Zhang, and Shu-Tao Xia. From verbatim to gist: Distilling pyramidal multimodal memory via semantic information bottleneck for long-horizon video agents.arXiv preprint arXiv:2603.01455, 2026

    Pith/arXiv arXiv 2026

  19. [19]

    Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning

    Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Pengxiang Zhao, Guangyi Liu, et al. Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 17608–17616, 2026

    2026

  20. [20]

    Self-distilled agentic reinforcement learning.arXiv preprint arXiv:2605.15155, 2026

    Zhengxi Lu, Zhiyuan Yao, Zhuowen Han, Zi-Han Wang, Jinyang Wu, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, et al. Self-distilled agentic reinforcement learning.arXiv preprint arXiv:2605.15155, 2026

    Pith/arXiv arXiv 2026

  21. [21]

    Computer-using agent

    OpenAI. Computer-using agent. https://openai.com/index/computer-using-agent/, January 2025. Accessed: 2026-07-02

    2025

  22. [22]

    Ui-tars: Pioneering automated gui interaction with native agents, 2025.URL https://arxiv

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents, 2025.URL https://arxiv. org/abs/2501.12326, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Androidworld: A dynamic benchmarking environment for autonomous agents

    Chris Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, et al. Androidworld: A dynamic benchmarking environment for autonomous agents. InInternational Conference on Learning Representations, volume 2025, pages 406–441, 2025

    2025

  24. [24]

    Self-distillation enables continual learning

    Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal. Self-distillation enables continual learning. arXiv preprint arXiv:2601.19897, 2026

    Pith/arXiv arXiv 2026

  25. [25]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. InProceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297, 2025

    2025

  26. [26]

    Megatron-lm: Training multi-billion parameter language models using model parallelism.arXiv preprint arXiv:1909.08053, 2019

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism.arXiv preprint arXiv:1909.08053, 2019

    Pith/arXiv arXiv 1909

  27. [27]

    Scaling laws for optimal data mixtures.Advances in Neural Information Processing Systems, 38:129554–129579, 2026

    Mustafa Shukor, Louis Bethune, Dan Busbridge, David Grangier, Enrico Fini, Alaaeldin El-Nouby, and Pierre Ablin. Scaling laws for optimal data mixtures.Advances in Neural Information Processing Systems, 38:129554–129579, 2026

    2026

  28. [28]

    Clawgui: A unified framework for training, evaluating, and deploying gui agents.arXiv preprint arXiv:2604.11784, 2026

    Fei Tang, Zhiqiong Lu, Boxuan Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen. Clawgui: A unified framework for training, evaluating, and deploying gui agents.arXiv preprint arXiv:2604.11784, 2026

    Pith/arXiv arXiv 2026

  29. [29]

    Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534, 2025

    Kimi Team, Yifan Bai, Yiping Bao, Y Charles, Cheng Chen, Guanduo Chen, Haiting Chen, Huarong Chen, Jiahao Chen, Ningxin Chen, et al. Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Ui-venus-1.5 technical report.arXiv preprint arXiv:2602.09082, 2026

    Venus Team, Changlong Gao, Zhangxuan Gu, Yulin Liu, Xinyu Qiu, Shuheng Shen, Yue Wen, Tianyu Xia, Zhenyu Xu, Zhengwen Zeng, et al. Ui-venus-1.5 technical report.arXiv preprint arXiv:2602.09082, 2026

    arXiv 2026

  31. [31]

    Consensus-driven multi-agent cognitive reasoning for enhancing the emotional intelligence of large language models

    Geng Tu, Dingming Li, Jun Huang, and Ruifeng Xu. Consensus-driven multi-agent cognitive reasoning for enhancing the emotional intelligence of large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 17751–17759, 2026

    2026

  32. [32]

    Ui-tars-2 technical report: Advancing gui agent with multi-turn reinforcement learning.arXiv preprint arXiv:2509.02544, 2025

    Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, Junjie Fang, Junting Lu, Longxiang Liu, Qinyu Luo, Shihao Liang, Shijue Huang, et al. Ui-tars-2 technical report: Advancing gui agent with multi-turn reinforcement learning.arXiv preprint arXiv:2509.02544, 2025

    Pith/arXiv arXiv 2025

  33. [33]

    Opencua: Open foundations for computer-use agents.Advances in Neural Information Processing Systems, 38:139756–139806, 2026

    Xinyuan Wang, Bowen Wang, Dunjie Lu, Junlin Yang, Tianbao Xie, Junli Wang, Jiaqi Deng, Xiaole Guo, Yiheng Xu, Chen Wu, et al. Opencua: Open foundations for computer-use agents.Advances in Neural Information Processing Systems, 38:139756–139806, 2026

    2026

  34. [34]

    Milestone-guided policy learning for long-horizon language agents.arXiv preprint arXiv:2605.06078, 2026

    Zixuan Wang, Yuchen Yan, Hongxing Li, Teng Pan, Dingming Li, Ruiqing Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen. Milestone-guided policy learning for long-horizon language agents.arXiv preprint arXiv:2605.06078, 2026. 13

    Pith/arXiv arXiv 2026

  35. [35]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. InInternational conference on machine learning, pages 23965–23998. P...

    2022

  36. [36]

    Os-atlas: Foundation action model for generalist gui agents

    Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, et al. Os-atlas: Foundation action model for generalist gui agents. InInternational Conference on Learning Representations, volume 2025, pages 5090–5108, 2025

    2025

  37. [37]

    Mimo-v2-flash technical report.arXiv preprint arXiv:2601.02780, 2026

    Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, Bowen Shen, Chen Zhang, Chenhong He, Chiheng Lou, Fuli Luo, Gang Wang, et al. Mimo-v2-flash technical report.arXiv preprint arXiv:2601.02780, 2026

    Pith/arXiv arXiv 2026

  38. [38]

    Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments.Advances in Neural Information Processing Systems, 37:52040–52094, 2024

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, et al. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments.Advances in Neural Information Processing Systems, 37:52040–52094, 2024

    2024

  39. [39]

    Scaling computer-use grounding via user interface decomposition and synthesis

    Tianbao Xie, Jiaqi Deng, Xiaochuan Li, Junlin Yang, Haoyuan Wu, Jixuan Chen, Wenjing Hu, Xinyuan Wang, Yuhui Xu, Zekun Wang, et al. Scaling computer-use grounding via user interface decomposition and synthesis. Advances in Neural Information Processing Systems, 38, 2026

    2026

  40. [40]

    Deepseek-v4: Towards highly efficient million-token context intelligence.arXiv preprint arXiv:2606.19348, 2026

    Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chenchen Ling, et al. Deepseek-v4: Towards highly efficient million-token context intelligence.arXiv preprint arXiv:2606.19348, 2026

    arXiv 2026

  41. [41]

    Mobile-agent-v3

    Haiyang Xu, Xi Zhang, Haowei Liu, Junyang Wang, Zhaozai Zhu, Shengjie Zhou, Xuhao Hu, Feiyu Gao, Junjie Cao, Zihua Wang, et al. Mobile-agent-v3. 5: Multi-platform fundamental gui agents.arXiv preprint arXiv:2602.16855, 2026

    arXiv 2026

  42. [42]

    Evocua: Evolving computer use agents via learning from scalable synthetic experience.arXiv preprint arXiv:2601.15876, 2026

    Taofeng Xue, Chong Peng, Mianqiu Huang, Linsen Guo, Tiancheng Han, Haozhe Wang, Jianing Wang, Xiaocheng Zhang, Xin Yang, Dengchang Zhao, et al. Evocua: Evolving computer use agents via learning from scalable synthetic experience.arXiv preprint arXiv:2601.15876, 2026. URLhttps://arxiv.org/abs/2601.15876

    arXiv 2026

  43. [43]

    Ties-merging: Resolving interference when merging models.Advances in neural information processing systems, 36:7093–7115, 2023

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models.Advances in neural information processing systems, 36:7093–7115, 2023

    2023

  44. [44]

    Step-gui technical report.arXiv preprint arXiv:2512.15431, 2025

    Haolong Yan, Jia Wang, Xin Huang, Yeqing Shen, Ziyang Meng, Zhimin Fan, Kaijun Tan, Jin Gao, Lieyu Shi, Mi Yang, et al. Step-gui technical report.arXiv preprint arXiv:2512.15431, 2025

    arXiv 2025

  45. [45]

    macosworld: A multilingual interactive benchmark for gui agents

    Pei Yang, Hai Ci, and Mike Zheng Shou. macosworld: A multilingual interactive benchmark for gui agents. Advances in Neural Information Processing Systems, 38:134014–134056, 2026

    2026

  46. [46]

    Learning beyond teacher: Generalized on-policy distillation with reward extrapolation.arXiv preprint arXiv:2602.12125, 2026

    Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiyong Yang, and Yankai Lin. Learning beyond teacher: Generalized on-policy distillation with reward extrapolation.arXiv preprint arXiv:2602.12125, 2026

    Pith/arXiv arXiv 2026

  47. [47]

    Nemotron-cascade 2: Post-training llms with cascade rl and multi-domain on-policy distillation.arXiv preprint arXiv:2603.19220, 2026

    Zhuolin Yang, Zihan Liu, Yang Chen, Wenliang Dai, Boxin Wang, Sheng-Chieh Lin, Chankyu Lee, Yangyi Chen, Dongfu Jiang, Jiafan He, et al. Nemotron-cascade 2: Post-training llms with cascade rl and multi-domain on-policy distillation.arXiv preprint arXiv:2603.19220, 2026

    arXiv 2026

  48. [48]

    Mobile-agent-v3: Fundamental agents for gui automation, 2025.URL https://arxiv

    Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, Junyang Wang, Zhaoqing Zhu, Ziwei Zheng, Feiyu Gao, Junjie Cao, Zhengxi Lu, et al. Mobile-agent-v3: Fundamental agents for gui automation, 2025.URL https://arxiv. org/abs/2508.15144, 4:21–27, 2025

    Pith/arXiv arXiv 2025

  49. [49]

    On-policy context distillation for language models

    Tianzhu Ye, Li Dong, Xun Wu, Shaohan Huang, and Furu Wei. On-policy context distillation for language models. arXiv preprint arXiv:2602.12275, 2026

    Pith/arXiv arXiv 2026

  50. [50]

    Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763, 2026

    Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763, 2026

    Pith/arXiv arXiv 2026

  51. [51]

    Self-distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734, 2026

    Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734, 2026

    Pith/arXiv arXiv 2026

  52. [52]

    Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems, 37:62557–62583, 2024

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody H Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems, 37:62557–62583, 2024. 14

    2024

  53. [53]

    price.docx\

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. In International Conference on Learning Representations, volume 2024, pages 15585–15606, 2024. 15 Appendix A Dataset Construction and Composition Androi...

    2024

  54. [54]

    Action: a short imperative describing what to do in the UI

  55. [55]

    name": <function-name>,

    A single <tool_call>...</tool_call> block containing only the JSON: {"name": <function-name>, "arguments": <args-json-object>}. Rules: - Output exactly in the order: Action, <tool_call>. - Be brief: one sentence for Action. - Do not output anything else outside those parts. - If finishing, use action=terminate in the tool call. 22 Mobile System Prompt (Qw...

  56. [56]

    Thought: one concise sentence explaining the next move

  57. [57]

    Action: a short imperative describing what to do

  58. [58]

    name": <function-name>,

    A single <tool_call>...</tool_call> block containing only the JSON: {"name": <function-name>, "arguments": <args-json-object>}. Rules: - Output exactly in the order: Thought, Action, <tool_call>. - Be brief: one sentence for Thought, one sentence for Action. - Do not output anything else outside those three parts. - If finishing, use mobile_use with actio...

  59. [59]

    - Follow the user instruction strictly, e.g., only return a single number, only return True or False, or only return items separated by comma

    Communication Rule: - Always use the answer action to reply to users. - Follow the user instruction strictly, e.g., only return a single number, only return True or False, or only return items separated by comma. - Never use answer to indicate waiting or loading; use wait instead. - The answer action terminates the task immediately

  60. [60]

    - If an action fails twice, try alternatives, e.g., long_press instead of click

    Efficiency First: - Choose the simplest path to complete tasks. - If an action fails twice, try alternatives, e.g., long_press instead of click

  61. [61]

    - For scrolling, scroll direction is inverse to swipe direction

    Smart Navigation: - Gather information when needed. - For scrolling, scroll direction is inverse to swipe direction. - If scroll fails, try the opposite direction

  62. [62]

    - For text manipulation, long press to select, use selection bar options, and delete by selecting then cutting

    Text Operations: - First click the input box to activate it before typing. - For text manipulation, long press to select, use selection bar options, and delete by selecting then cutting

  63. [63]

    # Decision Process

    Ask User: - If there is not enough information to complete the task, use ask_user. # Decision Process

  64. [64]

    Analyze goal, history, and current screen

  65. [65]

    Determine if the task is already complete, and use status if true

  66. [66]

    If not, choose the most appropriate action

  67. [67]

    The action must be a valid JSON string

    Output in the exact format below. The action must be a valid JSON string

  68. [68]

    action_type

    Only one tool call is allowed in one action. # Expected Output Format Thought: [Analysis including reference to key steps or points when applicable] Action: [Single JSON action] # Output Format Example Thought: I need to type the weather question into the search box. Action: {"action_type": "input_text", "text": "What is weather like in San Francisco today?"} 25