AndroidGen: Building an Android Language Agent under Data Scarcity

Hanyu Lai; Jie Tang; Junjie Gao; Shudan Zhang; Xiao Liu; Yifan Xu; Yuxiao Dong

REVIEW 3 major objections 5 minor 52 references

AndroidGen: Building an Android Language Agent under Data Scarcity

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AndroidGen reaches 46.8% average success on AndroidWorld by retrieving its own past trajectories as in-context examples.

desk verdict AndroidGen is a competent engineering pipeline for data-scarce Android agents, but its headline AndroidWorld numbers rest on a retrieval database that likely overlaps the test tasks, so the central comparison is not yet trustworthy. read the letter →

arxiv 2504.19298 v1 pith:U33QUOSW submitted 2025-04-27 cs.CL

Hanyu Lai , Junjie Gao , Xiao Liu , Yifan Xu , Shudan Zhang , Yuxiao Dong , Jie Tang This is my paper · ORCID

classification cs.CL

keywords Androidagentslanguagedatascarcityin-contextlearningtrajectoryretrievalmobileUIautomationsyntheticgenerationStepCritic

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a strong Android agent can be built without large-scale human-annotated trajectory data. Its AndroidGen framework combines four modules: ExpSearch retrieves a similar completed trajectory from a self-built database and injects it as a one-shot example; ReflectPlan initializes and revises step-by-step plans; AutoCheck verifies each operation before execution; and StepCritic decomposes tasks into sub-goals and scores each step. With GPT-4o, the framework reaches 46.8% average success on AndroidWorld under a standardized text-only action space, against 27.7% for M3A and 15.9% for SeeAct. The same pipeline also generates more than 1,000 filtered trajectories, and fine-tuning Llama-3-70B on them yields 35.3% on AndroidWorld and 74.0%/79.2% on AitW general/web splits, above the cited DigiRL agent. If correct, this means data scarcity is not a hard ceiling: one capable closed model plus automatic quality control can bootstrap both a strong agent and an open-source mobile agent.

What carries the argument

The load-bearing mechanism is the ExpSearch retrieval loop: a trajectory database is populated by self-sampled, StepCritic-assessed episodes, and a dense retriever (Contriever) selects the top-1 similar trajectory for each new instruction, which then serves as a one-shot example in the model prompt. This is what the ablation credits with the largest gains: adding ExpSearch to ReflectPlan and AutoCheck raises GPT-4o from 20.7% to 46.8% on AndroidWorld, with hard tasks going from 0.0% to 11.8%. StepCritic is the supporting evaluator that makes the loop data-honest: it decomposes each task into sub-goals, maps each sub-goal to a completion step or -1, and its fine-grained judgments are used both to filter training data and to update the database.

What would settle it

Open the released trajectory database and embed every AndroidWorld test instruction with the same retriever; if any test task or near-duplicate appears in the top-1 results under the same app context, the 46.8% success rate and the ExpSearch ablation gain partly measure retrieval of stored answers. Rerunning AndroidWorld with the database cleared of all AndroidWorld-derived instructions would settle it.

Watch

Extended reading notes

Core claim

The central claim, stated in the paper's own terms, is that retrieval-augmented self-data construction turns a language model into a competitive Android agent under data scarcity. AndroidGen operates by having the model sample trajectories for roughly 300 GPT-4o-generated tasks drawn from AndroidWorld instructions, using StepCritic to evaluate whether each sub-goal is completed, and storing the assessed trajectories in a database. At each episode, ExpSearch uses Contriever to encode the current instruction and retrieve the top-1 similar trajectory, which is placed in the prompt; ReflectPlan maintains and updates a plan based on the current environment; AutoCheck rejects operations whose target element or state is not present; and StepCritic provides fine-grained labels. The paper reports that this reaches 46.8% on AndroidWorld with GPT-4o, and that the same trajectories, filtered and augmented by StepCritic, let a LoRA-tuned Llama-3-70B reach 35.3% on AndroidWorld and outperform the DigiRL agent on AitW. The intended upshot is that expert-level trajectory data can be manufactured rather than annotated, and that closed and open models both benefit from it.

Load-bearing premise

The headline AndroidWorld numbers assume that the retrieval database consulted during evaluation is disjoint from the test tasks, since the database is built from GPT-4o-generated instructions that draw on the same AndroidWorld task set.

Editorial extensions

If this is right

On AndroidWorld, an untrained GPT-4o agent with the four AndroidGen modules more than doubles the M3A baseline (46.8% vs 27.7%) under the same text-based action space and accessibility-tree input.
The generated trajectories can substitute for human annotation: fine-tuning Llama-3-70B on StepCritic-filtered synthetic data reaches 35.3% on AndroidWorld, close to the 46.8% of the untrained GPT-4o system.
The same data pipeline transfers across benchmarks: the fine-tuned Llama-3-70B scores 74.0% on AitW General and 79.2% on AitW Web, above the DigiRL reinforcement-learning agent's 71.9% and 67.2%.
StepCritic selection outperforms no selection (25.2% vs 16.2% on AndroidWorld with Llama-3-70B) and approaches oracle-based selection (27.0%), showing fine-grained automatic evaluation is a workable quality filter.
Quality-controlled synthetic data construction is substantially cheaper than human annotation, at roughly 5% of the cost and 5.85 times the efficiency for 1,000 trajectories.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the retrieval database is truly disjoint from the test tasks, the +12.6-point ExpSearch ablation gain suggests top-1 trajectory retrieval transfers to unseen tasks, which would make the method a candidate for other GUI environments such as iOS or desktop web without retraining.
StepCritic's sub-goal decomposition could be reused as a dense reward or curriculum signal for reinforcement learning, not just as a filter; the paper itself only uses it for selection, augmentation, and database updates.
The error analysis points to a concrete next fix: memorization failures (20%) and math-counting failures (23%) suggest that an explicit external memory or a calculator-like tool inside the action space could extend the framework beyond what the current prompts achieve.
Because the 300 task instructions are drawn from AndroidWorld itself, the cleanest test of the contribution is whether the same pipeline can bootstrap a database from a disjoint task source and still produce comparable gains.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: AndroidGen is a serious engineering effort, and the open-source training pipeline is genuinely useful. But the headline AndroidWorld claim is not yet supported, because the ExpSearch retrieval database may contain near-duplicates of the test tasks.

Here is the load-bearing problem. Section 4.1 builds the trajectory database from about 300 GPT-4o-generated instructions that 'draw on' AndroidWorld, and Section 3.2.1 retrieves the top-1 similar trajectory for every episode. AndroidWorld tasks are parameterized templates — playlist names, song titles, contact names — so an instruction that draws on the same template with different parameters is effectively a stored solution. The paper never says the test tasks or their near-duplicates are excluded from retrieval. The only leakage disclaimer is about reward signals and golden labels in training, which does not address retrieval-time contamination. If the database contains a trajectory for 'Create a playlist in Retro Music titled X with songs Y', then the agent facing '... title Z ... songs W' is being handed the solution path. That would explain both the 46.8% AndroidWorld average and the +12.6 ExpSearch ablation gain, and it makes the comparison to M3A and SeeAct, which have no such database, unfair. This is not a minor caveat; it is the difference between an agent that generalizes and an agent that copies.

What is actually new and good: AndroidGen integrates retrieval, reflection, rule-based action checking, and LLM-based step evaluation into a coherent pipeline; the authors fine-tune open models without human-labeled trajectories; they release code, model, and data; the ablations are internally consistent; and the cost/efficiency analysis is useful. The AitW results for Llama-3-70B* are promising if the DigiRL protocol was followed faithfully, though the paper should state exactly how its 96-task sample overlaps DigiRL's test set. The StepCritic accuracy numbers in Table 5 are weakened by using the same pipeline's own trajectories as test data; that is an internal consistency check, not an independent validation.

Verdict: this deserves peer review, because the engineering contribution is real and the leakage question is exactly the kind of thing a referee can force the authors to answer. The paper should not be accepted until the authors either show the retrieval database is disjoint from AndroidWorld test tasks (with overlap statistics and examples) or rerun the experiments without contaminated retrieval. If the numbers survive that check, the result is interesting. If not, the central claim collapses to a smaller, still useful claim about data generation.

Referee Report

3 major / 5 minor

Summary. The paper introduces AndroidGen, a framework for building Android language agents under data scarcity. It combines four modules: ExpSearch, which retrieves similar trajectories from a database for in-context learning; ReflectPlan, which updates a task plan based on environment observations; AutoCheck, which verifies each generated action before execution; and StepCritic, which evaluates trajectories at sub-goal granularity. The authors use AndroidGen both as a zero-shot agent powered by GPT-4o and as a data-generation pipeline to fine-tune open-source models (GLM-4-9B and Llama-3-70B) without human-labeled trajectories. They report state-of-the-art results on AndroidWorld (46.8% average success for GPT-4o vs. 27.7% for M3A), strong AitW results, and improvements from the ExpSearch module in ablations. The paper also presents StepCritic as a more accurate evaluator than existing baselines and includes efficiency/cost comparisons with human annotation.

Significance. If the experimental results hold, this is a significant contribution: it demonstrates a fully synthetic data-generation pipeline for mobile agents, shows that retrieval-augmented self-reflection can substantially improve task completion, and releases code, models, and data. The framework addresses a real bottleneck (high-quality labeled trajectory scarcity) and its training-data pipeline for open-source models without human annotation is a valuable step toward reproducible mobile agents. However, the headline AndroidWorld result and the quantified benefit of ExpSearch depend critically on the integrity of the retrieval database, and the current manuscript does not provide sufficient evidence that the database is disjoint from the test tasks. The self-referential evaluation of StepCritic is a secondary but real concern. These issues are load-bearing for the central claims, so the paper needs substantive revision before the results can be accepted at face value.

major comments (3)

[Section 4.1 / Section 3.2.1 / Tables 2 and 6] The retrieval database used by ExpSearch is constructed from "about 300 task instructions drawing on the instructions in AndroidWorld" (Section 4.1), and AndroidWorld is exactly the benchmark reported in Table 2. The paper does not state that the AndroidWorld test tasks, or near-duplicates of them, were excluded from this database. Since Section 3.2.1 selects the top-1 similar trajectory for every episode, a database entry that paraphrases the test instruction (e.g., with different app-specific values) would provide a near-complete solution rather than a general demonstration. This could directly explain the +12.6 ExpSearch ablation gain in Table 6 and the 46.8% headline number. The sentence "We ensure no reward signals or golden labels are employed during training" addresses training labels only and does not rule out retrieval-time contamination. Please state explicitly whether the database is disjoint from the test tasks, describe any deduplication or near-duplicate removal, and, if necessary, re-run the AndroidWorld experiments with a database that provably excludes all test tasks and their paraphrases.
[Section 3.2.1 / Section 4.1] Section 3.2.1 states: "each time the agent completes a task, we use StepCritic to assess the trajectory and log it to the database, which enables our agent to self-improve iteratively." If this logging occurs during the evaluation episodes on AndroidWorld itself, the database would accumulate trajectories taken on the test tasks, and later episodes could retrieve solutions to earlier test tasks. The manuscript does not specify whether the database used in the reported experiments is a fixed pre-collected snapshot or whether it grows online during evaluation. This ambiguity is critical for interpreting both Table 2 and the ExpSearch ablation in Table 6. Please clarify the exact protocol: was the database frozen before evaluation, and are any trajectories collected during evaluation excluded from retrieval?
[Section 5.4 / Table 5] The StepCritic evaluator accuracy comparison uses "trajectories generated by ANDROIDGEN as the test set," but these trajectories were themselves produced and filtered by the same pipeline whose quality is being assessed, and StepCritic is itself GPT-4o. This self-referential setup can inflate the reported accuracy relative to independent baselines. Furthermore, the manuscript does not describe what the "environmental oracle prediction" is, nor how the manual sub-goal and step-label accuracy in Table 5 was obtained. An independent human-annotated evaluation set, or at least a set generated by a different pipeline, is needed to support the claim that StepCritic outperforms the Captioner-based baselines. Please provide details of the oracle construction and the manual annotation protocol, or revise the evaluation to avoid this circularity.

minor comments (5)

[Section 2 (Benchmarks of Mobile Agents)] The text cites "AITW (Xing et al., 2024)" but the reference list contains Rawles et al. (2023) for Android in the Wild; the citation appears to be conflated with the AndroidArena citation (Xing et al., 2024). Please correct the citation.
[Figure 2 caption] The caption ends with "subscribe to the channel.." (double period). Minor typographical error.
[Section 4.1] The description "about 300 task instructions" is imprecise. Please give the exact number of generated instructions, the number of successful trajectories, and the final size of the trajectory database, as these numbers are important for assessing the scope of the retrieval pool.
[Table 5] The column "Completion Step" is not defined. Please clarify what this metric measures (e.g., whether the model's predicted step index for each sub-goal must exactly match the human-annotated index) and how it is computed.
[Table 6 / Section 5.5] The "Oracle-Selection" row is not explained in the text. Please specify what oracle labels are used (presumably environment feedback) and how they differ from StepCritic labels.

Circularity Check

1 steps flagged · score 5.0 of 10

ExpSearch's retrieval database is built from AndroidWorld-derived task instructions, so the AndroidWorld success and ExpSearch ablation gain may reflect retrieving stored near-duplicate trajectories rather than agent generalization.

fitted input called prediction [Section 4.1 (Data Collection), Section 3.2.1 (ExpSearch), evaluated in Section 5.1 and Table 6]
"Task Formulation. We utilize GPT-4o to generate about 300 task instructions drawing on the instructions in AndroidWorld. We ensure no reward signals or golden labels are employed during training to prevent data leakage. ... Trajectory Retrieval ... compute similarity scores with embeddings from the database. The top-1 result is selected as our learning example."

The ExpSearch database is populated from instructions generated from AndroidWorld, and the benchmark being predicted is AndroidWorld itself. During evaluation, ExpSearch retrieves the top-1 most similar stored trajectory for every AndroidWorld episode; the paper only excludes reward signals and golden labels in training and never states that AndroidWorld test tasks or near-duplicates are removed from the retrieval database. Hence the +12.6 ExpSearch gain (Table 6, 34.2 to 46.8) and the 46.8% AndroidWorld score can be produced by copying a stored solution for a paraphrase of the test task, making the prediction dependent on the retrieval input rather than on agent capability.

full rationale

The central AndroidWorld result is partly compromised by the construction of ExpSearch's database from AndroidWorld instructions without documented test-task exclusion. However, the paper also evaluates on AitW (96 tasks from DigiRL) and popular apps, where the retrieval database is not derived from the evaluation tasks; those results provide independent evidence for AndroidGen. The StepCritic evaluator comparison is self-referential in test-set construction but is checked against an environmental oracle, so it is not a clear circularity. Overall, the strongest claim (AndroidWorld + ExpSearch) is vulnerable by construction, but the framework retains independent external support, so partial circularity (score 5) is appropriate.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper is empirical engineering; it introduces no fitted constants in a derivation. The free parameters listed are hand-chosen design choices that affect the central results. The axioms capture the reliance on GPT-4o for generation and evaluation, on Contriever for retrieval, and on benchmark protocols. The most critical assumption is the claimed disjointness of the ExpSearch retrieval database from the test benchmarks.

free parameters (2)

ExpSearch top-k retrieval count (k) = 1
Chosen by hand without sensitivity analysis. The largest reported ablation gain comes from ExpSearch, so the retrieval top-k choice is consequential for the main result.
Number of generated task instructions = ~300
Chosen by hand in Section 4.1. This determines the size of the retrieval database and the training set, and no scale analysis is provided.

assumptions (5)

domain assumption GPT-4o can generate diverse, realistic Android task instructions and execute them reliably enough to produce useful trajectories.
Section 4.1: both task formulation and agent sampling use GPT-4o. If the generator is biased or low-quality, the training data and retrieval database inherit that bias.
domain assumption Contriever embeddings provide similarity rankings that select genuinely helpful examples.
Section 3.2.1: ExpSearch picks the top-1 trajectory by embedding similarity. The paper does not analyze retrieval quality or failure cases.
domain assumption StepCritic's sub-goal completion judgments are accurate enough for data selection and augmentation.
Section 4.1 and Table 5: StepCritic reaches 87.9% overall accuracy on trajectories generated by the same pipeline. Data selection and augmentation rely fully on these labels.
domain assumption AndroidWorld and AitW benchmarks provide an unbiased measure of agent capability, and the selected 96 AitW tasks are comparable to DigiRL's reported test set.
Section 5.2: the random subset is selected without a reported seed, and human expert judging may not match the automatic metrics used for the cited DigiRL baselines.
domain assumption The trajectory database used at evaluation time does not overlap with the test tasks.
This is the most load-bearing assumption. The database is built from tasks drawn from AndroidWorld instructions (Section 4.1), and the paper never states that test tasks or near-duplicates are excluded from ExpSearch retrieval during evaluation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of AndroidGen: Building an Android Language Agent under Data Scarcity." pith.science (2026). https://pith.science/paper/U33QUOSW

@misc{pith2026250419298,
  author       = {Pith},
  title        = {Pith review of: AndroidGen: Building an Android Language Agent under Data Scarcity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U33QUOSW}},
  note         = {Machine review of arXiv:2504.19298}
}

read the original abstract

Large language models have opened up a world of possibilities for various NLP tasks, sparking optimism for the future. Despite their potential, LLMs have yet to be widely used as agents on real mobile devices. The main challenge is the need for high-quality data sources. Time constraints and labor intensity often hinder human annotation. On the other hand, existing LLMs exhibit inadequate completion rates and need a robust data filtration strategy. Given these challenges, we develop a framework called AndroidGen to enhance the capabilities of LLM-based agents under data scarcity. In addition, we leverage AndroidGen to collect trajectories given human tasks and train open-source LLMs on these trajectories to develop an open-source mobile agent without manually labeled trajectories. We extensively evaluate AndroidGen with AndroidWorld, AitW, and various popular applications, demonstrating its improvements and revealing potential areas for future improvement. Code, model, and data are available at https://github.com/THUDM/AndroidGen.

Figures

Figures reproduced from arXiv: 2504.19298 by the authors.

**Figure 2.** Examples of ANDROIDGEN’s execution on four user tasks. challenging and time-consuming, thereby further augmenting overall expenditures. Currently, prevalent methods of manual and automated data collection (Wang et al., 2022; Honovich et al., 2022; Peng et al., 2023; Mukherjee et al., 2023) face significant challenges. Manual annotation requires considerable time and financial resources, making collecting large vo… view at source ↗

**Figure 3.** Overview of ANDROIDGEN framework designed to complete tasks in Android. Our process comprises three stages: preliminary, task execution, and update. Preliminary (a): ExpSearch retrieve the top-1 similar tasks and trajectories from the database and feed them into the agent. Task Execution (b): ReflectPlan assesses the progress and updates the plan. Then, the agent generates operations based on the environment, plan, … view at source ↗

Figures from the paper (12 more)

**Figure 4.** Figure 4: ANDROIDGEN data construction workflow. In addition to the environment input, we integrate the information from each module in the architecture to facilitate the agent’s operations. For detailed prompt organization, refer to Appendix F. 4 Building An Open-Source Andro…

**Figure 5.** Figure 5: Audio Recorder [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

**Figure 6.** Figure 6: Phone Contact 17 [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

**Figure 7.** Figure 7: Delete File [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]

**Figure 8.** Figure 8: Marking Map 18 [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

**Figure 9.** Figure 9: Create Music Playlist [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

**Figure 10.** Figure 10: Turn Bluetooth Off 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

**Figure 11.** Figure 11: Play Music Playlist [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

**Figure 12.** Figure 12: Join a community 20 [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

**Figure 13.** Figure 13: Math Counting Error [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]

**Figure 14.** Figure 14: Memorization Error 21 [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]

**Figure 15.** Figure 15: Vision Error 22 [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 2 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023
[4]

Anthropic. 2023. Model card and evaluations for claude models

work page 2023
[5]

Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, Victor C a rbune, Jason Lin, Jindong Chen, and Abhanshu Sharma. 2024. Screenai: A vision-language model for ui and infographics understanding. arXiv preprint arXiv:2402.04615

arXiv 2024
[6]

Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, and Aviral Kumar. 2024. Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning. arXiv preprint arXiv:2406.11896

arXiv 2024
[7]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

2020
[8]

Wei Chen and Zhiyuan Li. 2024. Octopus v2: On-device language model for super agent. arXiv preprint arXiv:2404.01744

arXiv 2024

Show all 52 references

[9]

Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024. Seeclick: Harnessing gui grounding for advanced visual gui agents. arXiv preprint arXiv:2401.10935

2024 arXiv
[10]

Kahneman Daniel. 2017. Thinking, fast and slow

2017
[11]

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long...

2022
[12]

Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793

2024 arXiv
[13]

Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. 2023. Cogagent: A visual language model for gui agents. arXiv preprint arXiv:2312.08914

2023 arXiv
[14]

Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022. Unnatural instructions: Tuning language models with (almost) no human labor. arXiv preprint arXiv:2212.09689

2022 arXiv
[15]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

2021 arXiv
[16]

Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Towards unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118, 2(3)

2021 arXiv
[17]

Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, et al. 2024. Autowebglm: Bootstrap and reinforce a large language model-based web navigating agent. arXiv preprint arXiv:2404.03648

2024 arXiv
[18]

Xiao Liu, Hanyu Lai, Hao Yu, Yifan Xu, Aohan Zeng, Zhengxiao Du, Peng Zhang, Yuxiao Dong, and Jie Tang. 2023 a . Webglm: Towards an efficient web-enhanced question answering system with human preferences. arXiv preprint arXiv:2306.07906

2023 arXiv
[19]

Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. 2023 b . Agentbench: Evaluating llms as agents. arXiv preprint arXiv:2308.03688

2023 arXiv
[20]

Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568

2023 arXiv
[21]

Kai Mei, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, and Yongfeng Zhang. 2024. Llm agent operating system. arXiv preprint arXiv:2403.16971

2024 arXiv
[22]

Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. Generating training data with language models: Towards zero-shot language understanding. Advances in Neural Information Processing Systems, 35:462--477

2022
[23]

Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707

2023 arXiv
[24]

Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332

2021 arXiv
[25]

Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, and Alane Suhr. 2024. Autonomous evaluation and refinement of digital agents. arXiv preprint arXiv:2404.06474

2024 arXiv
[26]

Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277

2023 arXiv
[27]

Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, et al. 2024. Androidworld: A dynamic benchmarking environment for autonomous agents. arXiv preprint arXiv:2405.14573

2024 arXiv
[28]

Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. 2023. https://arxiv.org/abs/2307.10088 Android in the wild: A large-scale dataset for android device control . Preprint, arXiv:2307.10088

2023 arXiv
[29]

Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. https://doi.org/10.18653/v1/2022.naacl-main.191 Learning to retrieve prompts for in-context learning . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: ...

2022 doi
[30]

Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili \'c , Daniel Hesslow, Roman Castagn \'e , Alexandra Sasha Luccioni, Fran c ois Yvon, Matthias Gall \'e , et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2...

2022 arXiv
[31]

Timo Schick and Hinrich Sch \"u tze. 2020. Few-shot text generation with pattern-exploiting training. arXiv preprint arXiv:2012.11926

2020 arXiv
[32]

Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

2023 arXiv
[33]

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023 a . Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

2023 arXiv
[34]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 b . Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

2023 arXiv
[35]

Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024. Mobile-agent: Autonomous multi-modal mobile device agent with visual perception. arXiv preprint arXiv:2401.16158

2024 arXiv
[36]

Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. 2023 a . A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432

2023 arXiv
[37]

Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023 b . https://arxiv.org/abs/2305.04091 Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models . Preprint, arXiv:2305.04091

2023 arXiv
[38]

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations

2022
[39]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824--24837

2022
[40]

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, W...

2023 arXiv
[41]

Yaqi Xie, Chen Yu, Tongyao Zhu, Jinbin Bai, Ze Gong, and Harold Soh. 2023. https://arxiv.org/abs/2302.05128 Translating natural language to planning goals with large-language models . Preprint, arXiv:2302.05128

2023 arXiv
[42]

Mingzhe Xing, Rongkai Zhang, Hui Xue, Qi Chen, Fan Yang, and Zhen Xiao. 2024. Understanding the weakness of large language model agents within a complex android environment. arXiv preprint arXiv:2402.06596

2024 arXiv
[43]

Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244

2023 arXiv
[44]

Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023 a . Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441

2023 arXiv
[45]

Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2023 b . Appagent: Multimodal agents as smartphone users. arXiv preprint arXiv:2312.13771

2023 arXiv
[46]

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations

2022
[47]

Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. 2024. https://doi.org/10.18653/v1/2024.acl-long.670 Agent lumos: Unified and modular training for open-source language agents . In Proceedings of the 62nd Annual Meeti...

2024 doi
[48]

Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. 2022. Glm-130b: An open bilingual pre-trained model. In The Eleventh International Conference on Learning Representations

2022
[49]

Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

2022 arXiv
[50]

Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. 2024. Gpt-4v (ision) is a generalist web agent, if grounded. arXiv preprint arXiv:2401.01614

2024 arXiv
[51]

Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. 2023. Webarena: A realistic web environment for building autonomous agents. In Second Agent Learning in Open-Endedness Workshop

2023
[52]

Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910

2022 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023

[4] [4]

Anthropic. 2023. Model card and evaluations for claude models

work page 2023

[5] [5]

Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, Victor C a rbune, Jason Lin, Jindong Chen, and Abhanshu Sharma. 2024. Screenai: A vision-language model for ui and infographics understanding. arXiv preprint arXiv:2402.04615

arXiv 2024

[6] [6]

Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, and Aviral Kumar. 2024. Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning. arXiv preprint arXiv:2406.11896

arXiv 2024

[7] [7]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

2020

[8] [8]

Wei Chen and Zhiyuan Li. 2024. Octopus v2: On-device language model for super agent. arXiv preprint arXiv:2404.01744

arXiv 2024