Pith. sign in

REVIEW 5 major objections 5 minor 26 references

DPO Learning with LLMs-Judge Signal for Computer Use Agents

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning a 2-billion-parameter vision-language model with DPO on preference pairs from an LLM judge raises its OSWorld 15-step weighted-average success from 6.91 to 9.33.

desk verdict A concrete LLM-as-Judge + DPO pipeline for local CUAs, but the 15-step gains rest on an untested transfer assumption and the only baseline is the authors' own base model. read the letter →

arxiv 2506.03095 v1 pith:XNIEHSF5 submitted 2025-06-03 cs.AI cs.CV

Man Luo , David Cobbley , Xin Su , Shachar Rosenman , Vasudev Lal , Shao-Yen Tseng , Phillip Howard This is my paper
classification cs.AIcs.CV
keywords computeruseagentslocalvision-languagemodelsLLM-as-JudgedirectpreferenceoptimizationGUIinteractionOSWorldbenchmarkAGUVISprivacy-preservingAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a fully automatic way to train a small, local computer-use agent without human-labeled preference data. It samples multiple candidate next actions from a 2-billion-parameter model, has a large vision-language judge score each candidate against the ground-truth action, and turns those scores into preference pairs for Direct Preference Optimization. On the OSWorld benchmark, the best DPO-trained model raises the 15-step weighted-average success rate from 6.91 to 9.33, and the authors report it outperforms the base model even when the base is given 50 steps. The 50-step setting itself shows smaller or negative gains, which the authors attribute to longer-horizon error accumulation and some hardware failures during evaluation.

What carries the argument

The machinery is the LLM-as-Judge scoring loop feeding Direct Preference Optimization: a large language model grades candidate outputs, replacing human annotation. For each AGUVIS task the base model samples ten candidate next actions; GPT-4o scores all ten at once against the instruction and the ground-truth answer; two pairing strategies convert scores into preferred versus rejected pairs; DPO then optimizes the policy to prefer the higher-scored or ground-truth action. The judge prompt, the ground-truth answer shown to the judge, and the two pairing strategies are the components that carry the argument.

What would settle it

Run the same DPO pipeline with the judge's prompt modified to omit the ground-truth answer, then evaluate on OSWorld; if the 15-step improvement disappears or shrinks drastically, the preference signal is mostly the judge copying the reference answer rather than grading candidate quality.

Watch

Extended reading notes

Core claim

The central claim is that next-step action preferences can be manufactured automatically and used to make a lightweight local agent more reliable. Concretely: take AGUVIS tasks, sample 10 candidate next actions from UI-TARS-2B, ask GPT-4o to assign each a 0-100 score in a prompt that includes the task instruction and the ground-truth action, and construct DPO pairs either by matching each sub-80 candidate against ground truth or by matching higher-scoring against lower-scoring candidates. Training on these pairs yields DPO-3, which reaches 9.33 weighted-average success at 15 steps versus 6.91 for the base model, and the authors claim this outperforms existing baselines. The improvement is concentrated in domains such as Chrome, OS utilities, and Thunderbird, while GIMP performance slightly drops. At 50 steps, only modest overall gains remain and the combined-strategy model DPO-3 is below baseline, so the demonstrated advantage is specifically in short interaction budgets.

Load-bearing premise

The training works only if an automated judge's scores for single next-step actions, graded with the correct answer in view, predict which actions truly lead to completing multi-step tasks in a different environment.

Editorial extensions

If this is right

  • A 2B-parameter model fine-tuned this way can run locally and outperform its base model on OSWorld at 15 steps, reducing privacy and latency concerns for GUI automation.
  • The same automatic pipeline can scale preference data for computer-use agents without expensive human annotation.
  • The 15-step gains, up to 35 percent, indicate that DPO fine-tuning makes agents more efficient in short interaction budgets; the 15-step DPO models also beat the 50-step base model.
  • At 50 steps the gains shrink or reverse, so the demonstrated benefit does not yet extend to long-horizon tasks.
  • Failure analysis identifies syntax errors, wrong coordinates, and repeated wrong actions as the next targets for improving robustness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the judge sees the ground-truth answer, an untested possibility is that DPO is learning to imitate the reference action more than to judge candidate quality; removing the ground truth from the judge prompt would separate these effects.
  • The score distribution is heavily concentrated near zero, suggesting most sampled candidates are low quality; a more diverse sampling scheme or a judge that rewards semantic equivalence could strengthen the preference signal.
  • If next-step preference transfer holds, the same automatic pipeline could train local agents for personal-device tasks in domains that lack existing benchmarks.
  • The 50-step drop hints that per-step preferences teach local action choice but not global planning, so adding trajectory-level feedback or search would be a natural extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes an LLM-as-Judge pipeline for generating DPO preference data to fine-tune a 2B-parameter local computer-use agent (UI-TARS-2B). Candidate next-step actions are sampled from the policy on AGUVIS tasks, scored by GPT-4o in the presence of the ground-truth answer, and converted into preference pairs using two pairing strategies (against ground truth for low-scoring responses, and between model responses). The resulting DPO models (DPO-1, DPO-2, DPO-3) are evaluated on OSWorld under 15-step and 50-step action budgets. The paper reports that all DPO variants outperform the base model under the 15-step setting, with weighted-average success rates rising from 6.91 to 9.33 for DPO-3, and that DPO-3 falls below the baseline in the 50-step setting (8.24 vs. 8.83). The main claims are that the approach is privacy-preserving, avoids human annotation, and improves short-horizon task success.

Significance. If the results are robust, the paper would contribute a practical, low-cost recipe for improving small local GUI agents without human preference labels, using an external judge and standard DPO. The pipeline is simple and reproducible in principle, and the choice to evaluate on a different benchmark (OSWorld) than the training data source (AGUVIS) is a reasonable transfer test. However, the significance is currently limited by missing statistical validation, a single comparison baseline, and an unvalidated judge signal. The contribution is incremental relative to existing LLM-as-judge and DPO literature; its value hinges entirely on the demonstrated reliability of the judge and the transfer of next-step preferences to multi-step execution, neither of which is currently established.

major comments (5)
  1. [§4.4, Tables 1–2] The central claim that DPO models 'significantly outperform' the baseline is not backed by any measure of uncertainty. Although the evaluation was run three times, only averaged numbers are reported, with no error bars, confidence intervals, or significance tests. On OSWorld's small domain sizes (e.g., Chrome has only a handful of tasks), the Headline improvement from 6.91 to 9.33 weighted average could plausibly be within run-to-run variance. Please report per-run scores or standard deviations and perform a paired significance test (e.g., Wilcoxon or paired bootstrap) across the 369 tasks for the 15-step setting.
  2. [Abstract and §4.4] The abstract claims the fine-tuned model 'outperforms existing baselines,' but the experiments compare only against the base UI-TARS-2B model. No comparison is made to other published computer-use agents (e.g., AGUVIS, UI-TARS, or prior OSWorld results) under the same evaluation protocol. Either add such baselines or revise the abstract to claim improvement over the base model only.
  3. [§3, Figure 2] The judge's validity is untested. The prompt in Figure 2 includes 'Ground truth answer: {gt}', so GPT-4o scores may reward string-level or surface similarity to a single reference action rather than downstream task progress. No evidence is provided that judge scores correlate with human judgments, with next-step accuracy, or with task completion. This is load-bearing because every preference pair is derived from these scores; without a validation study (e.g., correlation between judge scores and execution outcomes on a held-out set), the training signal's quality is unknown.
  4. [§4.4, Table 2] The 50-step results are the only multi-step transfer probe in the paper, and they undermine the transfer claim: DPO-3 scores 8.24 versus the baseline's 8.83, a reversal of the 15-step trend. The paper attributes part of this to 'hardware failures during inference in some 50-step runs,' but no details are given about which models or domains were affected, or how such failures could be excluded from the averaged numbers. This needs to be addressed concretely: report the affected runs, rerun them under controlled conditions, or present the 50-step results as a limitation rather than an unexplained anomaly.
  5. [§3 (Data Sampling) and §4.4] The transfer from next-step preference learning to full multi-step task execution is a central assumption that is never directly tested. All training pairs are derived from AGUVIS next-step prediction tasks, while OSWorld requires coherent multi-step action sequences. The paper reports only the final OSWorld success rates; it does not show whether DPO improves next-step accuracy on AGUVIS validation, nor does it report intermediate metrics such as task progress or step efficiency. Adding a next-step accuracy evaluation on AGUVIS (or on a subset of OSWorld) would directly test whether the DPO signal improves the intended capability.
minor comments (5)
  1. [§2, Related Work] There is a typo: 'a lightweight vision-language model that can be operate entirely on local hardware' should read 'can operate.'
  2. [§4.3, Figure 3] The figure is referenced as 'LLM-as-judge Score distribution' but the axis labels are not described in the text. Please add explicit axis labels (e.g., 'GPT-4o score' and 'number of responses') to make the distribution interpretable.
  3. [§4.3] There is a typo in the model name: 'the baseline UI-TAR-2B model' should be 'UI-TARS-2B.'
  4. [Tables 1 and 2] The 'Max Diff.' columns are not consistent with the numbers in the tables. For example, in Table 1, Chrome's best DPO result is 12.43 versus baseline 5.07, giving a difference of 7.36, not the reported 10.48; in Table 2, Thunderbird's maximum difference should be negative (−4.44) but is reported as 0.00. Please clarify the definition of 'Max Diff.' or correct the values.
  5. [§4.2 vs. §3] The inference hyperparameters differ between sampling (temperature 0.7, top-k 10, top-p 0.9) and evaluation (temperature 1.0, top-p 0.7). The paper should state whether the evaluation settings were chosen to match the original UI-TARS-2B inference protocol, and justify the discrepancy to ensure the comparison to the base model is fair.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DPO training signal (GPT-4o judge on AGUVIS next-step preferences) and the OSWorld evaluation metric are distinct and externally grounded, with no fitted parameter or self-citation chain forcing the reported result.

full rationale

The paper's derivation chain is: sample UI-TARS-2B actions on AGUVIS tasks, have GPT-4o score them against ground-truth answers, build DPO preference pairs, fine-tune the policy, then evaluate on OSWorld. The judge is an external model (GPT-4o), the preference pairs are not derived from OSWorld outcomes, and the OSWorld results are not used to fit or select any model parameter. The closest by-construction identity would be if the evaluation criterion were the judge score itself, or if the judge had access to OSWorld execution success; neither is the case. The paper does not invoke a uniqueness theorem or rely on load-bearing self-citation: the base model (UI-TARS, ref. [13]), dataset (AGUVIS, ref. [20]), benchmark (OSWorld, ref. [19]), and algorithm (DPO, ref. [14]) are all external prior works with no author overlap with the present paper. The concerns raised in the skeptic note—that the judge prompt in Fig. 2 shows the ground-truth answer, that judge reliability is unvalidated, and that 50-step results are mixed—are validity and transfer-risk concerns, not circularity in the formal sense. They do not make the training signal equivalent to the evaluation signal. No equation or definition in the paper reduces a claimed prediction to an input by construction, so the honest finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities. It relies on one hand-chosen threshold, an arbitrary number of candidates, and unreported DPO hyperparameters, alongside several domain assumptions about judge validity, ground-truth reliability, and transferability from next-step to multi-step tasks.

free parameters (3)
  • Score threshold for DPO-1 rejection = 80
    In Strategy 1, model responses with scores below 80 are paired against ground truth. The threshold is chosen by hand with no ablation or justification, and it directly determines the composition of the training set.
  • Number of candidate responses per task = 10
    The pipeline samples 10 responses per AGUVIS task. This number is arbitrary and affects the diversity of preference pairs and the judge's comparison context.
  • DPO hyperparameters = not reported
    Learning rate, beta, and other DPO settings are not specified. These choices affect the training outcome and are free parameters in practice, though not reported.
assumptions (5)
  • domain assumption GPT-4o judge scores correlate with true action quality
    The entire preference construction relies on GPT-4o's 0-100 scores, but the paper provides no validation that these scores match human judgment or actual task success. This is a central unverified premise.
  • domain assumption AGUVIS ground-truth answers are correct and reliable
    In Strategy 1, ground-truth answers are always treated as preferred samples. If any ground-truth annotation is incorrect, the DPO pairs would contain wrong preferences.
  • domain assumption DPO improvements on next-step tasks transfer to full multi-step OSWorld tasks
    Training is on AGUVIS next-step prediction, while evaluation is on OSWorld full episodes with up to 15 or 50 steps. The paper does not demonstrate that next-step preference learning improves long-horizon planning and execution.
  • domain assumption OSWorld success rate is a valid metric of computer-use ability
    The paper uses success rate on 369 OSWorld tasks as the primary evaluation. If the benchmark has idiosyncrasies (e.g., specific environment state), results may not generalize to other settings.
  • standard math Standard DPO and RLHF math
    The DPO objective is taken as given from prior work, and the paper does not derive or modify it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DPO Learning with LLMs-Judge Signal for Computer Use Agents." pith.science (2026). https://pith.science/paper/XNIEHSF5

@misc{pith2026250603095,
  author       = {Pith},
  title        = {Pith review of: DPO Learning with LLMs-Judge Signal for Computer Use Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XNIEHSF5}},
  note         = {Machine review of arXiv:2506.03095}
}
read the original abstract

Computer use agents (CUA) are systems that automatically interact with graphical user interfaces (GUIs) to complete tasks. CUA have made significant progress with the advent of large vision-language models (VLMs). However, these agents typically rely on cloud-based inference with substantial compute demands, raising critical privacy and scalability concerns, especially when operating on personal devices. In this work, we take a step toward privacy-preserving and resource-efficient agents by developing a lightweight vision-language model that runs entirely on local machines. To train this compact agent, we introduce an LLM-as-Judge framework that automatically evaluates and filters synthetic interaction trajectories, producing high-quality data for reinforcement learning without human annotation. Experiments on the OS-World benchmark demonstrate that our fine-tuned local model outperforms existing baselines, highlighting a promising path toward private, efficient, and generalizable GUI agents.

Figures

Figures reproduced from arXiv: 2506.03095 by the authors.

Figure 1
Figure 1. Pipeline of creating DPO training data for the policy model (UI-TARs-2B) using LLM-as-judge method. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Prompt used to obtain GPT-4o scores for candidate re [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. shows the distribution of GPT-4o scores on re￾sponses generated by the baseline UI-TAR-2B model. The scores are heavily concentrated near zero, with the vast ma￾jority of samples receiving a score of 0 and only a small fraction scoring between 5 and 50. Responses above 50 are extremely rare. This distribution highlights that the baseline model, despite being state-of-the-art for its size, frequently generates low-qu… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 10 canonical work pages

  1. [1]

    Agent s2: A compositional generalist-specialist framework for computer use agents

    Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s2: A compositional generalist-specialist framework for computer use agents. arXiv preprint arXiv:2504.00906, 2025. 1, 2

    arXiv 2025

  2. [2]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023. 1

    arXiv 2023

  3. [3]

    Windows agent arena: Evaluating multi-modal os agents at scale

    Rogerio Bonatti, Dan Zhao, Francesco Bonacci, Dillon Dupont, Sara Abdali, Yinheng Li, Yadong Lu, Justin Wagle, Kazuhito Koishida, Arthur Bucker, et al. Windows agent arena: Evaluating multi-modal os agents at scale. arXiv preprint arXiv:2409.08264, 2024. 2

    arXiv 2024

  4. [4]

    Deep reinforcement learn- ing from human preferences.Advances in neural information processing systems, 30, 2017

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learn- ing from human preferences.Advances in neural information processing systems, 30, 2017. 1

    work page 2017

  5. [5]

    Mind2web: Towards a generalist agent for the web

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems, 36:28091–28114, 2023. 1, 2

    work page 2023

  6. [6]

    Detecting and preventing hallucinations in large vision language models

    Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vision language models. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 18135–18143, 2024. 2

    work page 2024

  7. [7]

    From gener- ation to judgment: Opportunities and challenges of llm-as-a- judge

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, et al. From gener- ation to judgment: Opportunities and challenges of llm-as-a- judge. arXiv preprint arXiv:2411.16594, 2024. 1

    arXiv 2024

  8. [8]

    Silkie: Preference distillation for large visual lan- guage models

    Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, and Lingpeng Kong. Silkie: Preference distillation for large visual lan- guage models. arXiv preprint arXiv:2312.10665, 2023. 2

    arXiv 2023

Show all 26 references
  1. [9]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. 1

    2023

  2. [10]

    Infiguiagent: A multimodal generalist gui agent with native reasoning and reflection.arXiv preprint arXiv:2501.04575, 2025

    Yuhang Liu, Pengxiang Li, Zishu Wei, Congkai Xie, Xueyu Hu, Xinchen Xu, Shengyu Zhang, Xiaotian Han, Hongxia Yang, and Fei Wu. Infiguiagent: A multimodal generalist gui agent with native reasoning and reflection.arXiv preprint arXiv:2501.04575, 2025. 2

    2025 arXiv

  3. [11]

    Hello gpt-4o

    OpenAI. Hello gpt-4o. https : / / openai . com / index/hello-gpt-4o/ , 2024. Accessed: 2025-05-16. 3

    2024

  4. [12]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Ad- vances in neural information processing systems, 35:...

    2022

  5. [13]

    Ui-tars: Pioneering automated gui inter- action with native agents

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shi- jue Huang, et al. Ui-tars: Pioneering automated gui inter- action with native agents. arXiv preprint arXiv:2501.12326,

    arXiv

  6. [14]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023. 2, 3

    2023

  7. [15]

    Androidinthewild: A large- scale dataset for android device control

    Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. Androidinthewild: A large- scale dataset for android device control. Advances in Neural Information Processing Systems, 36:59708–59728, 2023. 2

    2023

  8. [16]

    Proximal policy optimization algo- rithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms. arXiv preprint arXiv:1707.06347, 2017. 2

    2017 arXiv

  9. [17]

    mdpo: Conditional preference optimization for multimodal large language mod- els

    Fei Wang, Wenxuan Zhou, James Y Huang, Nan Xu, Sheng Zhang, Hoifung Poon, and Muhao Chen. mdpo: Conditional preference optimization for multimodal large language mod- els. arXiv preprint arXiv:2406.11839, 2024. 1, 2

    2024 arXiv

  10. [18]

    Os-copilot: Towards generalist computer agents with self-improvement

    Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. Os-copilot: Towards generalist computer agents with self-improvement. arXiv preprint arXiv:2402.07456, 2024. 2

    2024 arXiv

  11. [19]

    Osworld: Benchmark- ing multimodal agents for open-ended tasks in real computer environments

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, et al. Osworld: Benchmark- ing multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing...

    2024

  12. [20]

    Aguvis: Unified pure vision agents for autonomous gui interaction

    Yiheng Xu, Zekun Wang, Junli Wang, Dunjie Lu, Tian- bao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, and Caiming Xiong. Aguvis: Unified pure vision agents for autonomous gui interaction. arXiv preprint arXiv:2412.04454, 2024. 2, 3

    2024 arXiv

  13. [21]

    Gpt-4v in wonderland: Large multimodal models for zero-shot smartphone gui navigation

    An Yan, Zhengyuan Yang, Wanrong Zhu, Kevin Lin, Lin- jie Li, Jianfeng Wang, Jianwei Yang, Yiwu Zhong, Julian McAuley, Jianfeng Gao, et al. Gpt-4v in wonderland: Large multimodal models for zero-shot smartphone gui navigation. arXiv preprint arXiv:2311.07562, 2023. 2

    2023 arXiv

  14. [22]

    Direct preference optimiza- tion of video large multimodal models from language model reward

    Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander Hauptmann, Yonatan Bisk, et al. Direct preference optimiza- tion of video large multimodal models from language model reward. arXiv preprint arXiv:2404.01258, 2024. 2

    2024 arXiv

  15. [23]

    Gpt-4v (ision) is a generalist web agent, if grounded

    Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. Gpt-4v (ision) is a generalist web agent, if grounded. In International Conference on Machine Learning , pages 61349–61385. PMLR, 2024. 2

    2024

  16. [24]

    Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023. 1

    2023

  17. [25]

    Webarena: A realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023. 1, 2

    2023 arXiv

  18. [26]

    Fine-tuning language models from human preferences

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. 2

    1909 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.