Pith. sign in

REVIEW 3 major objections 95 references

Verification is a new scaling axis for language models: continuous scores from scoring-token logits, scaled on three knobs, select better agent trajectories without training a judge.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 07:01 UTC pith:H375NX5S

load-bearing objection Solid multi-domain systems paper: continuous score-token expectation plus G/K/C scaling and PPT ranking beat discrete judges and several trained robotics RMs, with real ablations—but novelty is packaging more than a new principle. the 3 major comments →

arxiv 2607.05391 v2 pith:H375NX5S submitted 2026-07-06 cs.AI cs.CLcs.LGcs.MAcs.RO

LLM-as-a-Verifier: A General-Purpose Verification Framework

Jacky Kwok , Shulu Li , Pranav Atreya , Yuejiang Liu , Yixing Jiang , Chelsea Finn , Marco Pavone , Ion Stoica
show 1 more author
Azalia Mirhoseini
This is my paper
classification cs.AI cs.CLcs.LGcs.MAcs.RO
keywords LLM-as-a-Verifierverification scalingscoring-token logitstrajectory reward modeltest-time scalingdense rewardsagentic tasksBradley-Terry ranking
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that deciding whether a solution is correct has been left behind while generation has scaled, and that verification itself can be scaled. Standard language-model judges collapse their scoring distribution into a single discrete token, which creates ties and weak discrimination among long agent trajectories. LLM-as-a-Verifier instead takes the expectation over the full distribution of scoring-token logits, producing continuous scores. That probabilistic formulation opens three practical knobs: finer score granularity, repeated evaluations, and decomposition into simpler criteria. With a budget-aware ranking procedure that concentrates comparisons on the most promising candidates, the same training-free verifier reaches new highs on coding, robotics, and medical agent benchmarks and supplies dense progress signals that speed up both off-policy and on-policy reinforcement learning.

Core claim

Verification quality improves systematically when continuous rewards are formed as the expectation over scoring-token logits and then scaled along score granularity, number of repeated evaluations, and criteria decomposition. Combined with a cost-efficient pivot tournament that converts those continuous scores into preference probabilities, the framework selects better trajectories at test time and supplies denser rewards for reinforcement learning, all without training a specialized reward model.

What carries the argument

The continuous reward of Equation 3.1: the average, over criteria and repeated evaluations, of the expected scalar value of the scoring-token distribution. Pairwise preferences follow from a Bradley–Terry model on those rewards; a Probabilistic Pivot Tournament then ranks N candidates in O(Nk) comparisons by first breaking positional bias with a random ring pass and then concentrating remaining budget on a small set of pivots.

Load-bearing premise

That a language model’s probability mass over scoring tokens, when prompted with hand-written criteria, is a calibrated, domain-general measure of trajectory quality rather than an artifact of prompt style or superficial cues.

What would settle it

On a held-out set of trajectory pairs whose ground-truth correctness is known, replace continuous logit expectations with ordinary discrete argmax scores (or with random continuous scores of matched variance) while keeping the same ranking procedure; if the accuracy and progress-correlation gains disappear, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Best-of-N selection for long-horizon coding, robotics, and medical agents can improve without any new reward-model training.
  • Live progress scores become available for monitoring and early abort of agentic systems.
  • Dense verifier rewards raise sample efficiency of both SAC-style and GRPO-style reinforcement learning.
  • Verification compute can be dialed independently of generation compute via the three scaling knobs and the pivot budget.
  • The same recipe transfers across text and multi-frame video once log-probabilities (or a two-stage open-model handoff) are available.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If scoring-token mass is truly calibrated, criteria decomposition may eventually be generated automatically per domain rather than hand-designed, turning the third knob into a learned object.
  • The same continuous signal could serve as a runtime monitor that freezes or rolls back an agent when the progress curve plateaus or declines, independent of final-outcome ranking.
  • Closed models that withhold logits may still participate via the two-stage handoff, which suggests a practical division of labor: frontier reasoning + open calibrated scoring.
  • Because the framework is training-free, gains should appear most clearly on domains where collecting preference data for a learned reward model is expensive or safety-critical.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper proposes LLM-as-a-Verifier, a training-free verification framework that scores agent trajectories by taking the expectation of a scalar map over the model’s scoring-token probability mass (Eq. 3.1), rather than collapsing to a discrete argmax score as in standard LM judges. It argues that this continuous formulation unlocks three complementary verification-scaling axes—score granularity G, repeated evaluation K, and criteria decomposition C—and introduces Probabilistic Pivot Tournament (PPT) to select among N candidates at O(Nk) pairwise cost. Empirically, the method reports improved pairwise verification accuracy under controlled G/K/C budgets (Fig. 4, Table 1, Fig. 7), competitive or state-of-the-art trajectory selection on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4% preference accuracy), and MedAgentBench (73.3%), Value-Order Correlation as a progress proxy, and denser rewards that improve sample efficiency of DSRL-SAC on LIBERO and GRPO on MATH (Fig. 9).

Significance. If the results hold under broader scrutiny, the paper makes a useful contribution to test-time scaling and agent evaluation: it reframes verification quality as something that can be improved by compute and prompt structure without reward-model training, and it demonstrates transfer across coding, robotics video, and medical agent settings with a single probabilistic recipe. Strengths include controlled scaling plots, an explicit SNR decomposition for granularity (Table 1), a concrete case study with tie-rate analysis (Table 2), a budget–accuracy characterization of PPT (Table 9), multi-benchmark selection results with Pass@1/oracle context (Table 3), and dual use of the same signal for progress monitoring and RL shaping. The practical artifacts (Claude Code/Codex-style extension, harness generalization in Appendix B.1, closed-model two-stage workaround in B.6) increase the work’s utility beyond a pure leaderboard claim.

major comments (3)
  1. §3.2 Eq. (3.1)–(3.2) and §4.1: The central interpretation—that continuous scoring-token expectations yield more calibrated correctness comparisons—is supported by SNR growth, lower tie rates, and accuracy gains, but the manuscript does not adequately rule out surface-form confounds (trajectory length, log verbosity, tool-call polish, formatting). The query-optimize case (Table 2, App. B.4) shows the model can identify a real methodological failure while still expressing it in hedged language, which is consistent with either calibrated belief or stylistic confidence. A load-bearing addition would be matched-surface ablations or partial correlations of R(x,τ) with length/verbosity after conditioning on success/failure; without this, the claim that verification is a distinct scaling axis (vs. a better readout of existing judge biases) remains incompletely stress-tested.
  2. §4.3 and the multi-domain experiments in §5: Criteria decomposition is presented as a general scaling axis, yet the concrete factors (Specification/Output/Errors for code; domain-specific criteria elsewhere) are hand-designed. Fig. 4 (right) shows ensemble gains, but there is no sensitivity analysis to criterion wording, number, or quality, nor a clear protocol for constructing criteria in a new domain. Because SOTA numbers use G=20, K=8, and the three-criterion setup, the paper should quantify how much of the reported selection accuracy is attributable to rubric engineering versus the continuous formulation and PPT; otherwise the “general-purpose / no additional training” claim overstates plug-and-play generality.
  3. §5 and Table 3: Absolute “state-of-the-art” framing should be tightened. The fairest evidence is improvement over Pass@1 on the same candidate pools (e.g., Terminal-Bench 83.1%→86.5%, SWE-Bench 76.1%→78.2%, MedAgent 70.2%→73.3%), which is real but modest relative to oracle headroom. Leaderboard comparisons mix harnesses, proposal models, and N; SWE-Bench further uses a heterogeneous three-model pool, so selection partly includes cross-model routing. Please report selection accuracy with uncertainty (bootstrap/task-level variance), fix N and harness when comparing to named baselines, and separate “best-of-N with our verifier” from “beats published single-trajectory leaderboard entries.”

Circularity Check

0 steps flagged

No significant circularity: continuous scores are a defined estimator evaluated against external ground truth, not predictions forced by construction.

full rationale

LLM-as-a-Verifier is an empirical methods paper. Equation 3.1 defines the continuous reward as the expectation of scoring-token values under the prompted model’s logits, averaged over criteria C and repeats K; Equations 3.2 and the Probabilistic Pivot Tournament then convert those scores into pairwise preferences for selection. These are definitions of the proposed estimator, not derivations that smuggle the target into the inputs. Verification accuracy, SOTA trajectory selection (Terminal-Bench V2, SWE-Bench Verified, RoboRewardBench, MedAgentBench), Value-Order Correlation, and RL sample-efficiency gains are all measured against external labels: hidden task graders, human preference annotations, chronological step order, and environment success rates. Hyperparameters G, K, C, k, λ, β are ablated under controlled budgets rather than fitted so as to force the reported accuracies. There is no self-definitional loop (X defined via Y then used to “predict” Y), no fitted parameter renamed as a prediction of the same quantity, no load-bearing uniqueness theorem imported from overlapping authors, and no ansatz smuggled in via self-citation that forbids alternatives. Minor self-citations in related work (e.g., prior robotics verification papers) are not load-bearing for the central claims. The derivation chain is therefore self-contained against external benchmarks; circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The central claims rest on standard probabilistic ranking assumptions, the premise that prompted LLM/VLM logits encode useful quality/progress beliefs, hand-chosen evaluation criteria and scaling hyperparameters, and access (direct or staged) to scoring-token probabilities. No new physical entities are postulated; the invented pieces are algorithmic constructs evaluated empirically.

free parameters (5)
  • score granularity G
    Number of scoring tokens (up to 20) chosen as a scaling knob; main results use G=20.
  • repeated evaluations K
    Monte Carlo repeats of verification; main results often use K=8 (robotics VOC uses K=5).
  • criteria count/decomposition C
    Hand-designed sub-criteria (e.g., Specification/Output/Errors for code) averaged in Eq. 3.1.
  • pivot count k in PPT
    Budget–accuracy trade-off parameter reducing comparisons from O(N^2) to O(Nk); ablated in Table 9.
  • RL shaping weights λ and β
    Trade-off coefficients for verifier progress/reasoning rewards in SAC (λ=1) and GRPO (β=0.1).
axioms (4)
  • standard math Bradley–Terry model maps continuous reward differences to pairwise preference probabilities (Eq. 3.2).
    Standard ranking assumption used to aggregate soft wins in PPT.
  • domain assumption Scoring-token logprobs of a prompted LLM/VLM are a useful, generalizable signal of trajectory correctness/progress without task-specific reward training.
    Load-bearing premise of the entire training-free framework (§3–§7).
  • ad hoc to paper Hand-written domain criteria and pairwise prompts adequately factor long-horizon quality for coding, robotics video, and medical agents.
    Criteria decomposition is presented as a scaling axis but designed by authors (§4.3).
  • ad hoc to paper For logit-restricted APIs, free-form frontier reasoning plus open-model logprob scoring recovers most continuous-verifier benefit.
    Appendix B.6 workaround needed for closed models; not the primary setup.
invented entities (2)
  • LLM-as-a-Verifier continuous reward (Eq. 3.1) independent evidence
    purpose: Define training-free fine-grained trajectory scores from expected scoring-token values averaged over criteria and repeats.
    Methodological construct; evaluated against external outcomes rather than postulated as a physical object.
  • Probabilistic Pivot Tournament (PPT) independent evidence
    purpose: Select best of N candidates under O(Nk) verification budget with ring-pass bias cancellation.
    Algorithmic invention; budget–accuracy characterized on Terminal-Bench candidate pools.

pith-pipeline@v1.1.0-grok45 · 32967 in / 3272 out tokens · 39755 ms · 2026-07-11T07:01:25.730013+00:00 · methodology

0 comments
read the original abstract

Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM-as-a-Verifier computes the expectation over the distribution of scoring token logits to generate continuous scores. This probabilistic formulation enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scaling repeated evaluation and criteria decomposition consistently lead to additional gains in verification accuracy through variance and complexity reduction. We further introduce a cost-efficient ranking algorithm for selecting the best solution among candidates using the verifier's continuous scores. LLM-as-a-Verifier achieves state-of-the-art performance on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine-grained signals from LLM-as-a-Verifier can also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show that LLM-as-a-Verifier can provide dense feedback for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

95 extracted references · 59 linked inside Pith

  1. [1]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling Laws for Neural Language Models, January 2020. URLhttp://arxiv.org/abs/2001.08361. arXiv:2001.08361 [cs]

    Pith/arXiv arXiv 2020

  2. [2]

    Scaling Laws for Reward Model Overoptimization, October 2022

    Leo Gao, John Schulman, and Jacob Hilton. Scaling Laws for Reward Model Overoptimization, October 2022. URLhttp://arxiv.org/abs/2210.10760. arXiv:2210.10760 [cs]

    Pith/arXiv arXiv 2022

  3. [3]

    Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters, August 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters, August 2024. URLhttp://arxiv.org/abs/ 2408.03314. arXiv:2408.03314 [cs]

    Pith/arXiv arXiv 2024

  4. [4]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena, 2023. URLhttps://arxiv.org/abs/ 2306.05685

    Pith/arXiv arXiv 2023

  5. [5]

    URLhttps://arxiv.org/abs/2603.04304

    HarmanSingh,XiuyuLi,KushaSareen,MonishwaranMaheswaran,SijunTan,XiaoxiaWu,Junxiong Wang, Alpay Ariyak, Qingyang Wu, Samir Khaki, Rishabh Tiwari, Long Lian, Yucheng Lu, Boyi Li, Alane Suhr, Ben Athiwaratkun, and Kurt Keutzer.𝑉1: Unifying Generation and Self-Verification for Parallel Reasoners, 2026. URLhttps://arxiv.org/abs/2603.04304

    arXiv 2026

  6. [6]

    Generative verifiers: Reward modeling as next-token prediction, 2025

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction, 2025. URLhttps://arxiv.org/ abs/2408.15240

    Pith/arXiv arXiv 2025

  7. [7]

    Training Verifiers to Solve Math Word Problems, November 2021

    KarlCobbe,VineetKosaraju,MohammadBavarian,MarkChen,HeewooJun,LukaszKaiser,Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training Verifiers to Solve Math Word Problems, November 2021. URLhttp://arxiv.org/abs/ 2110.14168. arXiv:2110.14168 [cs]

    Pith/arXiv arXiv 2021

  8. [8]

    Let’s Verify Step by Step, May 2023

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s Verify Step by Step, May 2023. URL http://arxiv.org/abs/2305.20050. arXiv:2305.20050 [cs]

    Pith/arXiv arXiv 2023

  9. [9]

    Robometer: Scaling general-purpose robotic reward models via trajectory comparisons.arXiv preprint arXiv:2603.02115, 2026

    Anthony Liang, Yigit Korkmaz, Jiahui Zhang, Minyoung Hwang, Abrar Anwar, Sidhant Kaushik, Aditya Shah, Alex S Huang, Luke Zettlemoyer, Dieter Fox, et al. Robometer: Scaling general-purpose robotic reward models via trajectory comparisons.arXiv preprint arXiv:2603.02115, 2026. 17 LLM-as-a-Verifier: A General-Purpose Verification Framework

    Pith/arXiv arXiv 2026

  10. [10]

    Topreward: Token probabilities as hidden zero-shot rewards for robotics.arXiv preprint arXiv:2602.19313, 2026

    Shirui Chen, Cole Harrison, Ying-Chun Lee, Angela Jin Yang, Zhongzheng Ren, Lillian J Ratliff, Jiafei Duan, Dieter Fox, and Ranjay Krishna. Topreward: Token probabilities as hidden zero-shot rewards for robotics.arXiv preprint arXiv:2602.19313, 2026

    arXiv 2026

  11. [11]

    RoboReward: General-purpose vision-language reward models for robotics, 2026

    Tony Lee, Andrew Wagenmaker, Karl Pertsch, Percy Liang, Sergey Levine, and Chelsea Finn. RoboReward: General-purpose vision-language reward models for robotics, 2026. URLhttps: //arxiv.org/abs/2601.00675

    arXiv 2026

  12. [12]

    Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023. URLhttps://arxiv.org/ abs/2306.03310

    Pith/arXiv arXiv 2023

  13. [13]

    Steering your diffusion policy with latent space reinforcement learning, 2025

    Andrew Wagenmaker, Mitsuhiko Nakamoto, Yunchu Zhang, Seohong Park, Waleed Yagoub, Anusha Nagabandi, Abhishek Gupta, and Sergey Levine. Steering your diffusion policy with latent space reinforcement learning, 2025. URLhttps://arxiv.org/abs/2506.15799

    Pith/arXiv arXiv 2025

  14. [14]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    Pith/arXiv arXiv doi:10.1038/s41586-025-09422-z

  15. [15]

    Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, 18 LLM-as-a-Verifier: A General-Purpose Verification Framework Dario Amodei, and Paul Christiano

    Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, 18 LLM-as-a-Verifier: A General-Purpose Verification Framework Dario Amodei, and Paul Christiano. Learning to summarize from human feedback, February 2022. URLhttp://arxiv.org/abs/2009.01325. arXiv:2009.01325 [cs]

    Pith/arXiv arXiv 2022

  16. [16]

    Gemini 2.5 Flash

    Google. Gemini 2.5 Flash. https://ai.google.dev/gemini-api/docs/models/gemini-2. 5-flash, 2026. Accessed: 2026-05-06

    2026

  17. [17]

    Merrill, Alexander G

    Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E. Kelly Buchanan, Junhong Shen, Guanghao Ye, Haowei Lin, JasonPoulos, MaoyuWang, MariannaNezhurina, JeniaJitsev, DiLu, OrfeasMenisMastromichalakis, Zhiwei Xu, Zizhao Chen, Yue Liu, Robert Zhang, Leon Liangyu Chen, Anurag Ka...

    Pith/arXiv arXiv 2026

  18. [18]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?, November

  19. [19]

    arXiv:2310.06770 [cs]

    URLhttp://arxiv.org/abs/2310.06770. arXiv:2310.06770 [cs]

    Pith/arXiv arXiv

  20. [20]

    YixingJiang,KameronC.Black,GloriaGeng,DannyPark,JamesZou,AndrewY.Ng,andJonathanH. Chen. A virtual ehr environment to benchmark medical llm agents.NEJM AI, 2(9), 2025. doi: 10.1056/AIdbp2500144

    doi:10.1056/aidbp2500144 2025

  21. [21]

    URL https://www.tbench.ai/leaderboard/terminal-bench/2.0/capy-build/unknown/ gpt-5.5%40openai

    Capy. URL https://www.tbench.ai/leaderboard/terminal-bench/2.0/capy-build/unknown/ gpt-5.5%40openai

  22. [22]

    URL https://github.com/harbor-framework/terminal-bench/tree/main/ terminal_bench/agents/terminus_2

    Terminus 2. URL https://github.com/harbor-framework/terminal-bench/tree/main/ terminal_bench/agents/terminus_2

  23. [23]

    Vision language models are in-context value learners

    Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani, Peng Xu, Danny Driess, Ted Xiao, Jonathan Tompson, Osbert Bastani, Dinesh Jayaraman, Wenhao Yu, Tingnan Zhang, Dorsa Sadigh, and Fei Xia. Vision language models are in-context value learners. InInternational Conference on Learning Representations, 2025....

    Pith/arXiv arXiv doi:10.48550/arxiv.2411.04549 2025

  24. [24]

    URLhttps://arxiv.org/abs/2410.24164

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky.𝜋0: A visio...

    Pith/arXiv arXiv 2026

  25. [25]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. 19 LLM-as-a-Verifier: A General-Purpose Verification Framework

    Pith/arXiv arXiv 2025

  26. [26]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2022. URLhttps://arxiv.org/abs/2201.11903

    Pith/arXiv arXiv 2022

  27. [27]

    Large language models are zero-shot reasoners, 2022

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, 2022. URLhttps://arxiv.org/abs/2205.11916

    Pith/arXiv arXiv 2022

  28. [28]

    Le, and Ed H

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V. Le, and Ed H. Chi. Least-to-most prompting enables complex reasoning in large language models. InInternational Conference on Learning Representations, 2023. URLhttps://openreview.net/forum?id=WZH7099tgfM

    2023

  29. [29]

    Self-consistency improves chain of thought reasoning in language models

    XuezhiWang, JasonWei, DaleSchuurmans, QuocLe, EdChi, SharanNarang, AakankshaChowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. URL http://arxiv.org/abs/2203.11171

    Pith/arXiv arXiv

  30. [30]

    Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems, 36:11809–11822, 2023

    ShunyuYao, DianYu, JeffreyZhao, IzhakShafran, TomGriffiths, YuanCao, andKarthikNarasimhan. Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems, 36:11809–11822, 2023

    2023

  31. [31]

    Graph of thoughts: Solving elaborate problems with large language models, 2023

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models, 2023. URLhttps://arxiv. org/abs/2308.09687

    Pith/arXiv arXiv 2023

  32. [32]

    ReAct: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. URLhttp://arxiv.org/abs/2210. 03629

  33. [33]

    Lan- guage agent tree search unifies reasoning acting and planning in language models.arXiv preprint arXiv:2310.04406, 2023

    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Lan- guage agent tree search unifies reasoning acting and planning in language models.arXiv preprint arXiv:2310.04406, 2023

    Pith/arXiv arXiv 2023

  34. [34]

    Reasoning with language model is planning with world model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URLhttps://arxiv.org/abs/2305. 14992

    2023

  35. [35]

    Large language models are better reasoners with self-verification, 2022

    Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification, 2022. URLhttps: //arxiv.org/abs/2212.09561

    Pith/arXiv arXiv 2022

  36. [36]

    Self-refine: Iterative refinement with self-feedback, 2023

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback, 2023. URLhttps://arxiv.org/abs/2303.17651

    Pith/arXiv arXiv 2023

  37. [37]

    Reflexion: Language agents with verbal reinforcement learning, 2023

    Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning, 2023. URLhttps: //arxiv.org/abs/2303.11366. 20 LLM-as-a-Verifier: A General-Purpose Verification Framework

    Pith/arXiv arXiv 2023

  38. [38]

    CRITIC: Large language models can self-correct with tool-interactive critiquing

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC: Large language models can self-correct with tool-interactive critiquing. InInternational Conference on Learning Representations, 2024. URLhttps://arxiv.org/abs/2305.11738

    Pith/arXiv arXiv 2024

  39. [39]

    Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J

    Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning, July 2025. URLhttp:...

    Pith/arXiv arXiv 2025

  40. [40]

    Alexander Novikov, Ngân V˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. AlphaEvolve: A coding agent for scientific and algor...

  41. [41]

    arXiv:2506.13131 [cs]

    URLhttp://arxiv.org/abs/2506.13131. arXiv:2506.13131 [cs]

    Pith/arXiv arXiv

  42. [42]

    Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushm...

    doi:10.1126/science.abq1158 2022

  43. [43]

    Le, Christopher Ré, and Azalia Mirhoseini

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling,

  44. [44]

    URLhttps://arxiv.org/abs/2407.21787

    Pith/arXiv arXiv

  45. [45]

    Inference-aware fine-tuning for best-of-N sampling in large language models, 2024

    Yinlam Chow, Guy Tennenholtz, Izzeddin Gur, Vincent Zhuang, Bo Dai, Sridhar Thiagarajan, Craig Boutilier, Rishabh Agarwal, Aviral Kumar, and Aleksandra Faust. Inference-aware fine-tuning for best-of-N sampling in large language models, 2024. URLhttps://arxiv.org/abs/2412.15287

    arXiv 2024

  46. [46]

    GPTScore: Evaluate as you desire, 2023

    Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. GPTScore: Evaluate as you desire, 2023. URLhttps://arxiv.org/abs/2302.04166

    Pith/arXiv arXiv 2023

  47. [47]

    G-eval: NLG evaluation using GPT-4 with better human alignment

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: NLG evaluation using GPT-4 with better human alignment. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore, December

    2023

  48. [48]

    doi: 10.18653/v1/2023.emnlp-main.153

    Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.153. URL https://aclanthology.org/2023.emnlp-main.153/

    doi:10.18653/v1/2023.emnlp-main.153 2023

  49. [49]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-bench and chatbot arena. InAdvances in Neural Information Processing Systems, 2023. URLhttps://openreview.net/forum?id=uccHPGDlao

    2023

  50. [50]

    Hashimoto

    Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. Length-controlled AlpacaEval: A simple way to debias automatic evaluators, 2024. URLhttps://arxiv.org/abs/ 2404.04475. 21 LLM-as-a-Verifier: A General-Purpose Verification Framework

    Pith/arXiv arXiv 2024

  51. [51]

    Gonzalez, and Ion Stoica

    Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks: Arena-Hard and BenchBuilder pipeline, 2024. URLhttps://arxiv.org/abs/2406.11939

    Pith/arXiv arXiv 2024

  52. [52]

    Flask: Fine-grained language model evaluation based on alignment skill sets.arXiv preprint arXiv:2307.10928, 2023

    Seonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, Seungone Kim, Yongrae Jo, James Thorne, Juho Kim, and Minjoon Seo. Flask: Fine-grained language model evaluation based on alignment skill sets.arXiv preprint arXiv:2307.10928, 2023

    Pith/arXiv arXiv 2023

  53. [53]

    Prometheus: Inducing fine-grained evaluation capability in language models

    Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. Prometheus: Inducing fine-grained evaluation capability in language models. InInternational Conference on Learning Representations,

  54. [54]

    URLhttps://openreview.net/forum?id=8euJaTveKw

  55. [55]

    Prometheus 2: An open source language model specialized in evaluating other language models, 2024

    Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. Prometheus 2: An open source language model specialized in evaluating other language models, 2024. URLhttps://arxiv.org/abs/2405. 01535

    2024

  56. [56]

    Generative judge for evaluating alignment

    Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. Generative judge for evaluating alignment. InInternational Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=gtkFw6sZGS

    2024

  57. [57]

    Themis: A reference-free NLG evaluation language model with flexibility and interpretability, 2024

    Xinyu Hu, Li Lin, Mingqi Gao, Xunjian Yin, and Xiaojun Wan. Themis: A reference-free NLG evaluation language model with flexibility and interpretability, 2024. URLhttps://arxiv.org/ abs/2406.18365

    Pith/arXiv arXiv 2024

  58. [58]

    HD-eval: Aligning large language model evaluators through hierarchical criteria decomposition

    Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. HD-eval: Aligning large language model evaluators through hierarchical criteria decomposition. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7641–7660, Bangkok, Thail...

    doi:10.18653/v1/2024.acl-long.413 2024

  59. [59]

    PandaLM:An automaticevaluationbenchmarkforLLMinstructiontuningoptimization

    Yidong Wang, Zhuohao Yu, Wenjin Yao, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, ChaoyaJiang, RuiXie, JindongWang, XingXie, WeiYe, ShikunZhang, andYueZhang. PandaLM:An automaticevaluationbenchmarkforLLMinstructiontuningoptimization. InInternationalConference on Learning Representations, 2024. URLhttps://openreview.net/forum?id=5Nn2BLV7SB

    2024

  60. [60]

    JudgeLM: Fine-tuned large language models are scalable judges

    Lianghui Zhu, Xinggang Wang, and Xinlong Wang. JudgeLM: Fine-tuned large language models are scalable judges. InInternational Conference on Learning Representations, 2025. URLhttps: //openreview.net/forum?id=xsELpEPn4A

    2025

  61. [61]

    Chateval: Towards better llm-based evaluators through multi-agent debate.arXiv preprint arXiv:2308.07201, 2023

    Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better llm-based evaluators through multi-agent debate.arXiv preprint arXiv:2308.07201, 2023

    Pith/arXiv arXiv 2023

  62. [62]

    Shrinking the generation-verification gap with weak verifiers.arXiv preprint arXiv:2506.18203, 2025

    Jon Saad-Falcon, E Kelly Buchanan, Mayee F Chen, Tzu-Heng Huang, Brendan McLaughlin, Tanvir Bhathal, Shang Zhu, Ben Athiwaratkun, Frederic Sala, Scott Linderman, et al. Shrinking the generation-verification gap with weak verifiers.arXiv preprint arXiv:2506.18203, 2025. 22 LLM-as-a-Verifier: A General-Purpose Verification Framework

    arXiv 2025

  63. [63]

    Large language models are not fair evaluators

    Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9440–9450, Bangkok, Thailand, August 2024. Association for...

    doi:10.18653/v1/2024.acl-long.511 2024

  64. [64]

    Evaluating large language models at evaluating instruction following

    Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following. InInternational Conference on Learning Representations, 2024. URLhttps://openreview.net/forum?id=tr0KidwPLc

    2024

  65. [65]

    Benchmarking cognitive biases in large language models as evaluators, 2023

    Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. Benchmarking cognitive biases in large language models as evaluators, 2023. URLhttps://arxiv. org/abs/2309.17012

    Pith/arXiv arXiv 2023

  66. [66]

    LLMs as narcissistic evaluators: When ego inflates evaluation scores, 2023

    Yiqi Liu, Nafise Sadat Moosavi, and Chenghua Lin. LLMs as narcissistic evaluators: When ego inflates evaluation scores, 2023. URLhttps://arxiv.org/abs/2311.09766

    Pith/arXiv arXiv 2023

  67. [67]

    JudgeBench: A benchmark for evaluating LLM-based judges

    Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Yuan Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica. JudgeBench: A benchmark for evaluating LLM-based judges. InInternational Conference on Learning Representations, 2025. URLhttps://openreview. net/forum?id=G0dksFayVq

    2025

  68. [68]

    An empirical study of LLM-as-a-judge for LLM evaluation: Fine-tuned judge model is not a general substitute for GPT-4

    Hui Huang, Xingyuan Bu, Hongli Zhou, Yingqi Qu, Jing Liu, Muyun Yang, Bing Xu, and Tiejun Zhao. An empirical study of LLM-as-a-judge for LLM evaluation: Fine-tuned judge model is not a general substitute for GPT-4. InFindings of the Association for Computational Linguistics: ACL 2025, pages 5880–5895, Vienna, Austria, July 2025. Association for Computatio...

    2025

  69. [69]

    Hongchao Jiang, Yiming Chen, Yushi Cao, Hung-yi Lee, and Robby T. Tan. CodeJudgeBench: Benchmarking LLM-as-a-judge for coding tasks, 2025. URLhttps://arxiv.org/abs/2507.10535

    Pith/arXiv arXiv 2025

  70. [70]

    MLLM-as-a-judge: Assessing multimodal LLM-as-a-judge with vision-language benchmark, 2024

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yinuo Liu, Yaochen Wang, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. MLLM-as-a-judge: Assessing multimodal LLM-as-a-judge with vision-language benchmark, 2024. URLhttps://arxiv.org/abs/2402.04788

    Pith/arXiv arXiv 2024

  71. [71]

    Prometheus-vision: Vision-language model as a judge for fine-grained evaluation, 2024

    Seongyun Lee, Seungone Kim, Sue Hyun Park, Geewook Kim, and Minjoon Seo. Prometheus-vision: Vision-language model as a judge for fine-grained evaluation, 2024. URLhttps://arxiv.org/ abs/2401.06591

    Pith/arXiv arXiv 2024

  72. [72]

    LLaVA-critic: Learning to evaluate multimodal models, 2024

    Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. LLaVA-critic: Learning to evaluate multimodal models, 2024. URLhttps://arxiv. org/abs/2410.02712

    Pith/arXiv arXiv 2024

  73. [73]

    Solving math word problems with process- and outcome-based feedback, 2022

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process- and outcome-based feedback, 2022. URLhttps://arxiv.org/abs/2211.14275

    Pith/arXiv arXiv 2022

  74. [74]

    VIP: Towards universal visual reward and representation via value-implicit pre-training,

    Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. VIP: Towards universal visual reward and representation via value-implicit pre-training,

  75. [75]

    23 LLM-as-a-Verifier: A General-Purpose Verification Framework

    URLhttps://arxiv.org/abs/2210.00030. 23 LLM-as-a-Verifier: A General-Purpose Verification Framework

    Pith/arXiv arXiv

  76. [76]

    LIV: Language-image representations and rewards for robotic control, 2023

    Yecheng Jason Ma, William Liang, Vaidehi Som, Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayaraman. LIV: Language-image representations and rewards for robotic control, 2023. URLhttps://arxiv.org/abs/2306.00958

    Pith/arXiv arXiv 2023

  77. [77]

    Sontakke, Jesse Zhang, Sébastien M

    Sumedh A. Sontakke, Jesse Zhang, Sébastien M. R. Arnold, Karl Pertsch, Erdem Biyik, Dorsa Sadigh, Chelsea Finn, and Laurent Itti. RoboCLIP: One demonstration is enough to learn robot policies,

  78. [78]

    URLhttps://arxiv.org/abs/2310.07899

    Pith/arXiv arXiv

  79. [79]

    Vision- language models are zero-shot reward models for reinforcement learning, 2023

    Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. Vision- language models are zero-shot reward models for reinforcement learning, 2023. URLhttps: //arxiv.org/abs/2310.12921

    Pith/arXiv arXiv 2023

  80. [80]

    RL-VLM-F: Reinforcement learning from vision language foundation model feedback, 2024

    Yufei Wang, Zhanyi Sun, Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. RL-VLM-F: Reinforcement learning from vision language foundation model feedback, 2024. URL https://arxiv.org/abs/2402.03681

    Pith/arXiv arXiv 2024

Showing first 80 references.