Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Reinforce LLM Reasoning through Multi-Agent Reflection

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that jointly training a solver and a critic with a DPO-style objective, on self-generated refinement trajectories, turns multi-turn reflection into a learnable skill that provably competes with any covered policy and…

desk verdict A useful incremental method for training multi-agent LLM refinement, with a theory that does not quite cover the practical algorithm; the empirical work is solid enough to warrant a careful revision rather than a desk reject. read the letter →

arxiv 2506.08379 v1 pith:YWJEO7GM submitted 2025-06-10 cs.LG cs.AIcs.CL

Yurun Yuan , Tengyang Xie This is my paper
classification cs.LGcs.AIcs.CL
keywords DPSDPmulti-agentreflectiondirectpreferenceoptimizationMarkovdecisionprocessiterativerefinementLLMreasoningtest-timecomputationmath
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces DPSDP, an RL algorithm that trains two specialized agents—a solver and a critic—to iteratively revise answers to math problems. The multi-turn critique-and-refine loop is cast as a Markov decision process, and both agents are trained with direct preference learning on data that a reference model generates by rolling out the loop. The central claim is that jointly optimizing the pair this way yields a policy that provably competes with any policy covered by the reference model, and empirically lifts first-turn accuracy on MATH 500 from 58.2% to 63.2% under majority voting over five refinements. A sympathetic reader would care because the method converts test-time reflection into a trainable skill rather than relying on a fixed prompt or external verifier.

What carries the argument

The load-bearing machinery is a PSDP-style backward induction over a three-step MDP (answer, feedback, refined answer), implemented with a closed-form KL-regularized policy update that collapses into the DPO loss. State transitions are Markovian by design: the actor sees only the latest answer and the critic's feedback, and the critic sees only the latest answer, which removes trajectory-level distribution shift and lets the trained policy refine any number of times. Q-values are estimated by rolling out the reference policy and scoring correctness, with the first-turn Q approximated by whether the initial answer is correct; the paper argues this is a constant shift from the ideal Q when later policies are optimal. The whole procedure runs offline in a single optimization step because all pairwise datasets are collected from the reference policy, avoiding loop-carried dependencies.

What would settle it

Train or evaluate the same DPSDP pipeline on a distribution where first-turn correct answers are often overturned by later corrections and first-turn wrong answers often become correct—the paper's own Table 4 shows such flips occur. If the actor trained with the immediate-correctness proxy performs worse than an oracle-Q variant that scores first-turn answers by their eventual majority-vote contribution, the proxy is the cause. Concretely, compare DPSDP against a version using rollout-based Q-values on MATH 500 and measure whether the proxy's misordering rate predicts the performance gap.

Watch

Extended reading notes

Core claim

DPSDP's central claim is that the verify-and-improve loop can be treated as a fixed-horizon MDP and solved by backward induction with a KL-regularized, DPO-compatible objective at each turn. The actor is trained to generate the initial answer and each refinement, the critic is trained to produce feedback, and the Q-value needed at each state is estimated from the correctness of answers produced by rolling out the reference policy. The paper proves that if the reference policy covers the optimal policy's state-action distribution and the in-distribution reward-learning error is bounded, then $J(\pi^\star)-J(\hat\pi)=O\bigl(H\sqrt{C_S^\star C_A \varepsilon_{\mathrm{stat}}}\bigr)$, so DPSDP matches any policy in distribution. The practical implementation reduces the training horizon to three steps by keeping only the most recent answer and feedback in the state, enabling the resulting policy to be applied for arbitrarily many refinement rounds at test time. Empirically, this yields consistent gains across Ministral, Llama-3.1, and Qwen2.5 models on MATH 500, GSM8K, MMLU-Pro Math, and Olympiad Bench, with the largest gains coming from the agent's ability to turn initial errors into correct answers rather than from majority voting alone.

Load-bearing premise

The load-bearing premise is that a first-turn answer's immediate correctness is a good stand-in for its long-term value under refinement; if that proxy is biased, the actor is trained to maximize first-turn hits rather than the success of the full refine-then-vote process.

Editorial extensions

If this is right

  • On MATH 500, majority voting over five refinements raises first-turn accuracy from 58.2% to 63.2% for Ministral-based models, from 55.8% to 58.4% for Llama-3.1, and from 60.4% to 62.0% for Qwen2.5.
  • The Markovian last-answer-only state means a policy trained with one refinement step can be rolled out for arbitrarily many refinement rounds without a train-test horizon mismatch.
  • Controlling for sampling, majority voting over five refined answers beats majority voting over five independent first-turn answers, so the gains come from correcting earlier errors rather than from simple self-consistency.
  • A generative text critic outperforms a binary-value critic on hard and out-of-distribution benchmarks, while the binary critic can win on easy GSM8K problems because the generative critic tends to overthink them.
  • The theoretical bound $J(\pi^\star)-J(\hat\pi)=O\bigl(H\sqrt{C_S^\star C_A \varepsilon_{\mathrm{stat}}}\bigr)$ shows that DPSDP inherits a PSDP-style performance guarantee whenever the reference policy has adequate coverage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same actor-critic DPO recipe could generalize to code repair or tool-use tasks where feedback is richer than binary correctness, since the critic is unrestricted text; the main unknown is whether the immediate-correctness proxy transfers.
  • The paper's finding that single-agent training slightly wins on easy benchmarks but loses on hard ones suggests a hybrid controller that switches between self-reflection and a separate critic based on problem difficulty.
  • The over-thinking failure mode on grade-school problems points to a cheap extension: a value head or confidence threshold that stops refinement once the answer is stable, which the paper itself suggests as early stopping.
  • Because the bound scales with coverage $C_S^\star$, out-of-distribution performance should degrade as the reference policy's coverage of OOD states shrinks; the small Olympiad gains are consistent with this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes DPSDP (Direct Policy Search by Dynamic Programming), an actor-critic algorithm that trains an LLM to iteratively refine its answers through multi-turn feedback. The method is formulated as an episodic MDP with horizon H, and the actor and critic are trained with DPO-style preference learning on self-generated trajectories. The paper claims a theoretical guarantee (Theorem 1) that DPSDP competes with any policy under single-policy concentrability and bounded in-distribution reward-learning error, and it reports empirical improvements across Ministral, Llama-3.1, and Qwen2.5 on MATH 500, GSM8K, MMLU-Pro Math, and Olympiad Bench, with ablations on single-agent versus multi-agent, Markovian versus non-Markovian, restart versus non-restart data collection, and generative versus non-generative critics.

Significance. The paper addresses an important problem: coordinating multiple LLM agents for iterative answer refinement. Its strengths include a broad empirical study across three model families and four benchmarks, careful ablations, and a PSDP-inspired theoretical framework. If the theoretical guarantee applied to the practical algorithm as stated, the paper would be a valuable bridge between RL theory and multi-agent LLM training. However, as detailed below, the Q-value approximation used in the practical algorithm severs the link to Theorem 1, so the theoretical claim currently covers only an idealized version (Algorithm 1) that is not what is evaluated. The empirical results may stand on their own, but they need stronger statistical support and clarification of the evaluation protocol.

major comments (4)
  1. [Section 3.3 / Appendix C.3, Eq. (5)] The identity eQ^{π1}_0(s0,a0)=r(s1)+(H-1)/2=Q^{π*}_0(s0,a0) is not implied by the MDP defined in Section 2. For H=3, Q^{π*}_0(s0,a0)=r(s1)+E[r(s3)|s0,a0,π*], and the second term generally depends on a0 unless one assumes that the optimal critic/refiner can always eventually produce a correct answer from any initial state. This assumption is not stated or justified. Because Algorithm 2 builds the first-turn DPO dataset from this proxy, the actor is trained to maximize immediate correctness rather than the full return J(π). Consequently, Assumption 2, which concerns the true Q^{bπ}_h, is not satisfied by the evaluated method, and Theorem 1 does not cover Algorithm 2. The paper's own Table 4 shows 7.8% incorrect-to-correct and 4.0% correct-to-incorrect transitions at turn 2, so the proxy is not nearly exact. Please either prove a bound on the approximation error, state the additional 'perfect refinement' assumption explicitly, or restrict the theoretical claim to Algorithm 1 and present the empirical results as a separate contribution.
  2. [Section 3.1 / Algorithm 1 / Theorem 1] Algorithm 1 outputs 'the final policy bπ=bπ0', but the proof of Theorem 1 uses Q^{bπ}_h for all h and states J(π*)−J(bπ). If bπ is meant to be the non-stationary policy sequence (bπ_0,...,bπ_{H-1}), this must be stated explicitly; as written, the guarantee for a single first-turn policy does not follow. In Algorithm 2, the same actor is used at h=0 and h=2, which is a different object from the per-step independent policies in Algorithm 1. The theory should either model parameter sharing and the alternation of actor and critic, or state clearly which theoretical object the practical implementation corresponds to.
  3. [Section 4.2 / Section E.1] The evaluation rule 'questions with no more than two correct responses are considered incorrect' is inconsistent with the stated definition of pass1@turn5 as 'at least one answer is correct'. This rule makes the p1@t5 numbers non-standard and potentially misleading; the main text must state exactly which metric is reported and why this threshold is used. In addition, no error bars or multiple seeds are reported anywhere. The headline improvement on MATH 500 (e.g., from 57.2 to 63.2 for Ministral) is a single-run measurement. Please provide variance estimates or at least multiple seeds for the main experiments, and clarify whether the reported differences are statistically significant.
  4. [Section 4.1 / Section D.1] Training uses OpenMathInstruct-2, which the paper describes as 'sourced or augmented from MATH and GSM8K', the same benchmarks used for evaluation. The paper does not state whether the MATH 500 test problems or the GSM8K test problems were excluded during training or during the preliminary SFT phase. Given that the in-distribution gains are a headline result, this is a potential contamination risk. Please clarify the overlap between the training data and the evaluation sets, and describe any deduplication performed.
minor comments (4)
  1. [Throughout] There are several typos: 'Lemme C.5' should be 'Lemma C.5', 'Ourput' in Algorithm 2 should be 'Output', and 'and and' appears in the Related Work section.
  2. [Section 4.2] The metric definitions should be stated unambiguously: specify that the 'no more than two correct responses' rule applies to all numbers in Table 1, and note that this makes p1@t5 different from standard pass@k.
  3. [Algorithm 1] For h=H−1, the update uses Q^{bπ_H}_h, which is undefined because bπ_H does not exist; please specify the boundary condition (e.g., Q=0 after the horizon).
  4. [Section 3.3] The notation eQ for estimated Q-values and bπ for the learned policy is overloaded; consider defining a distinct symbol, such as \hat{Q}, for the estimated values.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-citation in the theoretical support; the conditional bound and empirical measurements are otherwise self-contained.

  1. other [Section 3.2, paragraph immediately after Assumption 2]
    "By Lemme C.5 of Xie et al. (2024), we can immediately infer that ε_stat is small with high probability for large dataset."

    This sentence is the paper's only support for asserting that the in-distribution error ε_stat in Assumption 2 is small. Xie et al. (2024) shares co-author Tengyang Xie, so it is a self-citation. However, Theorem 1 is stated conditionally on Assumption 2 and its proof uses the performance difference lemma plus concentrability; the bound does not define the conclusion as equal to the cited lemma. The self-citation is thus auxiliary support rather than a definitional reduction, and it does not force the reported empirical gains.

full rationale

The central derivation chain is not circular. Theorem 1 is a conditional PSDP guarantee: under concentrability (Assumption 1) and bounded in-distribution regression error (Assumption 2), the proof decomposes J(π*)−J(bπ) via the performance difference lemma and bounds the three resulting terms by O(H√(C_S C_A ε_stat)). Assumption 2 is an input, not a conclusion; the theorem does not equate the performance gap to the regression error by construction. The only self-referential element is the sentence 'By Lemme C.5 of Xie et al. (2024), we can immediately infer that ε_stat is small with high probability for large dataset,' where the cited work shares an author with the present paper. This citation is used to argue that Assumption 2 is achievable, but the theorem itself is conditional and its proof does not depend on the lemma, so the circularity is minor. The practical Q-value estimations in Section 3.3 are heuristics: eQ for the first turn is set proportional to r(s1)=I[a0=a*], and Appendix C.3 attempts a constant-shift justification. That is a theory-practice gap and a potential correctness weakness, not a circular reduction, because the Q estimators are not fitted to produce the reported accuracies and the empirical results are measurements on external benchmarks (MMLU-Pro Math, Olympiad Bench) in addition to MATH 500 and GSM8K. No fitted parameter is renamed as a prediction, and no known result is merely relabeled. Overall circularity is low.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests primarily on coverage and in-distribution error assumptions borrowed from prior RL theory, plus two ad hoc modeling choices (first-turn Q proxy and reduced Markovian state). No new physical or mathematical entities are introduced.

free parameters (3)
  • β (KL coefficient) = 0.1 for actors; 1.0 for Ministral critic; 0.1 for Llama/Qwen critics
    Selected by grid search over {0.1, 0.5, 1.0}; controls the strength of the KL regularization in the DPO objective and affects the trained policy.
  • Number of candidate responses n = 8
    Chosen by authors for data collection; affects exploration in constructing preference pairs.
  • DPO epochs per model = 1 for Ministral actor/critic and Llama actor; 2 for Llama critic; 3 for Qwen critic
    Selected during training; part of the hyperparameter search.
assumptions (5)
  • domain assumption Assumption 1: The reference policy π_ref covers the optimal policy's state and action distributions (C_S^*, C_A bounded).
    Invoked in Section 3.2 to prove Theorem 1; not verified for LLM refinement.
  • ad hoc to paper Assumption 2: The learned policy bπ has small in-distribution reward-learning error ε_stat.
    Defined in Section 3.2; the theorem bounds J(π*) - J(bπ) in terms of ε_stat, so the guarantee is conditional on the training objective being solved.
  • standard math Lemma C.5 of Xie et al. (2024) implies ε_stat is small with high probability for large datasets.
    Cited in Section 3.2; from the authors' own prior work, not reproven or machine-checked here.
  • ad hoc to paper The value of a first-turn answer is well approximated by its immediate correctness r(s1).
    Used in Section 3.3 for Q-value estimation at h=0; the paper claims only a small fraction of first answers change correctness, but Table 4 shows about 11.8% change at the second turn.
  • domain assumption Only the most recent answer and feedback are needed for refinement (Markovian state).
    Design choice in Section 3.3 that enables horizon generalization; the paper provides an empirical ablation but no formal justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforce LLM Reasoning through Multi-Agent Reflection." pith.science (2026). https://pith.science/paper/YWJEO7GM

@misc{pith2026250608379,
  author       = {Pith},
  title        = {Pith review of: Reinforce LLM Reasoning through Multi-Agent Reflection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YWJEO7GM}},
  note         = {Machine review of arXiv:2506.08379}
}
read the original abstract

Leveraging more test-time computation has proven to be an effective way to boost the reasoning capabilities of large language models (LLMs). Among various methods, the verify-and-improve paradigm stands out for enabling dynamic solution exploration and feedback incorporation. However, existing approaches often suffer from restricted feedback spaces and lack of coordinated training of different parties, leading to suboptimal performance. To address this, we model this multi-turn refinement process as a Markov Decision Process and introduce DPSDP (Direct Policy Search by Dynamic Programming), a reinforcement learning algorithm that trains an actor-critic LLM system to iteratively refine answers via direct preference learning on self-generated data. Theoretically, DPSDP can match the performance of any policy within the training distribution. Empirically, we instantiate DPSDP with various base models and show improvements on both in- and out-of-distribution benchmarks. For example, on benchmark MATH 500, majority voting over five refinement steps increases first-turn accuracy from 58.2% to 63.2% with Ministral-based models. An ablation study further confirms the benefits of multi-agent collaboration and out-of-distribution generalization.

Figures

Figures reproduced from arXiv: 2506.08379 by the authors.

Figure 1
Figure 1. Inference time. Given a problem x, the actor πa generates an initial response a0. The critic πc then provides feedback a1, identifying potential errors in a0. The actor iteratively refines its response based on the feedback, continuing this process for L rounds. Finally, majority voting is applied to all generated answers to determine the final response a˜. d π 0 denotes the distribution over initial states. Policy … view at source ↗
Figure 2
Figure 2. Model training. DPSDP first samples a complete trajectory τ = (x, a0, a1, a2) from the reference policy πref. At each state along this trajectory, it generates n responses to explore possible answers and feedback. Q-values of these n candidate responses are then estimated as in Section 3.3 and a pairwise preference dataset is extracted for subsequent DPO training on both the actor and critic. i.e., Qeπb2 1 (s1, a1) … view at source ↗
Figure 3
Figure 3. Various metrics under different turns. Accuracies improve as the number of refinements increases. The rising pass1@turn-k scores indicate that iterative refinement enables models to solve previously unsolved problems. Note that the decrease in maj1@t2 accuracy arises from the requirement that both responses (2 out of 2) must be correct to count toward maj1@t2. We further analyze how the proportion of responses that … view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Reviewer precision and critique uptake come apart: a more precise specialized reviewer can be less effective than a less precise shared-state discussion because correct critiques often do not change the next candidate.

  2. Intern-S1-MO: Long-horizon Reasoning Agent for Olympiad?Level Mathematical Problem Solving

    cs.CL 2025-12 conditional novelty 6.0 of 10

    A multi-round, lemma-memory reasoning agent with hierarchical RL reaches reported gold-medal-level scores on Olympiad math benchmarks, though the proof-based scores are self-graded.

Reference graph

Works this paper leans on

14 extracted references · 6 canonical work pages · cited by 2 Pith papers

  1. [1]

    Therefore,A bπ h(sh, ah) = eAbπ h(sh, ah)

    Ath= 2, the estimated eQbπ3 2 (s2, a2) =r(s 3)is exact. Therefore,A bπ h(sh, ah) = eAbπ h(sh, ah)

    work page

  2. [2]

    We define the approximation error: ∆ =E sh∼dπ⋆ h ,ah∼π⋆(·|sh)[Aˆπ h(sh, ah)− eAˆπ h(sh, ah)]

    Ath= 1, the estimatedQ-value is: eQbπ2 1 (s1, a1) =E a2∼πref (·|s2)[r(s3)] =Q πref 1 (s1, a1). We define the approximation error: ∆ =E sh∼dπ⋆ h ,ah∼π⋆(·|sh)[Aˆπ h(sh, ah)− eAˆπ h(sh, ah)]

    work page

  3. [3]

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela

    URLhttps://arxiv.org/abs/2305.1 4325. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization, 2024. URLhttps: //arxiv.org/abs/2402.01306. Shanghua Gao, Ada Fang, Yepeng Huang, Valentina Giunchiglia, Ayush Noori, Jonathan Richard Schwarz, Yasha Ektefaie, Jovana Kondic, and Mar...

    arXiv 2024

  4. [5]

    Edward Junprung

    URLhttps://openreview.net/forum ?id=9Rtm2gAVjo. Edward Junprung. Exploring the intersection of large lan- guage models and agent-based modeling via prompt en- gineering, 2023. URLhttps://arxiv.org/abs/ 2308.07411. Sham M. Kakade and John Langford. Approximately op- timal approximate reinforcement learning. InInterna- tional Conference on Machine Learning,...

    arXiv 2023

  5. [6]

    Arindam Mitra, Hamed Khanpour, Corby Rosset, and Ahmed Awadallah

    Accessed: 2025-01-14. Arindam Mitra, Hamed Khanpour, Corby Rosset, and Ahmed Awadallah. Orca-math: Unlocking the poten- tial of slms in grade school math, 2024. URLhttps: //arxiv.org/abs/2402.14830. Sumeet Ramesh Motwani, Chandler Smith, Rocktim Jyoti Das, Markian Rybchuk, Philip H. S. Torr, Ivan Laptev, Fabio Pizzati, Ronald Clark, and Christian Schroede...

    arXiv 2025

  6. [7]

    doi: 10.18653/v1/2024.naacl-long.327

    Association for Computational Linguistics. doi: 10.18653/v1/2024.naacl-long.327. URLhttps://ac lanthology.org/2024.naacl-long.327/. Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling parameters for reasoning. In Proceedings of the International Conference on Learn- ing...

    work page doi:10.18653/v1/2024.naacl-long.327 2024

  7. [9]

    Qbπ h(sh, ah)−βlog bπ(ah |s h) πref (ah |s h) +c(s h) 2# From Assumption 1, we have Esh∼dπ⋆ h ,ah∼π⋆(·|sh)

    URLhttps://proceedings.neurips. cc/paper_files/paper/2023/file/271db 9922b8d1f4dd7aaef84ed5ac703-Paper-Con ference.pdf. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Good- man. STar: Bootstrapping reasoning with reasoning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors,Advances in Neural Informa- tion Processing Systems, 2022....

    arXiv 2023

  8. [12]

    Therefore, Eah∼π⋆(·|sh)[Aˆπ h(sh, ah)]≈E ah∼π⋆(·|sh)[Aπ⋆ h (sh, ah)] = 0, where the last equality follows from the definition ofA π h

    Ath= 0, we have eQˆπ1 0 (s0, a0) =r(s 1) + H−1 2 =Q π⋆ 0 (s0, a0). Therefore, Eah∼π⋆(·|sh)[Aˆπ h(sh, ah)]≈E ah∼π⋆(·|sh)[Aπ⋆ h (sh, ah)] = 0, where the last equality follows from the definition ofA π h. Following steps in Section C.1, we obtain the approximate upper bound by adding|∆|to the theoretical bound. To assess the impact of|∆|, we performed an abl...

    work page 2023

Show all 14 references
  1. [13]

    We then conduct DPO training on the critic, producing a refined critic modelbπc

    Next, we sample diverse feedback from the base critic and usebπ a 2, instead ofπ ref as in unified training, to generate refined answers based on this feedback, obtaining an accurate value ofQ bπa 2 1 . We then conduct DPO training on the critic, producing a refined critic modelbπc

  2. [14]

    Finally, we improve the actor’s first-turn responses based onbπa 2, following the same approach as in Algorithm 2. To ensure broad exposure to different problems, we use three distinct problem sets at each stage of step-by-step training, each matching the size of the dataset u...

  3. [862]

    Justin Chih-Yao Chen, Archiki Prasad, Swarnadeep Saha, Elias Stengel-Eskin, and Mohit Bansal

    URLhttps://aclanthology.org/P05-1 022/. Justin Chih-Yao Chen, Archiki Prasad, Swarnadeep Saha, Elias Stengel-Eskin, and Mohit Bansal. Magicore: Multi-agent, iterative, coarse-to-fine refinement for rea- soning.arXiv preprint arXiv:2409.12147, 2024a. Wenhu Chen, Ming Yin, Max K...

    2023

  4. [2023]

    Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and R´emi Munos

    URLhttps://arxiv.org/abs/2305.0 8844. Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and R´emi Munos. A general theoretical paradigm to un- derstand learning from human preferences, 2023. URL https://arxiv.org/abs/2310.12036...

    2023 arXiv

  5. [2024]

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yu- jie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun

    URLhttps://arxiv.org/abs/2402.1 0963. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yu- jie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with oly...

    2024 arXiv

  6. [3021]

    URLhttps: //proceedings.neurips.cc/paper_files /paper/2020/file/1f89885d556929e98d3 ef9b86448f951-Paper.pdf

    Curran Associates, Inc., 2020. URLhttps: //proceedings.neurips.cc/paper_files /paper/2020/file/1f89885d556929e98d3 ef9b86448f951-Paper.pdf. Gokul Swamy, Christoph Dann, Rahul Kidambi, Zhi- wei Steven Wu, and Alekh Agarwal. A minimaximal- ist approach to reinforcement learning ...

    2020 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.