Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Mirage or Method? How Model-Task Alignment Induces Divergent RL Conclusions

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RL's counterintuitive LLM tricks work only on familiar tasks

desk verdict The alignment pattern is real and the green-cell no-contamination result is the contribution, but the post hoc cutoff and single-seed numbers keep the 'only when' claim from being fully pinned down. read the letter →

arxiv 2508.21188 v2 pith:PB5MNKNU submitted 2025-08-28 cs.LG cs.CL

Haoze Wu , Cheng Wang , Wenshuo Zhao , Junxian He This is my paper
classification cs.LGcs.CL
keywords reinforcementlearninglargelanguagemodelsmodel-taskalignmentpass@kspuriousrewardsone-shotnegativesampletrainingtest-timeRL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tests whether three celebrated RL-for-LLM results — robustness to spurious rewards, one-shot training sufficiency, and negative-sample-only training — are universal properties of reinforcement learning or side effects of the specific model-task pair being studied. Across two model families and math plus logical reasoning tasks, it finds the latter: each phenomenon appears only when the pretrained model already generates correct answers at a high rate, measured by pass@k, on the evaluated task. On unfamiliar tasks the same techniques do little, while standard RL with correct rewards keeps working. The conclusion matters because it separates capability elicitation from genuine learning, and it points to base-model proficiency as the lever that determines which RL recipes will work.

What carries the argument

The carrying instrument is Model-Task Alignment, operationalized as pass@k accuracy: for a problem, the unbiased estimate 1 − C(n−c_i, k)/C(n, k), averaged over the evaluation set. Pass@k curves are used to sort every model-task pair into 'strong' or 'weak' alignment, and a prompt-truncation contamination test (greedy completion scored by ROUGE-L and exact match) separately tags each pair as contaminated or clean. The resulting red/gray/green partition — strong+contaminated, weak+clean, strong+clean — is the lens through which every experiment is read: a technique's success is predicted by the cell the pair falls in, not by the technique alone.

What would settle it

Pre-register a pass@k cutoff, then run spurious-reward, one-shot, and negative-only training on pairs spanning intermediate pass@k values — e.g., Llama-3.1 on Counterfactual (pass@16 ≈ 0.6) and Qwen2.5 on Puzzle (pass@16 < 0.1). A single high-pass@k pair where these techniques fail, or an intermediate pair where they succeed, breaks the 'only when' claim; so would a low-pass@k task where a model with zero initial rollout accuracy nonetheless learns from one example.

Watch

Extended reading notes

Core claim

The paper's central claim is a dependency it calls Model-Task Alignment: the appearance of counterintuitive RL phenomena in LLMs is governed by how well the pretrained model already matches the task, quantified by pass@k — the probability that at least one of k sampled solutions is correct. Experiments on Qwen2.5-7B and Llama-3.1-8B across mathematical and logical reasoning benchmarks partition model-task pairs into three groups: potentially contaminated with strong alignment (Qwen on math), uncontaminated with strong alignment (both models on Operation and Counterfactual subsets of KOR-Bench), and uncontaminated with weak alignment (Llama on math, both models on the remaining logical tasks)

Load-bearing premise

The 'strong' vs 'weak' alignment split is assigned by visually inspecting pass@k on the same evaluation sets that later serve as RL outcome measures, with no pre-specified cutoff; move the cutoff and some cells would change groups, blurring the clean 'only when aligned' pattern.

Editorial extensions

If this is right

  • Ground-truth rewards remain the gold standard: standard RLVR outperforms all alternatives and works in both strong- and weak-alignment settings, so the counterintuitive results do not imply that reward quality is unimportant.
  • Spurious-reward robustness, one-shot sufficiency, and negative-only training act as capability-elicitation mechanisms that activate existing competence rather than teach new skills.
  • Contamination is not the explanation: the no-contamination strong-alignment cells (Operation, Counterfactual) exhibit the same phenomena as the potentially contaminated math cells.
  • Under weak alignment, positive-only training (PSR) consistently beats negative-only training (NSR), so the choice of which signal to keep matters precisely when the model is unfamiliar with the task.
  • One-shot RL generalizes within the subtask of the training example but not across puzzle types, so a single example does not build new reasoning ability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because pass@k is cheap to measure before any RL training, a pre-registered cutoff would turn this paper's three-color taxonomy into a screening test: estimate pass@k on a new domain and predict whether cheap signals (spurious rewards, one-shot, negative-only) are worth trying before spending training compute.
  • The paper's dichotomy suggests a resource-allocation rule the authors gesture at but do not formalize: spend pretraining or mid-training compute to push a base model's pass@k up on a target domain and cheap RL afterwards becomes viable; otherwise budget for standard RL with accurate rewards.
  • Pass@k is a continuum, yet the conclusions are cast as a binary partition; if the dependency is real, intermediate-alignment pairs should show graded behavior, and locating the transition (sharp threshold vs. smooth curve) is a direct testable extension the paper does not run.
  • The within-distribution generalization finding leaves open whether a small curriculum of examples, one per subtask, could extend one-shot-style gains across puzzle types — a bridge the paper's data neither confirms nor rules out.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper asks whether recently reported counterintuitive RL phenomena for LLM reasoning generalize, and proposes that model-task alignment, measured by pass@k accuracy on the evaluated task, is the key condition. The authors evaluate Qwen2.5-7B-Base and Llama-3.1-8B-Instruct across math and logic benchmarks, partitioning model-task cells into three groups: red (potential contamination + strong alignment), gray (no contamination + weak alignment), and green (no contamination + strong alignment). Experiments cover reward-signal quality (correct, random, incorrect, format, self-rewarded), test-time RL, one-shot RL, and negative-sample-only training. The central claim is that standard RL with correct rewards is robustly effective across settings, whereas spurious rewards, one-shot training, TTRL, and negative-only training produce large gains only in strong-alignment cells, and that alignment rather than contamination drives these effects.

Significance. If validated, the paper is a valuable corrective to several high-profile RLVR claims: it would show that robustness to noisy rewards, one-shot sufficiency, and negative-only training are capability-elicitation effects rather than general RL phenomena. The study's strengths include broad model-family and task coverage, an explicit contamination analysis that separates contamination from alignment, and released code. The main limitation is that the central conclusion depends on a post hoc binary partition of model-task cells and on single-run point estimates; until those are addressed, the 'only when strong alignment' claim is not conclusive.

major comments (3)
  1. [Section 2.2, Figure 2, Appendix B] The central independent variable is a binary split into strong/weak alignment, but the split is made after visually inspecting pass@k curves on the same evaluation sets later used to measure RL gains. No cutoff is pre-specified, no numeric pass@k table is given, and no threshold sensitivity analysis is reported. This is load-bearing: intermediate cells exist, e.g., Llama-3.1 on Counterfactual has pass@16 near 0.6 while Qwen2.5 on Puzzle is below 0.1. A different threshold could move several cells between categories and blur the claimed red/gray/green pattern. Please report numeric pass@k for every model-task cell and either pre-specify the cutoff or show that the qualitative conclusions in Tables 2, 5, and 6 are stable over a range of thresholds; alternatively, model alignment as a continuous variable.
  2. [Section 4.1, Table 2] The claim that 'spurious rewards consistently fail to provide meaningful improvements' in weak-alignment settings is contradicted by the gray SynLogic cell for Qwen2.5-7B: random reward improves accuracy from 1.5 to 10.2 (+8.7). In contrast, the green/strong-alignment cell Llama Counterfactual gains only +0.8 under random reward. If +8.7 is considered not meaningful, the paper needs an explicit criterion for 'meaningful'; if it is meaningful, the binary conclusion must be qualified. Either way, the current table does not support the blanket statement.
  3. [Tables 2-6] All RL results are single-run point estimates; no seed variation, confidence intervals, or significance tests are reported. Several conclusions depend on small deltas, e.g., Llama Counterfactual correct +2.4 vs random +0.8, and Llama SynLogic TTRL +0.3. With single runs, a few cells could move categories due to seed noise, which would change the qualitative pattern. Please report variance across multiple seeds or provide bootstrap/statistical evidence that the claimed cross-cell pattern is robust.
minor comments (5)
  1. [Section 3] Typo: 'max promt length' should be 'max prompt length'; same typo appears again in the same paragraph.
  2. [Table 2] Several entries appear corrupted or inconsistent with the baselines: e.g., Qwen Logic EM shows '6.8−37.6' from a baseline of 8.0, and Format shows '1.50.0'. Please verify and correct the table formatting and values.
  3. [Appendix B] The pass@k results are given only as plots. A numeric table of pass@16 or pass@64 values for each model-task cell would make the categorization reproducible and would support the proposed threshold analysis.
  4. [Section 3] The metric 'avg@8' is used without a formal definition. Please define how it is computed and how it differs from pass@8.
  5. [Table 2 and Section 4.1] The row labels 'V ote' and 'EM' are unclear at first reading; the text should define 'EM' as entropy minimization on self-generated samples and avoid the spacing artifact in 'V ote'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: model-task alignment (pass@k) is measured on base models before RL and RL deltas are independent outcomes; the post-hoc red/gray/green partition is a selection-bias concern, not a definitional reduction.

full rationale

The paper's central claim is empirical: counterintuitive RL phenomena occur in model-task cells where the pretrained model already has high pass@k. The independent variable (pass@k) is computed from base-model samples on the evaluation sets before any RL training (Section 2.1, Eq. 1; Figure 2; Appendix B), and the dependent variables (RL accuracy deltas under spurious rewards, one-shot training, NSR/PSR, TTRL) are measured from separately run RL experiments (Sections 4-6, Tables 2-6). Nothing in the paper's equations defines RL outcomes as a function of pass@k, nor is any parameter fitted to make the RL deltas match pass@k. The 'Model-Task Alignment Dependency' is explicitly framed as a hypothesis that 'may or may not hold' (Section 2.1) and is then tested, not assumed. The red/gray/green categorization is chosen after inspecting pass@k curves on the same evaluation sets, and no pre-registered cutoff or threshold sensitivity analysis is reported; this is a legitimate post-hoc thresholding/selection-bias threat to the strength of the 'only when' conclusion, but it does not make the conclusion true by construction. All RL results are single-run point estimates, which adds uncertainty, but again is not circularity. The only notable author-overlap citation is SynLogic [Liu et al., 2025], used as one of several logical-reasoning benchmarks; the alignment measure and RL outcomes do not reduce to SynLogic, and the weak-alignment gray category also includes BBH, BBEH, and KOR-Bench subsets, so the self-citation is not load-bearing. Score 1 reflects the minor post-hoc partition design rather than any definitional circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim depends on two measurement assumptions: that pass@k on the evaluation set reflects inherent ability, and that the contamination test can rule out leakage. Both are plausible but unvalidated proxies. The only number introduced ad hoc is the strong/weak pass@k boundary, which is not pre-specified. No new theoretical entities, forces, or dimensions are introduced; 'model-task alignment' is an operational label for existing pass@k measurements.

free parameters (1)
  • pass@k boundary separating strong from weak model-task alignment = not reported; appears to be roughly pass@16 in the 0.4 to 0.6 range from Figure 2
    The paper assigns model-task pairs to strong vs. weak alignment based on pass@k curves (Section 2.2) without stating a cutoff; this boundary is load-bearing for every RQ claim and is chosen post hoc from the same eval data used for outcome measurement.
assumptions (4)
  • domain assumption pass@k on the evaluation set is a valid measure of inherent model-task alignment
    Used to operationalize the central hypothesis (Section 2.1), but pass@k can be inflated by test-set contamination; the paper attempts to separate these via a prompt-truncation proxy.
  • domain assumption The prompt-truncation contamination test reliably detects data leakage
    Section 2.3 and Appendix C use ROUGE-L/EM on partial prompts with greedy decoding; ROUGE-L close to 1.0 indicates leakage, but low ROUGE may miss other forms of memorization.
  • domain assumption DAPO with the reported hyperparameters is a representative standard RL algorithm for LLM reasoning
    All experiments use DAPO (Section 3), and the claim that 'standard RL remains consistently robust' depends on this single algorithm and these training budgets.
  • domain assumption The chosen models (Qwen2.5-7B, Llama3.1-8B) and tasks (math, synthetic/logic benchmarks) represent the broader LLM reasoning RL landscape
    No frontier-scale models, no coding or agentic tasks, and only two model families; generalization beyond this scope is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mirage or Method? How Model-Task Alignment Induces Divergent RL Conclusions." pith.science (2026). https://pith.science/paper/PB5MNKNU

@misc{pith2026250821188,
  author       = {Pith},
  title        = {Pith review of: Mirage or Method? How Model-Task Alignment Induces Divergent RL Conclusions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PB5MNKNU}},
  note         = {Machine review of arXiv:2508.21188}
}
read the original abstract

Recent advances in applying reinforcement learning (RL) to large language models (LLMs) have led to substantial progress. In particular, a series of remarkable yet often counterintuitive phenomena have been reported in LLMs, exhibiting patterns not typically observed in traditional RL settings. For example, notable claims include that a single training example can match the performance achieved with an entire dataset, that the reward signal does not need to be very accurate, and that training solely with negative samples can match or even surpass sophisticated reward-based methods. However, the precise conditions under which these observations hold - and, critically, when they fail - remain unclear. In this work, we identify a key factor that differentiates RL observations: whether the pretrained model already exhibits strong Model-Task Alignment, as measured by pass@k accuracy on the evaluated task. Through a systematic and comprehensive examination of a series of counterintuitive claims, supported by rigorous experimental validation across different model architectures and task domains, our findings show that while standard RL training remains consistently robust across settings, many of these counterintuitive results arise only when the model and task already exhibit strong model-task alignment. In contrast, these techniques fail to drive substantial learning in more challenging regimes, where standard RL methods remain effective.

Figures

Figures reproduced from arXiv: 2508.21188 by the authors.

Figure 1
Figure 1. Model-task alignment, which is measured by pass@k accuracy on the evaluated task, drives [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pass@k for different tasks. Different LLMs have significantly different abilities on different [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The changes in two models’ accuracy during the training. If the initial rollout accuracy [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Entropy Dynamics of Qwen2.5-7B during Training. NSR can maintain the exploration [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Pass@k for math tasks. Qwen demonstrates strong capabilities across all three mathematical [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Pass@k for KOR-Bench. Both models demonstrate strong inherent reasoning capabilities [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Training Dynamics of Qwen2.5-7B when trained with [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs

    cs.LG 2026-01 conditional novelty 6.0 of 10

    Spurious RLVR makes Qwen2.5-Math retrieve memorized answers via a layer 18-20 anchor and layer 21+ adapters, a shortcut that can be steered by scaling specific MLP keys.

Reference graph

Works this paper leans on

24 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [4]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948,

    arXiv

  2. [6]

    Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, San- ket Vaibhav Mehta, Lalit K

    URLhttps://arxiv.org/abs/2403.07974. Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, San- ket Vaibhav Mehta, Lalit K. Jain, Virginia Aglietti, Disha Jindal, Peter Chen, Nishanth Dikkala, Gladys Tyen, Xin Liu, Uri Shalit, Silvia Chiappa, Kate Olszewska, Yi Tay, Vinh Q. Tran, Quoc V . Le, and Orhan Firat. Big-bench ex...

    arXiv

  3. [7]

    Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ra- masesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al

    URL https://arxiv.org/abs/2502.19187. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ra- masesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843–3857,

    arXiv

  4. [8]

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Erran Li, Raluca Ada Popa, and Ion Stoica

    URL https://arxiv.org/abs/2505.19641. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. Link,

    arXiv

  5. [9]

    Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, and Rui Yan

    Notion Blog. Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, and Rui Yan. The climb carves wisdom deeper than the summit: On the noisy rewards in learning to reason. arXiv preprint arXiv:2505.22653,

    arXiv

  6. [10]

    URL https://arxiv.org/ abs/2410.06526. 11 Meta. The llama 3 herd of models,

    arXiv

  7. [11]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al

    URL https://arxiv.org/abs/2407.21783. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730– 27744,

    arXiv

  8. [12]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D

    URL https://arxiv.org/abs/2412.15115. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model,

    arXiv

Show all 24 references
  1. [13]

    Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, et al

    URL https://arxiv.org/abs/2305.18290. Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, et al. Spurious rewards: Rethinking training signals in rlvr. arXiv preprint arXiv:2506.10947,

    arXiv

  2. [14]

    Mastering chess and shogi by self-play with a general reinforcement learning algorithm, 2017a

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Si- monyan, and Demis Hassabis. Mastering chess and shogi by self-play with a general reinfor...

    arXiv

  3. [17]

    Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, et al

    URL https://qwenlm.github.io/blog/qwq-32b/. Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, et al. Reinforcement learning for reasoning in large language models with one training example. arXiv preprint...

    arXiv

  4. [18]

    Reasoning or memorization? unreliable results of reinforcement learning due to data contamination

    Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Yanwei Fu, Qin Liu, et al. Reasoning or memorization? unreliable results of reinforcement learning due to data contamination. arXiv preprint arXiv:2507.10532,

  5. [19]

    URL https://arxiv.org/abs/2409.12122. 12 An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jian...

    arXiv

  6. [20]

    Xinyu Zhu, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, and Yu Meng

    URL https://arxiv.org/abs/2505.09388. Xinyu Zhu, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, and Yu Meng. The surprising effectiveness of negative reinforcement in llm reasoning,

    arXiv

  7. [21]

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving

    URLhttps://arxiv.org/abs/ 2506.01347. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593,

    1909

  8. [23]

    The format reward is different from that of Shao et al

    13 A Implementation Details Following the setting described in Section 3, we train with different rewards for 300 steps on mathematical and logical reasoning tasks, respectively. The format reward is different from that of Shao et al. [2025], we use the same template as SynLog...

    2025

  9. [24]

    observed that in mathematical tasks, employing ground truth rewards decreases the frequency of code usage in model responses. Their study also revealed that, in contrast to Qwen2.5-Math [Yang et al., 2024], the accuracy improvement of the Qwen2.5 Base model was primarily attri...

    2024

  10. [1998]

    Challenging big-bench tasks and whether chain-of-thought can solve them

    Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261,

    arXiv

  11. [2019]

    Ttrl: Test-time reinforcement learning

    Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, et al. Ttrl: Test-time reinforcement learning. arXiv preprint arXiv:2504.16084,

    arXiv

  12. [2021]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al

    URL https://arxiv.org/abs/2103.03874. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720,

    arXiv

  13. [2022]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599,

    arXiv

  14. [2023]

    Jiangjie Chen, Qianyu He, Siyu Yuan, Aili Chen, Zhicheng Cai, Weinan Dai, Hongli Yu, Qiying Yu, Xuefeng Li, Jiaze Chen, Hao Zhou, and Mingxuan Wang

    [Accessed 26-08-2025]. Jiangjie Chen, Qianyu He, Siyu Yuan, Aili Chen, Zhicheng Cai, Weinan Dai, Hongli Yu, Qiying Yu, Xuefeng Li, Jiaze Chen, Hao Zhou, and Mingxuan Wang. Enigmata: Scaling logical reasoning in large language models with synthetic verifiable puzzles,

    2025

  15. [2024]

    [Accessed 26-08-2025]. AMC. Art of Problem Solving — artofproblemsolving.com. https://artofproblemsolving.com/ wiki/index.php/AMC_12_Problems_and_Solutions,

    2025

  16. [2025]

    Yanjun Chen, Dawei Zhu, Yirong Sun, Xinghao Chen, Wei Zhang, and Xiaoyu Shen

    URL https://arxiv.org/abs/ 2505.19914. Yanjun Chen, Dawei Zhu, Yirong Sun, Xinghao Chen, Wei Zhang, and Xiaoyu Shen. The accuracy paradox in rlhf: When better reward models don’t yield better language models. arXiv preprint arXiv:2410.06554,

    arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.