Pith. sign in

REVIEW 3 major objections 5 minor 5 cited by

Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces MathIF, a 420-question math benchmark with verifiable constraints, and argues that the training methods that make large reasoning models smarter also make them worse at following simple user instructions, with the gap…

desk verdict Useful new math-domain instruction-following benchmark, but the headline trade-off claim is partly a scoring artifact: exact-match correctness against unconstrained references penalizes models that correctly follow surface-form constraints. read the letter →

arxiv 2505.14810 v2 pith:IOWNV3U6 submitted 2025-05-20 cs.CL cs.AI

Tingchen Fu , Jiawei Gu , Yafu Li , Xiaoye Qu , Yu Cheng This is my paper
classification cs.CLcs.AI
keywords instructionfollowinglargereasoningmodelschain-of-thoughtmathematicalbenchmarkreinforcementlearningcontrollabilityMathIF
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces MathIF, a benchmark of 420 math problems wrapped in one to three verifiable formatting, length, language, and affix constraints, and uses it to test 23 large reasoning models (LRMs). Its central finding is that the training recipes that make models reason better—supervised fine-tuning on distilled long chains of thought and reinforcement learning with outcome rewards—also make them worse at obeying simple user directives, and the effect worsens as generated reasoning gets longer. The authors argue this reflects a real trade-off between reasoning strength and controllability, and they show that a trivial intervention, restating the instruction at the end of the chain of thought, recovers some obedience at a measurable cost in math accuracy. If true, the result puts instruction adherence on the critical path for evaluating and training reasoning models.

What carries the argument

MathIF itself carries the argument. It defines 15 Python-verifiable constraints in four categories (length, lexical, format, affix), composes them into 30 double- and 15 triple-constraint prompts, and applies them to 420 problems drawn from GSM8K, MATH-500, Minerva, Olympiad, and AIME. Two metrics are defined per query: hard accuracy (all constraints satisfied) and soft accuracy (fraction of constraints satisfied), computed by exact string checks rather than a judge model; math correctness is scored by exact match to the unconstrained ground truth. The paper's controlled experiments use budget forcing, rollout-length capping in GRPO, and a 'repeat the instruction' intervention to move the distance between instruction and answer.

What would settle it

Re-score MathIF with a judge that accepts any mathematically correct answer satisfying the stated constraints, regardless of exact match; if the accuracy gap between constrained and unconstrained runs disappears or inverts, the reasoning–obedience trade-off is an artifact of string matching. Alternatively, find any long-CoT SFT or outcome-RL run that improves both math accuracy and instruction adherence at fixed length, which would break the claimed monotone trade-off.

Watch

Extended reading notes

Core claim

The paper's claim, stated in its own terms, is that scaling reasoning capacity does not guarantee control: across the MathIF benchmark, most LRMs fail to follow most user constraints (best model 50.71% hard accuracy), correctness drops when constraints are added, and both reasoning-oriented SFT and outcome-reward RL consistently lower instruction-following accuracy relative to the base model. The degradation tracks chain-of-thought length: longer CoTs correlate with lower hard and soft accuracy, budget-forcing more 'thinking' monotonically reduces instruction adherence, and capping response length during RL preserves obedience at the cost of math performance. Restating the constraint after the CoT improves compliance but reduces correctness. The authors interpret this as mutual interference between intelligence and obedience during both training and inference.

Load-bearing premise

The paper assumes that a math answer is correct only if it exactly matches the reference string from the unconstrained problem, even when the prompt demands a different language, casing, wrapper, or format, so a mathematically right and instruction-compliant answer that is phrased differently counts as a reasoning failure.

Editorial extensions

If this is right

  • MathIF can serve as a standard, judge-free evaluation target for instruction adherence in math reasoning, enabling precise comparison of training recipes.
  • Reasoning-oriented SFT and RL should be audited for instruction-following as a primary axis, not as a side effect.
  • Longer chain-of-thought training runs an alignment cost that grows with length; length capping is one lever, but it trades away reasoning gains.
  • Restating the constraint at the end of the CoT is a cheap, transferable intervention that improves obedience at a predictable accuracy cost.
  • Model scale alone does not buy instruction adherence; architecture and training pathway matter more, so leaderboards should report both axes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is contextual distance, then instruction-following could be improved by architectural attention that keeps the instruction active across long generations, not by training-time trade-offs alone.
  • The benchmark's exact-match correctness rule may penalize instruction-compliant answers that are mathematically right but differently phrased, so the reported trade-off could be partly a measurement artifact; a judge-based re-scoring would test this.
  • The same tension may appear in non-math reasoning domains, such as code generation with style constraints or agentic tool use, where long reasoning traces are also standard.
  • The finding suggests an 'obedience cost' of test-time scaling: any method that pushes models to think longer should be evaluated for constraint adherence, not just final accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MathIF, a benchmark of 420 mathematical reasoning problems with Python-verifiable instruction-following constraints (15 constraint types; single, double, and triple compositions). It evaluates 23 instruction-tuned and reasoning-oriented models, reporting low instruction-following accuracy across the board, a negative association between chain-of-thought length and instruction adherence, and a claimed trade-off between reasoning performance and instruction-following. The paper also presents controlled training experiments (SFT, cold-RL, SFT+RL, format reward) and inference-time interventions (budget forcing, repeating the instruction) to support the trade-off narrative.

Significance. If the empirical claims hold, MathIF fills a real gap: there is no dedicated, verifiable benchmark for instruction-following in reasoning models, and the observation that reasoning-oriented training can reduce instruction adherence is an important constraint on how LRMs are trained and evaluated. The design has genuine strengths: the constraints are Python-verifiable rather than judge-based, the data cover diverse difficulty levels, and code/data are released. The training comparisons against base models are a useful contribution. However, the stronger 'fundamental tension' and 'less controllable' conclusions rest on a correctness-with-constraints metric that is currently contaminated by exact-match scoring, so the central interpretive claim needs to be re-evidenced before the paper is accepted.

major comments (3)
  1. [Section 3, Table 3] The correctness metric is defined as whether the final answer 'exactly matches the ground-truth, regardless of constraint satisfaction.' For several MathIF constraints (e.g., 'Wrap your entire response with double quotation marks,' 'First repeat the request word for word,' 'Finish your response with this exact phrase,' 'all lowercase letters'), a fully compliant response necessarily has a different surface form from the unconstrained reference, so it will be scored as a reasoning error even when the mathematics is correct. This contaminates the Diff column of Table 3, the (Correct, Followed) cell in Figure 4, and the Correctness column of Table 6 (e.g., +repeat raises HAcc from 17.14 to 21.66 but lowers 'correctness' from 31.67 to 22.38, which may reflect the added prefix rather than a loss of reasoning). The claim that following instructions costs reasoning performance is therefore partly a scoring artifact. I request a constraint-aware correctness score (e.g., strip wrappers/prefixes/suffixes or match the mathematical content only) and a recomputation of the trade-off analyses with that score.
  2. [Section 5.2, Table 4] The central causal evidence that reasoning-oriented training degrades instruction-following rests on small HAcc/SAcc differences: e.g., Qwen2.5-1.5B HAcc drops from 10.00 to 7.86, and Qwen2.5-Math-7B SAcc drops from 23.53 to 21.03. With n=420, differences of 2-3 points are plausibly within sampling noise, yet no variance, confidence intervals, or significance tests are reported. The 'consistently drop across the board' claim needs either statistical support or a discussion of effect size; otherwise the trade-off conclusion is not yet established by the training experiments.
  3. [Section 5.1 (Figure 6) and Section 5.3] The statement 'Longer CoTs Impair Instruction Following' is supported primarily by Figure 6, which bins responses by CoT length. This is confounded by problem difficulty: harder problems elicit both longer CoTs and lower instruction compliance. The budget-forcing experiment in Figure 7 is a better causal probe, but it is run on a single small model and the repeated 'Wait' tokens may disrupt instruction-following through a mechanism other than length. I ask that the causal language be softened or that the analysis be re-run controlling for problem difficulty (or on a matched subset), and that the budget-forcing intervention be tested on at least a second model.
minor comments (5)
  1. [Section 3] Section 3 does not specify how the 'final answer' is extracted from a long CoT response (e.g., after </think> or as the last line); please define the extraction procedure so the correctness results are reproducible.
  2. [Figures 2 and 3] Figures 2 and 3 average over 23 models without confidence intervals; reporting per-model variance or bootstrap intervals would support the comparisons between subsets and constraint counts.
  3. [Table 8] Table 8 has a misaligned header: 'single double triple' followed by 'Acc HAcc SAcc HAcc SAcc' should be restructured so it is clear that Acc refers to the single-constraint subset; also 'SimplRL' in Tables 9-10 should be 'SimpleRL'.
  4. [Abstract] The abstract's 'consistent tension' phrasing overstates Table 4, where several training-induced changes are within 1-2 points; consider wording such as 'frequent degradation' unless the significance analysis requested above changes the conclusion.
  5. [Section 4] The decoding temperature T=1.0 in Section 4 may not match the recommended setting for some reasoning models; a greedy-decoding sensitivity check would make the benchmark numbers more robust.

Circularity Check

1 steps flagged · score 6.0 of 10

The 'obedience costs reasoning' half of the advertised trade-off is partly self-definitional: correctness is scored by exact match to the unconstrained ground truth, so compliant responses to surface-form constraints (quotes, lowercase, fixed suffix, repeat-the-request) are counted as reasoning errors by construction; the primary HAcc/SAcc instruction-following findings remain independent…

  1. self definitional [Section 3 (Evaluation Metric), propagated to Table 3 Diff column, Figure 4, Table 6, and Section 5.1-5.3]
    "Apart from instruction-following ability, we also measure the correctness of the math problem solution on our proposed MathIF, defined as whether the final answer exactly matches the ground-truth, regardless of constraint satisfaction. ... Table 1 (Affix): "First repeat the request word for word without change, then give your answer." / "Wrap your entire response with double quotation marks.""

    Many MathIF constraints are surface-form instructions (Table 1): wrap the entire response in double quotation marks, all-lowercase, finish with a fixed suffix, first repeat the request word for word, or answer in Chinese. With correctness defined as exact string match to the unconstrained ground truth, any model that complies with such a constraint produces a final answer whose string necessarily differs from the reference; compliance and 'correct' are mutually exclusive by construction. Hence the negative Diff column in Table 3, the large (Incorrect, Followed) cells in Figure 4, and the +repeat correctness drops in Table 6 (31.67 to 22.38; 51.90 to 30.00) are partly theorems of the metric.

full rationale

MathIF's instruction-following scores (HAcc/SAcc) are direct Python-verifiable constraint checks, not outputs of any fitted quantity; the 23-model comparison uses external checkpoints; the training study compares SFT/RL variants against their base checkpoints on external math benchmarks; and the budget-forcing and rollout-length experiments measure SAcc/HAcc directly. Self-citations ([1], [11], [21]) appear only as background or as a RoPE scaling convention and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. The single circular element is the correctness half of the trade-off claim. Correctness is defined as 'whether the final answer exactly matches the ground-truth, regardless of constraint satisfaction,' and a substantial subset of MathIF constraints (wrap-in-quotes, all-lowercase, fixed suffix, repeat-the-request prefix, section/bullet formatting, non-English language) changes the response's surface form on compliance, so exact-match labels compliant answers incorrect by construction. The negative Diff column in Table 3, the (Incorrect, Followed) proportion in Figure 4, and the +repeat correctness drops in Table 6 therefore partly encode the scoring definition rather than a genuine interference, and Section 5.1's dilemma reading ('more likely to fail in problem-solving when attempting to follow constraints') inherits that artifact. This is partial, not total, circularity: content-level constraints may still show real costs, and the paper's primary empirical regularity, that long-CoT SFT and outcome-RL degrade HAcc/SAcc while longer CoTs and budget forcing lower SAcc, stands on measurements that do not reduce to the metric. Score 6 reflects a central interpretive claim partially forced by construction, with the benchmark's core measurements otherwise self-contained.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted constants appear in the benchmark or the measurements. The load-bearing assumptions are all measurement or generalization assumptions: exact-match correctness under constraints, single-sample point estimates, small per-subset samples, and the representativeness of one RL recipe. These are domain assumptions rather than ad hoc entities.

assumptions (4)
  • domain assumption Exact string matching of the final answer against the unconstrained ground truth is a valid measure of math correctness even when the prompt imposes formatting, language, prefix, suffix, or wrapping constraints.
    Used to compute correctness with constraints and the Diff column in Table 3. A compliant answer that wraps or repeats text may fail exact match despite correct math.
  • domain assumption A single nucleus-sampled generation at T=1.0 per model and query is representative enough to rank instruction-following performance.
    Section 4 specifies one decoding run with T=1.0 and p=0.95, and all reported accuracies are point estimates without confidence intervals or multiple seeds.
  • domain assumption Per-subset sample sizes of 60 to 90 problems are adequate for the fine-grained comparisons reported in Tables 8 to 10.
    The paper reports per-source and per-constraint-count accuracies without statistical uncertainty, so small differences may be within binomial noise.
  • domain assumption GRPO training on the DeepScaler dataset with the reported hyperparameters is representative of reasoning-oriented training generally.
    Section 5.2 uses one RL algorithm and one dataset to support the general claim that reasoning-oriented training degrades instruction adherence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models." pith.science (2026). https://pith.science/paper/IOWNV3U6

@misc{pith2026250514810,
  author       = {Pith},
  title        = {Pith review of: Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IOWNV3U6}},
  note         = {Machine review of arXiv:2505.14810}
}
read the original abstract

Instruction-following is essential for aligning large language models (LLMs) with user intent. While recent reasoning-oriented models exhibit impressive performance on complex mathematical problems, their ability to adhere to natural language instructions remains underexplored. In this work, we introduce MathIF, a dedicated benchmark for evaluating instruction-following in mathematical reasoning tasks. Our empirical analysis reveals a consistent tension between scaling up reasoning capacity and maintaining controllability, as models that reason more effectively often struggle to comply with user directives. We find that models tuned on distilled long chains-of-thought or trained with reasoning-oriented reinforcement learning often degrade in instruction adherence, especially when generation length increases. Furthermore, we show that even simple interventions can partially recover obedience, though at the cost of reasoning performance. These findings highlight a fundamental tension in current LLM training paradigms and motivate the need for more instruction-aware reasoning models. We release the code and data at https://github.com/TingchenFu/MathIF.

Figures

Figures reproduced from arXiv: 2505.14810 by the authors.

Figure 1
Figure 1. Performance of Instruct LLMs and LRMs on IFEval [12] and FollowBench [13]. Overview As shown in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The accuracy on each subset MathIF averaged over 23 LRMs. We first scrutinize the model performance on each subset and visualize the average accuracy of 23 LRMs in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The HAcc (solid line) and SAcc (dashed line) on the single/double/triple￾constraint subset. Next, we investigate the impact of the constraint number and plot the instruction-following accuracy of three LRMs in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Error set analysis for Qwen3-0.6B, DeepSeek-R1-Distill-Qwen-1.5B, Open-Reasoner-Zero-7B, and DeepSeek-R1-Distill-Llama-8B (from left to right). 5 When Scaling Reasoning Meets Losing Control As discussed in Section 4.1, there may exist a trade-off between the instructio…
Figure 5
Figure 5. Figure 5: Relative correctness drop of four LRMs across five subsets [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: HAcc (solid line) and SAcc (dashed line) across six CoT length bins; higher indices correspond to longer CoT gen￾erations. Longer CoTs Impair Instruction Following. We fur￾ther analyze the impact of CoT length on instruction￾following performance. Specifically, for eac…
Figure 7
Figure 7. Figure 7: The trend of SAcc variation on GSM8K subset as the number of budget forc￾ing increases from 2 to 8. The More Thinking, the Less Following. We begin by artificially increasing the CoT length using budget forc￾ing [14], which appends the token "Wait" each time the model …

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reasoning Up the Instruction Ladder for Controllable Language Models

    cs.CL 2025-10 conditional novelty 6.0 of 10

    RLVR on ~7K verifiable system/user conflict examples teaches LLMs to prioritize higher-priority instructions, improving instruction-hierarchy and safety benchmarks.

  2. Structured Thoughts For Improved Reasoning And Context Pruning

    cs.CL 2026-07 conditional novelty 5.5 of 10

    Structured try/outcome SFT improves math reasoning by up to 8% over standard SFT and enables pruning ~85% of context with ~9% accuracy drop.

  3. AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning

    cs.CL 2025-07 conditional novelty 5.0 of 10

    AutoTIR applies GRPO with a hand-designed action reward so a 7B instruct model learns to mix search and code tools, beating tool-using baselines on ten benchmarks.

  4. Activation Steering for Chain-of-Thought Compression

    cs.AI 2025-07 conditional novelty 5.0 of 10

    A single steering vector extracted from paired verbose and concise rationales compresses chain-of-thought output at inference time without retraining.

  5. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Reference graph

Works this paper leans on

54 extracted references · 10 canonical work pages · cited by 5 Pith papers

  1. [1]

    A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond.arXiv preprint arXiv:2503.21614, 2025

    Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, et al. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond.arXiv preprint arXiv:2503.21614, 2025

    arXiv 2025

  2. [2]

    Introducing openai o3 and o4-mini

    OpenAI. Introducing openai o3 and o4-mini. https://openai.com/index/ introducing-o3-and-o4-mini/

    work page

  3. [3]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    2025

  4. [4]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang, Haotian Yao, Haotian Zhao, Haoyu Lu, Haoze Li, Haoz...

    2025

  5. [5]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems, 2024

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems, 2024

    2024

  6. [6]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    arXiv 2021

  7. [7]

    Aime problem set 1983-2024, 2023

    Hemish Veeraboina. Aime problem set 1983-2024, 2023

    1983

  8. [8]

    Z. Z. Ren, Zhihong Shao, Junxiao Song, Huajian Xin, Haocheng Wang, Wanjia Zhao, Liyue Zhang, Zhe Fu, Qihao Zhu, Dejian Yang, Z. F. Wu, Zhibin Gou, Shirong Ma, Hongxuan Tang, Yuxuan Liu, Wenjun Gao, Daya Guo, and Chong Ruan. Deepseek-prover-v2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition, 2025

    2025

Show all 54 references
  1. [9]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  2. [10]

    Crossing the reward bridge: Expanding rl with verifiable rewards across diverse domains, 2025

    Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, and Dong Yu. Crossing the reward bridge: Expanding rl with verifiable rewards across diverse domains, 2025

    2025

  3. [11]

    A survey on llm-as-a-judge, 2025

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. A survey on llm-as-a-judge, 2025

    2025

  4. [12]

    Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    2023 arXiv

  5. [13]

    Followbench: A multi-level fine-grained constraints following benchmark for large language models.arXiv preprint arXiv:2310.20410, 2023

    Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. Followbench: A multi-level fine-grained constraints following benchmark for large language models.arXiv preprint arXiv:2310.20410, 2023. 11

    arXiv

  6. [14]

    s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  7. [15]

    Limo: Less is more for reasoning, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025

    2025

  8. [16]

    Demystifying long chain-of-thought reasoning in llms, 2025

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms, 2025

    2025

  9. [17]

    SFT memorizes, RL generalizes: A comparative study of foundation model post-training

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of foundation model post-training. InThe Second Conference on Parsimony and Learning (Recent Spotlight Track), 2025

    2025

  10. [18]

    There may not be aha moment in r1-zero-like training — a pilot study

    Zichen Liu, Changyu Chen, Wenjun Li, Tianyu Pang, Chao Du, and Min Lin. There may not be aha moment in r1-zero-like training — a pilot study. https://oatllm.notion.site/ oat-zero, 2025. Notion Blog

    2025

  11. [19]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  12. [20]

    Process reinforcement through implicit rewards.arXiv preprint arXiv:2502.01456, 2025

    Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards.arXiv preprint arXiv:2502.01456, 2025

    2025 arXiv

  13. [21]

    Learning to reason under off-policy guidance, 2025

    Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, and Yue Zhang. Learning to reason under off-policy guidance, 2025

    2025

  14. [22]

    Thinking preference optimization.arXiv preprint arXiv:2502.13173, 2025

    Wang Yang, Hongye Jin, Jingfeng Yang, Vipin Chaudhary, and Xiaotian Han. Thinking preference optimization.arXiv preprint arXiv:2502.13173, 2025

    2025 arXiv

  15. [23]

    Hashimoto

    Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback, 2023

    2023

  16. [24]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023

    2023

  17. [25]

    FOFO: A benchmark to evaluate LLMs’ format-following capability

    Congying Xia, Chen Xing, Jiangshu Du, Xinyi Yang, Yihao Feng, Ran Xu, Wenpeng Yin, and Caiming Xiong. FOFO: A benchmark to evaluate LLMs’ format-following capability. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Associ...

    2024

  18. [26]

    Xiangru Tang, Yiming Zong, Jason Phang, Yilun Zhao, Wangchunshu Zhou, Arman Cohan, and Mark Gerstein. Struc-bench: Are large language models good at generating complex structured tabular data? In Kevin Duh, Helena Gomez, and Steven Bethard, editors,Proceedings of the 2024 Conf...

    2024

  19. [27]

    Multi-if: Benchmarking llms on multi-turn and multilingual instructions following.arXiv preprint arXiv:2410.15553, 2024

    Yun He, Di Jin, Chaoqi Wang, Chloe Bi, Karishma Mandyam, Hejia Zhang, Chen Zhu, Ning Li, Tengyu Xu, Hongjiang Lv, et al. Multi-if: Benchmarking llms on multi-turn and multilingual instructions following.arXiv preprint arXiv:2410.15553, 2024

    arXiv

  20. [28]

    Structflowbench: A structured flow benchmark for multi-turn instruction following.arXiv preprint arXiv:2502.14494, 2025

    Jinnan Li, Jinzhe Li, Yue Wang, Yi Chang, and Yuan Wu. Structflowbench: A structured flow benchmark for multi-turn instruction following.arXiv preprint arXiv:2502.14494, 2025

    2025 arXiv

  21. [29]

    Can language models follow multiple turns of entangled instructions?arXiv preprint arXiv:2503.13222, 2025

    Chi Han. Can language models follow multiple turns of entangled instructions?arXiv preprint arXiv:2503.13222, 2025. 12

    2025

  22. [30]

    Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms.arXiv preprint arXiv:2501.17399, 2025

    Ved Sirdeshmukh, Kaustubh Deshpande, Johannes Mols, Lifeng Jin, Ed-Yeremai Cardona, Dean Lee, Jeremy Kritz, Willow Primack, Summer Yue, and Chen Xing. Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms.arXiv preprint arXiv:250...

    2025 arXiv

  23. [31]

    Lifbench: Evaluating the instruction following performance and stability of large language models in long-context scenarios.arXiv preprint arXiv:2411.07037, 2024

    Xiaodong Wu, Minhao Wang, Yichen Liu, Xiaoming Shi, He Yan, Xiangju Lu, Junmin Zhu, and Wei Zhang. Lifbench: Evaluating the instruction following performance and stability of large language models in long-context scenarios.arXiv preprint arXiv:2411.07037, 2024

    2024 arXiv

  24. [32]

    Xifbench: Evaluating large language models on multilingual instruction following

    Zhenyu Li, Kehai Chen, Yunfei Long, Xuefeng Bai, Yaoyin Zhang, Xuchen Wei, Juntao Li, and Min Zhang. Xifbench: Evaluating large language models on multilingual instruction following. arXiv preprint arXiv:2503.07539, 2025

    2025

  25. [33]

    IHEval: Evaluating language models on following the instruction hierarchy

    Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, and Meng Jiang. IHEval: Evaluating language models on following the instruction hierarchy. In Luis Chiruzzo, Alan R...

    2025

  26. [34]

    Chain-of-instructions: Compositional instruction tuning on large language models

    Shirley Anugrah Hayati, Taehee Jung, Tristan Bodding-Long, Sudipta Kar, Abhinav Sethy, Joo-Kyung Kim, and Dongyeop Kang. Chain-of-instructions: Compositional instruction tuning on large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39...

    2025

  27. [35]

    RefuteBench: Evaluating refuting instruction-following for large language models

    Jianhao Yan, Yun Luo, and Yue Zhang. RefuteBench: Evaluating refuting instruction-following for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 13775–13791, Bangkok, Th...

    2024

  28. [36]

    Refutebench 2.0 – agentic benchmark for dynamic evaluation of llm responses to refutation instruction, 2025

    Jianhao Yan, Yun Luo, and Yue Zhang. Refutebench 2.0 – agentic benchmark for dynamic evaluation of llm responses to refutation instruction, 2025

    2025

  29. [37]

    Benchmarking complex instruction-following with multiple constraints composition.Advances in Neural Information Processing Systems, 37:137610– 137645, 2024

    Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxing Xu, et al. Benchmarking complex instruction-following with multiple constraints composition.Advances in Neural Information Processing Systems, 37:137610– 137645, 2024

    2024

  30. [38]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  31. [39]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    arXiv

  32. [40]

    Minerva: Accelerating data analysis in next-generation ssds

    Arup De, Maya Gokhale, Rajesh Gupta, and Steven Swanson. Minerva: Accelerating data analysis in next-generation ssds. In2013 IEEE 21st Annual International Symposium on Field-Programmable Custom Computing Machines, pages 9–16. IEEE, 2013

    2013

  33. [41]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. InProceedings of the ACM SIGOPS 29th Symposium on Operating System...

    2023

  34. [42]

    Qwen3, April 2025

    Qwen Team. Qwen3, April 2025

    2025

  35. [43]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild.arXiv preprint arXiv:2503.18892, 2025

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild.arXiv preprint arXiv:2503.18892, 2025. 13

    2025 arXiv

  36. [44]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement.arXiv preprint arXiv:2409.12122, 2024

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via se...

    2024 arXiv

  37. [45]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl, 2025. Notion Blog

    2025

  38. [46]

    L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    2025 arXiv

  39. [47]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025

    2025 arXiv

  40. [48]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025

    2025

  41. [49]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025

    2025

  42. [50]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025

    2025

  43. [51]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    2024

  44. [52]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  45. [53]

    The impact of reasoning step length on large language models

    Mingyu Jin, Qinkai Yu, Dong Shu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, and Mengnan Du. The impact of reasoning step length on large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguisti...

    2024

  46. [2025]

    Association for Computational Linguistics

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.