Pith. sign in

REVIEW 5 major objections 5 minor 81 references

Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Post-training can teach language models to scale reasoning length with problem difficulty, cutting token use up to roughly 30% without sacrificing accuracy.

desk verdict The core idea—difficulty-conditioned CoT compression with SFT+DPO—is reasonable, but the paper's own Table 2 undermines the accuracy-maintenance claim and the 'proportional thinking' effect is never directly measured. read the letter →

arxiv 2509.05226 v1 pith:5RN62FYY submitted 2025-09-05 cs.CL

Abdul Waheed , Chancharik Mitra , Laurie Z. Wang , Deva Ramanan , Bhiksha Raj This is my paper
classification cs.CL
keywords chain-of-thoughtdifficulty-awarereasoningefficiencydirectpreferenceoptimizationknowledgedistillationmathematicalmultimodaltoken
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that language models can be taught to match reasoning effort to problem difficulty without any architectural change. It builds a dataset by scoring math problems on a 1–10 difficulty scale and condensing long chain-of-thought traces so that easier problems receive shorter traces. Student models are post-trained with supervised fine-tuning followed by direct preference optimization, and this combination yields models that use fewer reasoning tokens while keeping or improving accuracy. If true, this matters because the cost and latency of reasoning models scale with generated tokens, and fixed-length chain-of-thought wastes compute on easy problems. The paper's broader claim is that thinking proportionally is learnable behavior, not a design constraint.

What carries the argument

The load-bearing object is the difficulty-aware compression function s(r_long, d(x)): a prompt-based summarizer that takes a long teacher reasoning trace and a difficulty score and emits a compressed trace whose length is intended to be proportional to d(x). The paper then turns these compressed traces into preferred responses and the original long traces into rejected responses, training students first by imitation (SFT) and then by preference optimization (DPO). This two-stage mechanism is what carries the claim, because the difficulty-proportional behavior is injected entirely through the curated training data.

What would settle it

Take a held-out set of math problems with independent human difficulty ratings, run the trained SFT+DPO models, and check whether average generated token count rises monotonically with human-rated difficulty; if it does not, or if the GPT-4o-mini ratings disagree strongly with human ratings on the training data, the proportional-thinking claim collapses.

Watch

Extended reading notes

Core claim

The central claim is that post-training alone, with no architectural modification, can make an LLM allocate reasoning length according to problem difficulty. The evidence comes from training student models on difficulty-compressed traces: on unimodal math benchmarks, models trained with SFT followed by DPO match or exceed base accuracy while using about 8–10% fewer tokens; on multimodal visual-math benchmarks the combined model reduces reasoning tokens by up to roughly 30% while gaining accuracy over its teacher. The authors also report a functional division: SFT transfers the compressed trace format and brevity, DPO preserves and improves reasoning accuracy, and their combination produces c

Load-bearing premise

The whole pipeline depends on the difficulty grader's scores being accurate and on the summarizer actually writing shorter traces for easier problems; if those labels are noisy, the model's apparently proportional thinking is not independently supported.

Editorial extensions

If this is right

  • SFT alone captures trace format and brevity but drops accuracy; DPO alone restores accuracy but keeps verbosity; the SFT+DPO combination is the configuration that balances both.
  • On unimodal benchmarks, the trained 1.5B and 7B models use roughly 8–10% fewer tokens than their base R1-distill counterparts while matching or exceeding accuracy on most of AIME, AMC, MATH, GSM8K, and HMMT.
  • On multimodal visual-math benchmarks, the 7B SFT+DPO model reduces reasoning tokens by up to roughly 30% and gains 4–5 accuracy points over its teacher, outperforming larger baselines on MathVerse and MMStar.
  • LLM-as-judge ratings place SFT+DPO traces above base models on clarity, completeness, correctness, and redundancy, supporting the claim that compression improves readability as well as cost.
  • Because the approach changes only the training data and the post-training procedure, it can be applied to existing open-weight student models without modifying their architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit: report absolute compressed trace lengths per difficulty bin, not just compression ratios, since ratios alone cannot show whether absolute reasoning length actually increases with difficulty.
  • The SFT-learns-format / DPO-learns-correctness split suggests a reusable recipe for other token-cost-sensitive generation tasks, such as constrained code generation or agentic tool use, where output structure must be imposed without sacrificing quality.
  • The difficulty grader could be replaced with an open, calibrated difficulty model validated against human ratings, which would make the pipeline reproducible without querying a proprietary model and would directly test the stability of the difficulty labels.
  • The method should transfer to long-document question answering and other settings where reasoning length is the dominant cost, as long as a difficulty signal can be assigned to each input.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a difficulty-aware chain-of-thought (CoT) distillation framework. The authors estimate a difficulty score d(x) for each math problem using GPT-4o-mini, compress long teacher CoT traces with a difficulty-guided summarizer, and train Qwen-based student models with SFT, DPO, and SFT+DPO. They evaluate on unimodal (AIME, MATH, AMC, GSM8K, HMMT) and multimodal (MathVerse, HLE(V), MathVision, OlympiadBench, MMStar) benchmarks, claiming that the SFT+DPO combination reduces reasoning tokens by up to 30% while maintaining or improving accuracy, and that trained models 'think proportionally' to problem difficulty. The abstract and Section 1 frame the central contribution as teaching models to dynamically vary reasoning length without architectural changes.

Significance. If the central claims were supported, the paper would make a useful empirical contribution to efficient math reasoning: it tackles a practical problem (over-long CoT), covers both text and multimodal settings, compares against an explicit length-control baseline (L1), and includes ablations of the compression prompt (Section A.5, Table 4) and qualitative trace comparisons (Appendix A.6). The paper also reports hyperparameters in Table 3. However, the significance as presented is substantially undermined by the gap between the claims and the evidence. The unimodal accuracy-efficiency claim is contradicted by the paper's own Table 2, the 'proportional thinking' claim is never directly measured, and the DPO objective in Section 3.2 is not the standard DPO loss and is not well-defined as written. Because these issues bear directly on the paper's two research questions, the contribution cannot be accepted on the current evidence.

major comments (5)
  1. [§5, Table 2] The headline claim in Section 1 and the abstract—'maintaining or improving accuracy' with 'up to 30%' token reduction—is contradicted by the unimodal results in Table 2. For the 7B SFT+DPO model versus the R1-Distill-Qwen-7B base: AIME 31.7 vs 35.0, AMC 69.9 vs 71.1, GSM8K 86.9 vs 88.0, HMMT 20.0 vs 24.4; only MATH improves (88.8 vs 86.0). Token savings are approximately 8–10%, not 30%. For the 1.5B model, AMC (57.8 vs 54.2), MATH (79.4 vs 77.0), and AIME (23.4 vs 20.0) improve, but GSM8K (82.3 vs 83.9) and HMMT (12.2 vs 12.2) do not improve. Thus RQ1—'match or exceed the accuracy of full CoT models'—fails on the majority of unimodal benchmarks for the 7B model, which is the paper's flagship configuration.
  2. [§5, Table 2 (RQ2)] The claim of better accuracy-efficiency trade-offs than explicit length-controlled baselines is not supported by the L1 comparison. L1-Qwen-1.5B-Max beats the SFT+DPO 1.5B model on AIME (24.4 vs 23.4), AMC (67.5 vs 57.8), MATH (83.6 vs 79.4), and GSM8K (87.3 vs 82.3), while using roughly 2.5–3× fewer tokens (e.g., 2,280 vs 6,192 tokens on AIME). The only benchmark where the proposed model wins is HMMT (12.2 vs 11.1), and that is a small gap. The text's discussion in Section 5 cherry-picks AIME ('surpasses its base counterpart') and does not address the fact that the proposed model is dominated by L1 on nearly every benchmark. This directly contradicts the RQ2 claim as stated.
  3. [§3.2, DPO objective] The DPO objective as written is not the standard DPO loss and is not mathematically well-formed. The loss is defined as -Σ log [ exp(β·KL(f_θ(x_i), r̃_i)) / (exp(β·KL(f_θ(x_i), r̃_i)) + exp(β·KL(f_θ(x_i), r_long_i))) ]. Here KL(f_θ(x_i), r̃_i) is written between a model distribution and a raw string, which is not a defined KL divergence. Standard DPO uses log-likelihood ratios of preferred/rejected completions under the current and reference policies (Rafailov et al., 2024); no reference model or likelihood term appears here. Because DPO is a central component of the method and the paper attributes accuracy gains to it, the missing or incorrect formulation makes the training procedure irreproducible and prevents the reader from verifying the SFT+DPO results.
  4. [§3.1, Table 1, Figure 5] The central behavioral claim that trained models 'think proportionally' is unsupported and circular as presented. The compression prompt in Figure 5 explicitly instructs the summarizer that 'the degree of refinement ... should be proportional to the problem's difficulty.' Training on traces produced under that instruction and then observing any correlation between output length and difficulty would be an artifact of the prompt, not evidence that the model learned proportional thinking. Moreover, Table 1 reports compression ratios, not absolute compressed lengths. For unimodal data the compression ratio rises with difficulty (79.1% at d=1 to 90.6% at d=8), which means harder problems are compressed more aggressively; whether absolute lengths increase with difficulty depends on the original lengths, which are never reported. The paper also never reports E[T|d(x)] for the trained models, so
  5. [§3.1, Appendix A.2.1] The difficulty-estimation step is load-bearing but its validity is asserted rather than demonstrated. The text states that the GPT-4o-mini ratings are 'stable, human-aligned' but provides no human agreement study, no stability analysis across runs or prompts, and no validation of the AoPS-scaling calibration on the actual training and evaluation distributions. Since d(x) controls the degree of compression and hence the entire training signal, noisy or miscalibrated difficulty scores would propagate through the pipeline. The limitations section acknowledges that GPT-4o-mini estimates 'could propagate,' but the manuscript does not quantify this risk or provide any supporting measurement.
minor comments (5)
  1. [§3.1] Typo: 'GTP-4o-mini' should be 'GPT-4o-mini'.
  2. [§4.3] The sentence 'For unimodal we directly extract the and match it with the ground truth answer' is incomplete; a word (probably 'final answer') is missing.
  3. [Table 2 caption] The caption says 'across seven math reasoning benchmarks' but the table lists five (AIME, AMC, MATH, GSM8K, HMMT).
  4. [§5 / Figure 1] The multimodal results are reported only as bar plots without numerical labels or error bars, and the text alternates between comparing to 'teacher' and to 'base.' This makes it difficult to verify the claimed 25–30% token reductions in the multimodal setting.
  5. [References] L1 is cited as both [1] and [2]; the reference list should be consolidated. Also, the related-work description of L1 attributes it to [31] in the text but [1]/[2] in the bibliography; please reconcile.

Circularity Check

1 steps flagged · score 7.0 of 10

The 'proportional thinking' finding is written into the training data by the compression prompt and then reported as a discovery; the accuracy/efficiency results are empirical but partly contradict the abstract.

  1. self definitional [Section 2 (formulation, |˜r|≈α·d(x)); Section 3.1 and Figure 5 (difficulty-aware compression prompt); Section 5 (claim of adaptive token allocation)]
    "'A student model fS is trained to map x 7→ (˜r, y), with |˜r| ≈ α·d(x), α > 0' (Section 2); 'This function adapts the verbosity of the reasoning proportionally to the problem’s difficulty—minimal steps for easy problems and more elaborate chains for hard ones' (Section 3.1); 'The degree of refinement (i.e., the length and detail of the refined CoT) should be proportional to the problem’s difficulty' (Figure 5)."

    The paper's central behavioral claim—that models can be taught to 'think proportionally', i.e. E[T|d(x)]∝d(x)—is not derived from independent evidence; it is imposed by construction. The compression prompt explicitly instructs the summarizer to make the refined trace length proportional to the difficulty rating, producing training tuples (xi, ˜ri, yi) with |˜r|≈α·d(x). The student is then trained to imitate exactly these traces. Therefore, any post-training correlation between output length and d(x) is a check that SFT/DPO successfully imitated the training targets, not an emergent or newly discovered property. The abstract's 'Remarkably, we show...' and Section 5's 'our SFT+DPO models expend more tokens on challenging tasks while compressing easy cases' present this construction as a find

full rationale

The strongest circularity is in the behavioral headline: the target length-versus-difficulty relationship is explicitly engineered into the data-generation prompt (Figure 5) and the mathematical formulation (|˜r|≈α·d(x) in Section 2). Training on such traces and then reporting that models allocate tokens adaptively is a self-definitional check of imitation quality, not an independent discovery. The paper does not even report absolute trained-model output lengths by difficulty bucket, so the claimed 'quantitative metrics confirm' is unsupported. The accuracy/efficiency comparisons against base models and L1 are genuine empirical content and not circular, though the unimodal results in Table 2 show only 8–10% savings and some accuracy losses, and L1 dominates the 1.5B row—tensions that are correctness/evaluation concerns rather than circularity. No load-bearing self-citation chain or imported uniqueness theorem appears. Overall, the central behavioral claim reduces by construction to the training-data definition, giving a score of 7 rather than 0; the independent benchmark content prevents a fully definitional score of 10.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests on model-generated difficulty labels, an implicit target proportionality constant set by a prompt, and several unvalidated domain assumptions (human-alignment of the grader, correctness preservation after compression, no train/eval contamination). The DPO objective as written is inconsistent with the standard formulation, and the compression-ratio table contradicts the proportionality instruction, so the data-generation assumption itself is shaky.

free parameters (4)
  • Difficulty label d(x) for each problem = GPT-4o-mini output on 1-10 AoPS scale
    Controls the compression target length; no human validation provided.
  • Proportionality constant alpha in |r~| approx alpha * d(x) = unreported
    The mapping from difficulty to target length is set implicitly by the gpt4o_DA2 prompt and is never measured directly; Table 1 reports only compression ratios, not absolute compressed lengths.
  • DPO beta = 0.1
    Preference temperature, fixed without search; affects the DPO objective balance.
  • LoRA rank and learning rates = rank 16; SFT LR 3e-4/2e-4; DPO LR 7e-6/5e-6
    Standard training hyperparameters, not central to the claim.
assumptions (6)
  • domain assumption GPT-4o-mini's AoPS-based difficulty ratings are stable and human-aligned
    Asserted in Section 3.1 without any human agreement data; all downstream length targets depend on this.
  • domain assumption gpt4o_DA2 summarization preserves the mathematical correctness needed for student training
    No correctness audit of compressed traces; Section A.5 shows the alternative gpt4o_basic summarizer omits critical steps, and DA2 was chosen on length balance, not verified correctness.
  • domain assumption The AoPS contest-tier rubric can be collapsed onto a scalar 1-10 difficulty
    The rubric (Figures 3, 4) is a heuristic interpolation across contest levels; used to produce d(x).
  • ad hoc to paper Training problems from OpenMathReasoning/AoPS forums do not overlap with the evaluation problems
    No contamination check is run, though AIME/AMC/HMMT problems routinely appear on AoPS forums; if contaminated, the accuracy gains are inflated.
  • ad hoc to paper The DPO objective in Section 3.2 corresponds to the implemented training
    The printed loss (softmax over KL divergences with no reference model) is not the standard DPO objective described in [43]; the paper does not reconcile this.
  • ad hoc to paper The summarizer produced absolute trace lengths proportional to difficulty
    Table 1 shows unimodal compression ratios increasing with difficulty (harder problems compressed more), opposite to the prompt's 'do not compress hard problems aggressively' instruction; absolute lengths by difficulty are never reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation." pith.science (2026). https://pith.science/paper/5RN62FYY

@misc{pith2026250905226,
  author       = {Pith},
  title        = {Pith review of: Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5RN62FYY}},
  note         = {Machine review of arXiv:2509.05226}
}
read the original abstract

Chain-of-thought reasoning, while powerful, can produce unnecessarily verbose output for simpler problems. We present a framework for difficulty-aware reasoning that teaches models to dynamically adjust reasoning depth based on problem complexity. Remarkably, we show that models can be endowed with such dynamic inference pathways without any architectural modifications; we simply post-train on data that is carefully curated to include chain-of-thought traces that are proportional in length to problem difficulty. Our analysis reveals that post-training via supervised fine-tuning (SFT) primarily captures patterns like reasoning length and format, while direct preference optimization (DPO) preserves reasoning accuracy, with their combination reducing length and maintaining or improving performance. Both quantitative metrics and qualitative assessments confirm that models can learn to "think proportionally", reasoning minimally on simple problems while maintaining depth for complex ones.

Figures

Figures reproduced from arXiv: 2509.05226 by the authors.

Figure 1
Figure 1. The left bar plot shows pass@1 and the right plot shows average token count in reasoning [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. LLM-as-Judge evaluation scores for unimodal and multimodal models. Evaluations are [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Math problem difficulty grading prompt. The task is to classify math problems into difficulty [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Multimodal math problem difficulty grading prompt. Extends AoPS standards to problems [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Chain-of-thought refinement prompt for mathematical solutions. Specifies verbosity [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Multimodal chain-of-thought refinement prompt. Maintains identical structure to mathemat [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Answer verification prompt. Directly replicates original instructions for strict answer [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Step-by-step explanation prompt. Directly replicates original instructions for detailed [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Answer submission prompt. Directly replicates original instructions for answer-only [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Step-by-step solution prompt. Directly replicates original instructions for mathematical [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Structured problem-solving prompt. Specifies comprehensive thinking process with [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: LLM-as-Judge evaluation scores with chain-of-thought rationale. [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Prompt for evaluating clarity of mathematical solutions. [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Prompt for evaluating completeness of mathematical solutions. [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Prompt for evaluating correctness of intermediate steps in mathematical solutions. [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: Prompt for evaluating redundancy in mathematical solutions. [PITH_FULL_IMAGE:figures/full_fig_p025_16.png]
Figure 17
Figure 17. Figure 17: Extended clarity evaluation prompt with rationale and explicit XML-style output format. [PITH_FULL_IMAGE:figures/full_fig_p025_17.png]
Figure 18
Figure 18. Figure 18: Difficulty ratings assigned to each problem using the AoPS scale. [PITH_FULL_IMAGE:figures/full_fig_p026_18.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

81 extracted references · 41 canonical work pages

  1. [2]

    Aggarwal, A

    P. Aggarwal, A. Madaan, Y . Yang, and Mausam. Let’s sample step by step: Adaptive-consistency for efficient reasoning and coding with llms.Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16624–16648, 2023

    2023

  2. [3]

    Balunovi´c, J

    M. Balunovi´c, J. Dekoninck, I. Petrov, N. Jovanovi´c, and M. Vechev. Matharena: Evaluating llms on uncontaminated math competitions, Feb. 2025. URLhttps://matharena.ai/

    work page 2025

  3. [4]

    Besta, N

    M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, L. Gianinazzi, J. Gajda, T. Lehmann, M. Podstawski, H. Niewiadomski, P. Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models.arXiv preprint arXiv:2308.09687, 2023

    arXiv 2023

  4. [5]

    Bills, A

    S. Bills, A. S. Jermyn, N. Werning, J. S. Kim, K. Wu, and K. Ellis. Language models represent space and time.Advances in Neural Information Processing Systems, 36, 2023

    work page 2023

  5. [6]

    Boye and B

    J. Boye and B. Moell. Large language models and mathematical reasoning failures, 2025. URL https://arxiv.org/abs/2502.11574

    arXiv 2025

  6. [7]

    T. Cai, Y . Li, Z. Chen, Q. V . Le, J. Z. K. Yang, and C. Li. Medusa: Simple framework for accelerating llm generation with multiple decoding heads. 2024

    work page 2024

  7. [8]

    H. Chen, H. Tu, F. Wang, H. Liu, X. Tang, X. Du, Y . Zhou, and C. Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models, 2025. URL https://arxiv.org/abs/2504.11468

    arXiv 2025

  8. [9]

    L. Chen, J. Li, X. Dong, P. Zhang, Y . Zang, Z. Chen, H. Duan, J. Wang, Y . Qiao, D. Lin, and F. Zhao. Are we on the right way for evaluating large vision-language models?, 2024. URL https://arxiv.org/abs/2403.20330

    arXiv 2024

Show all 81 references
  1. [10]

    X. Chen, J. Xu, T. Liang, Z. He, J. Pang, D. Yu, L. Song, Q. Liu, M. Zhou, Z. Zhang, R. Wang, Z. Tu, H. Mi, and D. Yu. Do not think that much for 2+3=? on the overthinking of o1-like llms,

  2. [11]

    Chevalier, A

    A. Chevalier, A. Wettig, A. Ajith, and D. Chen. Adapting language models to compress contexts. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3829–3846, Singapore, Dec. 2023. Association for Computational Linguistics. doi: 10.18...

    2023 doi

  3. [12]

    Z. Chu, J. Chen, Q. Chen, W. Yu, T. He, H. Wang, W. Peng, M. Liu, B. Qin, and T. Liu. Navigate through enigmatic labyrinth a survey of chain of thought reasoning: Advances, frontiers and future. InThe 62nd Annual Meeting of the Association for Computational Linguistics: ACL 20...

    2024

  4. [13]

    Cobbe, V

    K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  5. [14]

    Damani, I

    M. Damani, I. Shenfeld, A. Peng, A. Bobu, and J. Andreas. Learning how hard to think: Input- adaptive allocation of lm computation, 2024. URLhttps://arxiv.org/abs/2410.04707

    2024 arXiv

  6. [16]

    DeepSeek-AI, D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 10

    2025 arXiv

  7. [17]

    H. Duan, J. Yang, Y . Qiao, X. Fang, L. Chen, Y . Liu, X. Dong, Y . Zang, P. Zhang, J. Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. InProceedings of the 32nd ACM International Conference on Multimedia, pages 11198–11201, 2024

    2024

  8. [18]

    Y . Fu, H. Peng, T. Khot, O. Tafjord, P. Clark, and N. Tandon. Complexity-based prompting for multi-step reasoning.arXiv preprint arXiv:2210.00720, 2022

    2022 arXiv

  9. [19]

    T. Ge, H. Jing, L. Wang, X. Wang, S. Chen, and F. Wei. In-context autoencoder for context com- pression in a large language model. InInternational Conference on Learning Representations, 2024

    2024

  10. [20]

    J. Gu, X. Jiang, Z. Shi, H. Tan, X. Zhai, C. Xu, W. Li, Y . Shen, S. Ma, H. Liu, S. Wang, K. Zhang, Y . Wang, W. Gao, L. Ni, and J. Guo. A survey on llm-as-a-judge, 2025. URL https://arxiv.org/abs/2411.15594

    2025 arXiv

  11. [21]

    J. Guo, D. Li, L. Liu, P. Zhou, and S. Liao. Softcot: Soft chain-of-thought for efficient reasoning with llms.arXiv preprint arXiv:2402.10200, 2024

    2024 arXiv

  12. [22]

    S. Hao, Y . Gu, H. Ma, J. J. Hong, Z. Wang, D. Peng, Z. Duan, H. Sun, K. P. Murphy, T. L. Lai, A. Wang, and L. Wang. Reasoning with language model is planning with world model.arXiv preprint arXiv:2305.14992, 2023

    2023 arXiv

  13. [23]

    C. He, R. Luo, Y . Bai, S. Hu, Z. L. Thai, J. Shen, J. Hu, X. Han, Y . Huang, Y . Zhang, J. Liu, L. Qi, Z. Liu, and M. Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems, 2024. URL https://arxiv.org/ abs/2...

    2024 arXiv

  14. [24]

    Hendel, M

    R. Hendel, M. Geva, and A. Globerson. In-context learning creates task vectors. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 9318–9333. Association for Computational Linguistics, 2023

    2023

  15. [25]

    Hendrycks, C

    D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. URL https://arxiv. org/abs/2103.03874

    2021 arXiv

  16. [26]

    Hsieh, C.-L

    C.-Y . Hsieh, C.-L. Li, C.-K. Yeh, H. Nakhost, Y . Fujii, A. Ratner, R. Krishna, C.-Y . Lee, and T. Pfister. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes.arXiv preprint arXiv:2305.02301, 2023. URL https://arxiv.o...

    2023 arXiv

  17. [27]

    Huang, C

    B. Huang, C. Mitra, A. Arbelle, L. Karlinsky, T. Darrell, and R. Herzig. Multimodal task vectors enable many-shot multimodal in-context learning.arXiv preprint arXiv:2406.15334, 2024

    2024 arXiv

  18. [28]

    Kojima, S

    T. Kojima, S. S. Gu, M. Reid, Y . Matsuo, and Y . Iwasawa. Large language models are zero-shot reasoners.Advances in Neural Information Processing Systems, 35:22199–22213, 2022

    2022

  19. [29]

    Y . Lang, Z. Ji, H. Wang, W. Yao, J. Xu, M. Yan, T. Gui, and Q. Zhang. Coconut: Combining implicit and explicit knowledge for chain-of-thought reasoning. 2024

    2024

  20. [30]

    Leviathan, M

    Y . Leviathan, M. Kalman, and Y . Matias. Fast inference from transformers via speculative decoding. 2023

    2023

  21. [31]

    H. Li, Z. Chen, J. Z. Kolter, A. Parameswaran, G. Neubig, and Y . A. Jiang. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2024

    2024 arXiv

  22. [32]

    Z. Liu, Y . Chen, Y . Lin, M. Tegmark, and B. Zhou. Visual chain of thought: Bridging logical gaps with multimodal infillings.arXiv preprint arXiv:2305.09666, 2023

    2023 arXiv

  23. [33]

    P. Lu, S. Mishra, T. Xia, L. Qiu, K.-W. Chang, S.-C. Zhu, O. Tafjord, P. Clark, and A. Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507–2522, 2022

    2022

  24. [34]

    C. Miao, M. Ye, F. Wang, and F. Huang. Seed: Accelerating reasoning tree construction via scheduled speculative decoding. 2024. 11

    2024

  25. [35]

    Moshkov, D

    I. Moshkov, D. Hanley, I. Sorokin, S. Toshniwal, C. Henkel, B. Schifferer, W. Du, and I. Git- man. Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset.arXiv preprint arXiv:2504.16891, 2025

    2025 arXiv

  26. [36]

    J. Mu, X. L. Li, and N. D. Goodman. Learning to compress prompts with gist tokens.Advances in Neural Information Processing Systems, 36, 2024

    2024

  27. [37]

    Muennighoff, Z

    N. Muennighoff, Z. Yang, W. Shi, X. L. Li, L. Fei-Fei, H. Hajishirzi, L. Zettlemoyer, P. Liang, E. Candès, and T. Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  28. [38]

    Olsson, N

    C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. Hao, T. Henighan, A. DasSarma, Y . Bai, D. Her- nandez, A. Askell, et al. In-context learning and induction heads. InThe Tenth International Conference on Learning Representations (ICLR), 2022

    2022

  29. [39]

    Openai o1 system card, 2024

    OpenAI. Openai o1 system card, 2024. URLhttps://arxiv.org/abs/2412.16720

    2024 arXiv

  30. [40]

    A. Patil. Advancing reasoning in large language models: Promising methods and approaches,

  31. [41]

    L. Phan, A. Gatti, and Others. Humanity’s last exam, 2025. URL https://arxiv.org/abs/ 2501.14249

    2025 arXiv

  32. [42]

    X. Pi, F. Wang, S. Zhang, C. Fu, C. Fu, E. Lo, and Y . Zou. Visual chain-of-thought: Bridging logical gaps with multimodal infillings.arXiv preprint arXiv:2305.02317, 2023

    2023 arXiv

  33. [43]

    URLhttps://arxiv.org/abs/2502.03671

    arXiv

  34. [44]

    gradient descent

    M. Reid, D. Zhou, Y . Fu, P. Liu, and G. Neubig. Automatic prompt optimization with "gradient descent" and beam search. InFindings of the Association for Computational Linguistics: EMNLP 2023, 2023. URLhttps://aclanthology.org/2023.findings-emnlp.1001

    2023

  35. [45]

    Samsi, D

    S. Samsi, D. Zhao, J. McDonald, B. Li, A. Michaleas, M. Jones, W. Bergeron, J. Kepner, D. Tiwari, and V . Gadepally. From words to watts: Benchmarking the energy costs of large language model inference. In2023 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–...

    2023

  36. [46]

    Rafailov, A

    R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn. Direct preference optimization: Your language model is secretly a reward model, 2024. URL https://arxiv. org/abs/2305.18290

    2024 arXiv

  37. [47]

    Q. Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlm.github.io/blog/qwq-32b/

    2025

  38. [48]

    E. Todd, M. L. Li, A. S. Sharma, A. Mueller, B. C. Wallace, and D. Bau. Function vectors in large language models. InProceedings of the 2024 International Conference on Learning Representations, 2024

    2024

  39. [49]

    N. Team. Sky-t1: Train your own o1 preview model within $450. https://novasky- ai.github.io/posts/sky-t1, 2025. Accessed: 2025-01-09

    2025

  40. [50]

    P. Wang, Z. Wang, Z. Li, Y . Gao, B. Yin, and X. Wei. Scott: Self-consistent chain-of-thought distillation. 2023

    2023

  41. [51]

    X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, and D. Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  42. [52]

    K. Wang, J. Pan, W. Shi, Z. Lu, H. Ren, A. Zhou, M. Zhan, and H. Li. Measuring mul- timodal mathematical reasoning with math-vision dataset. InThe Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/f...

    2024

  43. [53]

    Wingate, M

    D. Wingate, M. Shoeybi, and T. Sorensen. Prompt compression and contrastive conditioning for controllability and toxicity reduction in language models. InFindings of the Association for Computational Linguistics: EMNLP 2022, pages 5621–5634, Abu Dhabi, United Arab Emirates, De...

    2022 doi

  44. [54]

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y . Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush. Transformers: State-of-the-art natu...

    2020

  45. [55]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022. 12

    2022

  46. [56]

    G. Xu, P. Jin, H. Li, Y . Song, L. Sun, and L. Yuan. Llava-cot: Let vision language models reason step-by-step, 2024. URLhttps://arxiv.org/abs/2411.10440

    2024 arXiv

  47. [57]

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X...

    2024 arXiv

  48. [58]

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffith, Y . Xu, and X. Shen. Tree of thoughts: Deliberate problem solving with large language models.arXiv preprint arXiv:2305.10601, 2023

    2023 arXiv

  49. [59]

    K. Wu, J. J. Cao, T. He, R. Pudipeddi, P. Ghanathe, P. Tantia, V . Thaker, Y . Liu, A. Nagrani, S. Kejriwal, et al. Med-palm 2: Towards expert-level medical question answering with large language models.arXiv preprint arXiv:2305.09617, 2023

    2023 arXiv

  50. [60]

    Zhang, D

    R. Zhang, D. Jiang, Y . Zhang, H. Lin, Z. Guo, P. Qiu, A. Zhou, P. Lu, K.-W. Chang, P. Gao, and H. Li. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?,

  51. [61]

    Zhang, A

    Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola. Multimodal chain-of-thought reasoning in language models.arXiv preprint arXiv:2302.00923, 2023

    2023 arXiv

  52. [62]

    Zheng, B

    G. Zheng, B. Yang, J. Tang, H.-Y . Zhou, and S. Yang. Ddcot: Duty-distinct chain-of-thought prompting for multimodal reasoning in language models.arXiv preprint arXiv:2310.16436, 2023

    2023 arXiv

  53. [63]

    Zhang, Z

    P. Zhang, Z. Liu, S. Xiao, N. Shao, Q. Ye, and Z. Dou. Soaring from 4k to 400k: Extending llm’s context with activation beacon.arXiv preprint arXiv:2401.03462, 2024

    2024 arXiv

  54. [64]

    Zheng, R

    Y . Zheng, R. Zhang, J. Zhang, Y . Ye, Z. Luo, Z. Feng, and Y . Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thaila...

    2024 arXiv

  55. [65]

    URLhttps://arxiv.org/abs/2403.14624

    arXiv

  56. [68]

    Zheng, W.-L

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. URLhttps://arxiv.org/abs/2306.05685

    2023 arXiv

  57. [70]

    think hard

    P. Y . Zhou, B. Y . Lin, M. Yasunaga, and X. Ren. Distilling multi-step reasoning capabilities of large language models into smaller models via verifier-guided sequence generation. 2024. A Appendix A.1 Hyperparameters In Table 3 we provide a detailed list of parameters used in...

    2024

  58. [72]

    Examine the logical progression from one step to the next

  59. [73]

    Determine whether key concepts and reasoning steps are explained in a clear and concise manner

  60. [74]

    Instructions: •Do notattempt to solve, fix, or fact-check the solution

    Based on the criteria above, assign a clarity score from 1 to 5. Instructions: •Do notattempt to solve, fix, or fact-check the solution. • Focus solely on how clearly the reasoning is communicated. • Provide your score in the following format:[[score]] Input: Problem:{question...

  61. [76]

    Identify the key logical and mathematical steps required to solve the problem

  62. [77]

    Assess whether these steps are present and clearly stated in the solution

  63. [78]

    Instructions: •Do notattempt to solve, fix, or fact-check the solution

    Based on the criteria above, assign a completeness score from 1 to 5. Instructions: •Do notattempt to solve, fix, or fact-check the solution. • Focus solely on whether all essential reasoning steps are present. • Provide your score in the following format:[[score]] Input: Prob...

  64. [80]

    Identify and examine each intermediate reasoning step

  65. [81]

    Determine whether each step is mathematically correct and logically valid

  66. [82]

    Instructions: •Do notsolve or complete the problem yourself

    Based on the criteria above, assign a correctness score from 1 to 5. Instructions: •Do notsolve or complete the problem yourself. • Focus only on evaluating the accuracy of the intermediate steps shown in the solution. • Provide your score in the following format:[[score]] Inp...

  67. [83]

    Carefully read both the problem and the provided solution

  68. [84]

    Identify any repetition or restatement of the same concepts or steps

  69. [85]

    Assess whether the repetition is necessary for clarity or if it is excessive and avoidable

  70. [86]

    Instructions: •Do notattempt to solve or correct the solution

    Based on the criteria above, assign a redundancy score from 1 to 5. Instructions: •Do notattempt to solve or correct the solution. • Focus only on identifying and evaluating unnecessary repetition or redundant reasoning. • Provide your score in the following format:[[score]] I...

  71. [412]

    URLhttps://aclanthology.org/2022.findings-emnlp.412/

    2022

  72. [2024]

    URLhttps://arxiv.org/abs/2309.15402

    arXiv

  73. [2025]

    URLhttps://arxiv.org/abs/2412.21187

    arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.