Pith. sign in

REVIEW 4 major objections 5 minor 87 references

Decomposing Elements of Problem Solving: What "Math" Does RL Teach?

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GRPO reinforcement learning primarily sharpens how reliably models execute math steps they already know, without enabling them to solve any previously unsolvable problems.

desk verdict The coverage wall and temperature distillation are well-measured and credible; the paper's novel 'execution not planning' claim is the soft part, resting on one unvalidated automated grader. read the letter →

arxiv 2505.22756 v1 pith:XIXGY6GD submitted 2025-05-28 cs.AI cs.CLcs.LG

Tian Qin , Core Francisco Park , Mujin Kwun , Aaron Walsman , Eran Malach , Nikhil Anand , Hidenori Tanaka , David Alvarez-Melis This is my paper
classification cs.AIcs.CLcs.LG
keywords reinforcementlearningGRPOmathematicalreasoninglargelanguagemodelscoveragewalltemperaturedistillationplanningvsexecutionsynthetictask
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the accuracy gains from reinforcement learning (GRPO) on math benchmarks do not mean the model learned new problem-solving strategies. Instead, GRPO mostly "distills" the model's existing behavior: it makes correct solutions more likely and less sensitive to sampling temperature on problems the model already partially solves. On MATH-500, the number of problems solvable at high sampling effort (Pass@64 at the optimal temperature) does not increase; on the held-out test set, GRPO unlocks zero previously unsolvable problems. The paper decomposes reasoning into Plan, Execute, and Verify, and presents evidence that even small models plan most problems well but fail at execution, with GRPO mainly patching low-level execution errors. A synthetic solution-tree navigation task reproduces these findings and suggests conditions, such as removing spurious training context, shrinking the action space, and adding more RL data, under which RL can expand coverage.

What carries the argument

The central object is the decomposition of mathematical problem solving into Plan (mapping a question to a sequence of solution steps), Execute (carrying those steps out correctly), and Verify (detecting and correcting errors), viewed as navigation through a solution tree whose nodes are intermediate steps. Planning locates the correct branch; execution is the reliable traversal of that branch; the tree visualization shows how sampled generations branch and how semantically irrelevant wording changes success probability. The metric for the coverage wall is Coverage(K, T*), the number of problems solved at least once in K samples at the optimal temperature, measured at K=64 after best-of-64 sampling over five seeds. The synthetic setup encodes actions in transitions of "problem states" and adds spurious context tokens correlated with actions during pretraining, which reproduces temperature distillation and the coverage wall in a controlled environment.

What would settle it

Re-annotate the same pre- and post-GRPO solution traces with human graders using the paper's own rubric; if humans find that GRPO changes the fraction of traces with correct plans, or if a second independent grader disagrees with GPT-4.1-mini on a substantial share of traces, the execution-only conclusion fails. Alternatively, find any held-out MATH-500 problem that is incorrect for all pre-GRPO samples yet correct in at least one post-GRPO sample within a pre-registered sample budget.

Watch

Extended reading notes

Core claim

The central claim is that GRPO produces a "temperature distillation" effect rather than an expansion of mathematical capability: after RL, precision curves flatten across sampling temperatures on problems the model already knows, but the coverage wall, defined as the limit of Pass@K as K increases, remains fixed. Using a Plan/Execute/Verify decomposition, the paper reports that planning ability is largely present even in a 0.5B model, execution is the bottleneck, and GRPO improves execution reliability by reducing elementary math and basic logic errors without reducing high-school-level factual errors. On the training split GRPO unlocks only two new problems; on the MATH-500 test set it solves no new problems. The authors build a minimal synthetic task in which states, actions, problem states, and spuriously correlated contexts mimic the structure of math problem solving, and in that setting they identify conditions under which RL does improve coverage: removing spurious correlations from pretraining, shrinking the action space, and increasing RL data diversity.

Load-bearing premise

The claim that GRPO improves execution rather than planning rests on automated grading by GPT-4.1-mini of whether a solution trace contains the correct plan and whether the steps are executed correctly, and the paper reports no human inter-rater agreement or benchmark to validate that split.

Editorial extensions

If this is right

  • On MATH-500, Pass@64 at the optimal temperature does not significantly increase after GRPO, so increasing test-time sampling effort does not recover new problems post-RL.
  • Per-problem precision gains concentrate on medium-difficulty problems the model already partially solves, with a training-to-test generalization gap in that regime.
  • GRPO reduces elementary mathematical and basic logical errors in generated traces, stabilizing execution against spurious cues like word choice, but does not reduce high-school-level factual errors.
  • The synthetic solution-tree task reproduces temperature distillation and limited coverage gains, and shows that removing spurious pretraining correlations, reducing the action space, or increasing RL data can turn coverage gains on.
  • Because even a 0.5B model plans most MATH-500 problems correctly, the bottleneck for small-model math reasoning is execution, not knowing the overall approach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Plan/Execute distinction generalizes beyond MATH-500, then RL scaling on verifiable-reward benchmarks is likely to saturate on novel multi-step problems unless the training distribution exposes the model to new solution paths; a testable extension is to run the same matched-problem analysis on olympiad-level benchmarks where planning is harder.
  • The synthetic result that more RL data helps only when coverage is already improving suggests a practical diagnostic: measure per-problem precision gains early in training, and if no medium-precision improvement appears, additional data alone may not break the coverage wall.
  • Because the Plan/Execute labels come from a single automated grader, a direct human-annotation study of the same traces would be the cleanest check on whether "execution-only improvement" is a true capability boundary or partly an artifact of grading.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a decomposition of mathematical problem-solving into Plan, Execute, and Verify capabilities, and uses this lens to analyze what GRPO-style reinforcement learning teaches LLMs. Based on experiments with Qwen2.5-Instruct models (0.5B, 1.5B, 7B) trained on MATH and evaluated on MATH-500, GSM8K, and AIME24, the authors report two main phenomena: a 'temperature distillation' effect in which GRPO makes correct solutions more robust to sampling temperature, and a 'coverage wall' in which GRPO does not increase the number of problems solvable by best-of-64 sampling. They further claim that GRPO primarily improves execution rather than planning, and they construct a synthetic solution-tree navigation task that reproduces the temperature distillation and coverage-wall effects, while identifying conditions under which RL can overcome the coverage wall in that toy setting.

Significance. If the findings hold, the paper provides a fine-grained, falsifiable account of what RLVR does and does not teach, going beyond aggregate Pass@1 metrics. The main measurements use external benchmarks (MATH-500, GSM8K, AIME24) rather than fitted parameters, and the synthetic environment makes concrete predictions about when coverage can improve. The paper is clearly written, releases code, reports multiple seeds and model scales, and engages with concurrent work on coverage. The most novel contribution—the Plan/Execute decomposition—is currently supported by an unvalidated LLM-based annotation pipeline, and the coverage-wall claim is made from finite-sampling experiments; both need strengthening before the central conclusions can be fully credited.

major comments (4)
  1. [Section 4.2, Appendix E] The Plan/Execute decomposition, which is the paper's central novel claim, rests entirely on annotations produced by GPT-4.1-mini. The only validation reported is the statement that 'Manual inspection confirms that GPT's annotations align closely with human judgment' and 20 examples in Appendix E. No inter-rater agreement statistic, no human benchmark set, and no second annotator are provided. Because the same model writes the approach summary and then grades whether a student trace 'contains the approach,' the Plan label may reflect lexical similarity to the generated summary rather than a distinct planning capability. Please provide a quantitative validation of the annotation (e.g., agreement with human labels on a held-out sample, with per-class precision and recall) or explicitly temper the planning-versus-execution conclusions to reflect the reliance on an unvalidated proxy.
  2. [Abstract versus Section 4.2] The abstract states that RL-trained models hit a 'coverage wall' due to insufficient planning skills, but Section 4.2 reports that even the 0.5B model achieves high plan grades (71% in Fig. 4) and identifies execution, not planning, as the primary bottleneck. If the unsolved problems in Section 3.3 are those whose plans are missing, the paper must show this directly, for example by reporting plan grades restricted to the problems that remain unsolved after GRPO. As written, the mechanism attributed to the coverage wall is internally inconsistent with the paper's own Plan/Execute measurements.
  3. [Section 3.3 and Appendix C] The coverage-wall claim is based on best-of-64 sampling across 5 seeds after 60 epochs of GRPO on models up to 7B. Appendix C justifies K=64 by citing log-linear pass@k scaling, but Fig. 9 does not show that the curve has plateaued by k=64; the trend appears to continue increasing through k=200. Since coverage is defined as the limit as K→∞, the observed 'no improvement' may be an artifact of finite K and limited compute. Furthermore, the statement that GRPO 'fails to solve any new problems' on the test set is a zero-count observation whose resolution depends on K and the number of seeds. Please either demonstrate saturation of pass@k for the relevant models and temperatures or soften the claim to 'no significant coverage gain under the compute budget explored.'
  4. [Section 5.3 and Fig. 8] The synthetic experiments identify conditions under which RL 'can' improve coverage, but these conditions involve substantial changes to the task itself: removing spurious context correlations, reducing the action space, and providing 4x more RL data. The paper does not test whether any of these manipulations transfer to real LLMs, nor does it connect them to the real-model experiments of Section 3. As presented, the claim that the paper 'suggests a path toward overcoming these barriers' is a conjecture based on a toy model, and the conclusion should state this clearly rather than imply an actionable recipe for real RL training.
minor comments (5)
  1. [Section 3.3] The word 'minimial' should be 'minimal.'
  2. [Section 4.3] The phrase 'high shcool' should be 'high school,' and in the Fig. 5 caption 'examinbe' should be 'examine.'
  3. [Section 5.3] The sentence 'RL still somehow doesn't reach 100% and his a wall' should read 'hits a wall.'
  4. [Fig. 2 caption] The word 'T emperature' should be 'Temperature.'
  5. [Appendix H.1] In Table 2, 'V ocabulary' should be 'Vocabulary.'

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the Plan/Execute conclusion rests on an unvalidated proxy grader, but it is not a definitional or fitted reduction.

full rationale

The paper's central quantitative claims are measured against external benchmarks (MATH-500, GSM8K, AIME24) and are not derived from fitted parameters or from the paper's own definitions. The coverage-wall and temperature-distillation findings are empirical observations of pre/post-GRPO sampling behavior, not consequences of how those terms are defined. The synthetic environment in Section 5 is an explicitly designed analogy intended to reproduce observed phenomena; its perturbations in Fig. 8 are controlled experiments with falsifiable outcomes, not predictions that equal their inputs by construction. The main circularity-adjacent concern is the Plan/Execute decomposition in Section 4.2: GPT-4.1-mini both generates the approach summary from the ground-truth solution and grades whether student traces contain that plan and execute it correctly, and the only validation reported is 'Manual inspection confirms that GPT's annotations align closely with human judgment' with no inter-rater agreement or human benchmark. This is a real validity weakness for the claim that GRPO mainly improves execution rather than planning, but it is not circularity: the constructs Plan and Execute are defined independently of the GPT labels, and the labels are an external (if imperfectly validated) measurement instrument rather than a parameter fitted to the conclusion. The paper also contains minor author-overlapping citations (e.g., Refs. [54,56,57]) but they are contextual and not load-bearing for the empirical or synthetic results. No equation or definition reduces a claimed prediction to its own input, so the appropriate finding is no significant circularity, with a modest score reflecting the proxy-validation gap rather than any self-referential derivation.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper introduces conceptual labels (Plan/Execute/Verify, temperature distillation, coverage wall) but no new physical or mathematical entities. The free parameters are hand-chosen synthetic-data and measurement settings; the key assumptions are the unvalidated GPT grader and the transferability of the toy model.

free parameters (5)
  • alpha_train (synthetic) = 0.3
    Dirichlet concentration for pretraining action probabilities; hand-chosen to create biased action priors that mimic spurious correlations. Appendix H.2.
  • beta_train (synthetic) = 0.3
    Dirichlet concentration for pretraining context-action correlations; hand-chosen to create spurious context cues. Appendix H.2.
  • n_irr_ctx (synthetic) = 10
    Number of irrelevant context tokens per prompt; hand-chosen. Appendix H.2.
  • k_min, k_max (synthetic) = 5, 10
    Min and max actions per problem; hand-chosen sequence length range. Appendix H.2.
  • Coverage K for real models = 64
    Chosen as a practical limit to estimate coverage; justified by log-linear scaling with K. If true coverage grows beyond K=64, the coverage wall claim is measurement-dependent. Appendix C.
assumptions (3)
  • domain assumption GPT-4.1-mini grades of plan and execution align with human judgment
    The Plan/Execute separation is measured entirely through this automated grader; no human agreement statistics are reported (Section 4.2, Appendix E).
  • domain assumption The synthetic graph-navigation task is a faithful analogy to mathematical problem solving
    The conditions for overcoming the coverage wall are derived in this toy setting and assumed to transfer to real math (Sections 5.1-5.3).
  • domain assumption Pass@64 at optimal temperature approximates asymptotic coverage
    Coverage is defined as best Pass@64 across temperatures; extrapolation to the limit relies on the observed log-linear scaling (Section 3.3, Appendix C).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decomposing Elements of Problem Solving: What "Math" Does RL Teach?." pith.science (2026). https://pith.science/paper/XIXGY6GD

@misc{pith2026250522756,
  author       = {Pith},
  title        = {Pith review of: Decomposing Elements of Problem Solving: What "Math" Does RL Teach?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XIXGY6GD}},
  note         = {Machine review of arXiv:2505.22756}
}
read the original abstract

Mathematical reasoning tasks have become prominent benchmarks for assessing the reasoning capabilities of LLMs, especially with reinforcement learning (RL) methods such as GRPO showing significant performance gains. However, accuracy metrics alone do not support fine-grained assessment of capabilities and fail to reveal which problem-solving skills have been internalized. To better understand these capabilities, we propose to decompose problem solving into fundamental capabilities: Plan (mapping questions to sequences of steps), Execute (correctly performing solution steps), and Verify (identifying the correctness of a solution). Empirically, we find that GRPO mainly enhances the execution skill-improving execution robustness on problems the model already knows how to solve-a phenomenon we call temperature distillation. More importantly, we show that RL-trained models struggle with fundamentally new problems, hitting a 'coverage wall' due to insufficient planning skills. To explore RL's impact more deeply, we construct a minimal, synthetic solution-tree navigation task as an analogy for mathematical problem-solving. This controlled setup replicates our empirical findings, confirming RL primarily boosts execution robustness. Importantly, in this setting, we identify conditions under which RL can potentially overcome the coverage wall through improved exploration and generalization to new solution paths. Our findings provide insights into the role of RL in enhancing LLM reasoning, expose key limitations, and suggest a path toward overcoming these barriers. Code is available at https://github.com/cfpark00/RL-Wall.

Figures

Figures reproduced from arXiv: 2505.22756 by the authors.

Figure 1
Figure 1. Decomposing problem solving and modeling the effects of RL in a toy setting. (a): We decompose math problem solving into fundamental capabilities. To solve a problem model must first plan (map the question to a sequence of actions) and then execute (navigate the tree) without errors. (b): We design a minimal toy environment that explicitly captures these core reasoning elements in a controlled setting. (c): GRPO pri… view at source ↗
Figure 2
Figure 2. GRPO improves precision (average@K) through temperature distillation but does not improve coverage (measured as pass@K). Left, Middle: Precision (probability of correctly solving the problem) for MATH-500 questions before and after performing GRPO on Qwen2.5-1.5B-Instruct. GRPO “tightens" the model’s behaviors, which we study as the “temperature distillation effect". Right: Pass@64 at various temperature settings (s… view at source ↗
Figure 3
Figure 3. Per-problem precision improvement on train subset (left) and test set (right). We sort problems by pre-GRPO model’s precision, and examine the improvement from running GRPO. We report mean precision and standard deviations from 5 random seeds. GRPO improves precision on questions of medium-level difficulty while encountering a train-test generalization gap (green, purple star). GRPO also encounters a coverage wall (… view at source ↗
Figures from the paper (32 more)
Figure 4
Figure 4. Figure 4: Two elements to ace a math exam: Plan then Execute. Left: Examples of planning and execution failures from Qwen2.5-1.5B-Instruct on MATH-500 questions. Right: For MATH-500, even a small model knows how to approach (i.e., plan) most problems. However, the ability to cor…
Figure 5
Figure 5. Figure 5: Model cannot reliably execute math and logic reasoning due to spurious correlations. Left: One example of solution tree from Qwen2.5-1.5B-Instruct on a MATH-500 question. In this question, the model’s likelihood of outputting the correct solution rely heavily on a sema…
Figure 6
Figure 6. Figure 6: Synthetic Math Setup and Training Curves a) Synthetic Setup. Our setup is an analogy to mathematical problem solving. We design a graph navigation task where a model should internalize target states given the current state and action. However, the problem statement onl…
Figure 7
Figure 7. Figure 7: RL improves Execution Robustness. a, b) Precision of the trained autoregressive Transformer on 512 eval set questions before and after GRPO. c) Pass@128 of the RL training set and eval set depending on temperature [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: When Does RL improves Coverage? a) Pass@128 vs. Temperature, when the data has no context providing spurious correlations. b) Pass@128 vs. Temperature, when the action space defining the problem is smaller. c) Pass@128 vs. Temperature, when providing 4 times more RL da…
Figure 9
Figure 9. Figure 9: Sweeping different k values to measure coverage. Sweeping through model’s pass@k on MATH-500, we conclude that k = 64 is sufficiently large to measure how many problems are solvable by the model. algebra counting geometry interalgebra number prealgebra precalculus all …
Figure 10
Figure 10. Figure 10: GRPO improves mathematical reasoning in a subject-agnostic manner. We train GRPO models on seven subject-specific subsets of the MATH training data and evaluate their performance on number theory questions. Regardless of the training subject, all models show similar i…
Figure 11
Figure 11. Figure 11: The temperature distillation effect and coverage wall are consistent across settings. We run GRPO across different model sizes, evaluation benchmarks, and model families, and observe the same qualitative outcomes: model precision becomes more stable ("sharpens") acros…
Figure 12
Figure 12. Figure 12: Prompts used for evaluating model reasoning. [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Plan Failure Example 1 Plan Failure Example 2 (1) Does the solution contain attempts to solve the problem using the approach described above? The student attempts to solve the problem by simplifying the given rational expressions and testing a potential root by direct…
Figure 14
Figure 14. Figure 14: Plan Failure Example 2 22 [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Plan Failure Example 3 23 [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]
Figure 16
Figure 16. Figure 16: Plan Failure Example 4 24 [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: Plan Failure Example 5 Execution Failure Example 1 (1) Does the solution contain attempts to solve the problem using the approach described above? Yes. The student correctly identifies the hourly wages, uses the exchange rates to convert the wages into US dollars, and…
Figure 18
Figure 18. Figure 18: Execution Failure Example 1 25 [PITH_FULL_IMAGE:figures/full_fig_p025_18.png]
Figure 19
Figure 19. Figure 19: Execution Failure Example 2 26 [PITH_FULL_IMAGE:figures/full_fig_p026_19.png]
Figure 20
Figure 20. Figure 20: Execution Failure Example 3 Execution Failure Example 4 (1) Does the solution contain attempts to solve the problem using the approach described above? Yes. The student attempts to recognize the general form of the equation and compare it to standard conic forms. They…
Figure 21
Figure 21. Figure 21: Execution Failure Example 4 27 [PITH_FULL_IMAGE:figures/full_fig_p027_21.png]
Figure 22
Figure 22. Figure 22: Execution Failure Example 5 28 [PITH_FULL_IMAGE:figures/full_fig_p028_22.png]
Figure 23
Figure 23. Figure 23: A screenshot of the visualization tool we created to explore model’s generation as a solution tree. For a given problem, we sampled model’s generations and construct a tree based on these generations. The tree branches out whenever the generations differ from each oth…
Figure 24
Figure 24. Figure 24: Examples of Model Generated Solutions on MATH-500 Questions [PITH_FULL_IMAGE:figures/full_fig_p030_24.png]
Figure 25
Figure 25. Figure 25: Execution Failure Annotation Prompts G.2 Example We list 10 randomly selected examples below [PITH_FULL_IMAGE:figures/full_fig_p031_25.png]
Figure 26
Figure 26. Figure 26: Basic Math and Logic Error Example 1 Example: Elementary & Logic Error Example 2 Step-by-step analysis: 1. The prime numbers less than 30 are correctly listed as: 2, 3, 5, 7, 11, 13, 17, 19, 23, 29 2. The odd numbers greater than zero are correctly listed as: 1, 3, 5,…
Figure 27
Figure 27. Figure 27: Basic Math and Logic Error Example 2 32 [PITH_FULL_IMAGE:figures/full_fig_p032_27.png]
Figure 28
Figure 28. Figure 28: Basic Math and Logic Error Example 3 Example: Elementary & Logic Error Example 4 (1) The solution contains multiple factual mistakes: - The student claims that the denominator of the reciprocal (which is an odd integer) must be of the form 2 m to have a terminating de…
Figure 29
Figure 29. Figure 29: Basic Math and Logic Error Example 4 33 [PITH_FULL_IMAGE:figures/full_fig_p033_29.png]
Figure 30
Figure 30. Figure 30: Basic Math and Logic Error Example 5 34 [PITH_FULL_IMAGE:figures/full_fig_p034_30.png]
Figure 31
Figure 31. Figure 31: High School Level Math Error Example 1 35 [PITH_FULL_IMAGE:figures/full_fig_p035_31.png]
Figure 32
Figure 32. Figure 32: High School Level Math Error Example 2 Example: High School Math Error Example 3 (1) The student states that there are 4 sets of points that lie on the same straight line, but then counts 3 rows + 3 columns = 6 sets, divides by 2 to get 3 sets, and adds 2 diagonals to…
Figure 33
Figure 33. Figure 33: High School Level Math Error Example 3 Example: High School Math Error Example 4 (1) The solution contains a factual mistake in modular arithmetic and divisibility reasoning. Specifically, the student incorrectly concludes that m = 4 3 (26 − 3k) must be an integer, an…
Figure 34
Figure 34. Figure 34: High School Level Math Error Example 4 36 [PITH_FULL_IMAGE:figures/full_fig_p036_34.png]
Figure 35
Figure 35. Figure 35: High School Level Math Error Example 5 37 [PITH_FULL_IMAGE:figures/full_fig_p037_35.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

87 extracted references · 30 canonical work pages

  1. [1]

    DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.arXiv [cs.CL], 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y K Li, Y Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.arXiv [cs.CL], 2024. URLhttp://arxiv.org/abs/2402.03300

    arXiv 2024

  2. [2]

    High stakes testing in higher education and employment: appraising the evidence for validity and fairness.Am

    Paul R Sackett, Matthew J Borneman, and Brian S Connelly. High stakes testing in higher education and employment: appraising the evidence for validity and fairness.Am. Psychol., 63 (4):215–227, 2008. URLhttp://dx.doi.org/10.1037/0003-066X.63.4.215

    work page doi:10.1037/0003-066x.63.4.215 2008

  3. [3]

    High-stakes testing and curricular control: A qualitative metasynthesis.Educ

    Wayne Au. High-stakes testing and curricular control: A qualitative metasynthesis.Educ. Res., 36(5):258–267, 2007. URLhttp://dx.doi.org/10.3102/0013189x07306523

    work page doi:10.3102/0013189x07306523 2007

  4. [4]

    Why g matters: The complexity of everyday life.Intelligence, 24(1): 79–132, 1997

    Linda S Gottfredson. Why g matters: The complexity of everyday life.Intelligence, 24(1): 79–132, 1997. URLhttp://dx.doi.org/10.1016/s0160-2896(97)90014-3. 10

    work page doi:10.1016/s0160-2896(97)90014-3 1997

  5. [5]

    Homo heuristicus: Why biased minds make better inferences

    Gerd Gigerenzer and Henry Brighton. Homo heuristicus: Why biased minds make better inferences. InHeuristics, pages 2–26. Oxford University Press, 2011. URL http://dx.doi. org/10.1093/acprof:oso/9780199744282.003.0001

    arXiv 2011

  6. [6]

    Farrar, Straus and Giroux, 2011

    Daniel Kahneman.Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011

    2011

  7. [7]

    Daniel Bobrow.Natural language input computer problem solving system. 1964

    1964

  8. [8]

    MAWPS: A math word problem repository

    Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. MAWPS: A math word problem repository. InProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Stroudsburg, PA, USA, 2016. Association for Computational Linguistics. URL http://dx.d...

    doi:10.18653/v1/n16-1136 2016

Show all 87 references
  1. [9]

    Deep neural solver for math word problems

    Yan Wang, Xiaojiang Liu, and Shuming Shi. Deep neural solver for math word problems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA, 2017. Association for Computational Linguistics. URL http://dx. doi.org/10.1865...

    2017 doi

  2. [10]

    MathQA: Towards interpretable math word problem solving with operation-based formalisms.arXiv [cs.CL], 2019

    Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. MathQA: Towards interpretable math word problem solving with operation-based formalisms.arXiv [cs.CL], 2019. URLhttp://arxiv.org/abs/1905.13319

    2019 arXiv

  3. [11]

    Training verifiers to solve math word problems.arXiv [cs.LG], 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv [cs.LG], 2021. URL http://arxiv.org...

    2021 arXiv

  4. [12]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. arXiv [cs.LG], 2021. URLhttp://arxiv.org/abs/2103.03874

    2021 arXiv

  5. [13]

    A survey of deep learning for mathematical reasoning

    Pan Lu, Liang Qiu, Wenhao Yu, Sean Welleck, and Kai-Wei Chang. A survey of deep learning for mathematical reasoning. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Stroudsburg, PA, USA, 2023. Association for C...

    2023 doi

  6. [14]

    OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems.arXiv [cs.CL], 2024

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scienti...

    2024

  7. [15]

    PutnamBench: Evaluating neural theorem-provers on the putnam mathematical competition.arXiv [cs.AI], 2024

    George Tsoukalas, Jasper Lee, John Jennings, Jimmy Xin, Michelle Ding, Michael Jennings, Amitayush Thakur, and Swarat Chaudhuri. PutnamBench: Evaluating neural theorem-provers on the putnam mathematical competition.arXiv [cs.AI], 2024. URL http://arxiv.org/ abs/2407.11214

    2024 arXiv

  8. [16]

    Omni-MATH: A universal olympiad level mathematic benchmark for large language models

    Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni-MATH: A universal ...

    2024 arXiv

  9. [17]

    MiniF2F: a cross-system benchmark for formal olympiad-level mathematics.arXiv [cs.AI], 2021

    Kunhao Zheng, Jesse Michael Han, and Stanislas Polu. MiniF2F: a cross-system benchmark for formal olympiad-level mathematics.arXiv [cs.AI], 2021. URL http://arxiv.org/abs/ 2109.00110

    2021 arXiv

  10. [18]

    NaturalProofs: Mathematical theorem proving in natural language.arXiv [cs.IR], 2021

    Sean Welleck, Jiacheng Liu, Ronan Le Bras, Hannaneh Hajishirzi, Yejin Choi, and Kyunghyun Cho. NaturalProofs: Mathematical theorem proving in natural language.arXiv [cs.IR], 2021. URLhttp://arxiv.org/abs/2104.01112. 11

    2021 arXiv

  11. [19]

    Learn to explain: Multimodal reasoning via thought chains for science question answering.arXiv [cs.CL], 2022

    Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering.arXiv [cs.CL], 2022. URL http://arxiv.org/abs/ 2209.09513

    2022 arXiv

  12. [20]

    GeoQA: A geometric question answering benchmark towards multimodal numerical reasoning

    Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P Xing, and Liang Lin. GeoQA: A geometric question answering benchmark towards multimodal numerical reasoning. arXiv [cs.AI], 2021. URLhttp://arxiv.org/abs/2105.14517

    2021 arXiv

  13. [21]

    MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI.arXiv [cs.CL], 2023

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MMMU: A m...

    2023

  14. [22]

    Training language models to follow instructions with human feedback.arXiv [cs.CL], 2022

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and ...

    2022 arXiv

  15. [23]

    STaR: Bootstrapping reasoning with reasoning.arXiv [cs.LG], 2022

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D Goodman. STaR: Bootstrapping reasoning with reasoning.arXiv [cs.LG], 2022. URLhttp://arxiv.org/abs/2203.14465

    2022 arXiv

  16. [24]

    How far can camels go? exploring the state of instruction tuning on open resources.arXiv [cs.CL], 2023

    Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, and Hannaneh Hajishirzi. How far can camels go? exploring the state of instruction tuning on open resources.arXiv [cs.CL], 2...

    2023 arXiv

  17. [25]

    Reinforced self-training (ReST) for language modeling.arXiv [cs.CL], 2023

    Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. Reinforced self-training (ReST) for language mode...

    2023 arXiv

  18. [26]

    Proximal policy optimization algorithms.arXiv [cs.LG], 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv [cs.LG], 2017. URL http://arxiv.org/abs/1707. 06347

    2017

  19. [27]

    VinePPO: Unlocking RL potential for LLM reasoning through refined credit assignment.arXiv [cs.LG], 2024

    Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux. VinePPO: Unlocking RL potential for LLM reasoning through refined credit assignment.arXiv [cs.LG], 2024. URL http://arxiv.org/abs/2410. 01679

    2024

  20. [28]

    V APO: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv [cs.AI], 2025

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, Tiantian Fan, Zhengyin Du, Xiangpeng Wei, Xiangyu Yu, Gaohong Liu, Juncai Liu, Lingjun Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Ru Zhang, X...

    2025 arXiv

  21. [29]

    TÜLU 3: Pushing frontiers in open language model post-training.arXiv [cs.CL], 2024

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca ...

    2024 arXiv

  22. [30]

    Kimi k1.5: Scaling reinforcement learning with LLMs.arXiv [cs.AI], 2025

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, ...

    2025 arXiv

  23. [31]

    DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning.arXiv [cs.CL], 2025

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z F Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Fen...

    2025 arXiv

  24. [32]

    SimpleRL-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. SimpleRL-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025. URLhttp://arxiv.org/abs/2503.18892

    2025 arXiv

  25. [33]

    Light-R1: Curriculum SFT, DPO and RL for long COT from scratch and beyond.arXiv [cs.CL],

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, and Xiangzheng Zhang. Light-R1: Curriculum SFT, DPO and RL for long COT from scratch and beyond.arXiv [cs.CL],

  26. [34]

    Let’s verify step by step.arXiv [cs.LG],

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step.arXiv [cs.LG],

  27. [35]

    OpenAI o1 system card.arXiv [cs.AI], 2024

    OpenAI, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, ...

    2024 arXiv

  28. [36]

    Understanding R1-zero-like training: A critical perspective.arXiv [cs.LG], 2025

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding R1-zero-like training: A critical perspective.arXiv [cs.LG], 2025. URLhttp://arxiv.org/abs/2503.20783

    2025 arXiv

  29. [37]

    DeepCoder: A fully open-source 14B coder at O3-mini level

    Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaoxiang Shi, Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Er- ran Li, Raluca Ada Popa, and Ion Stoica. DeepCoder: A fully open-source 14B coder at O3-mini level. https://pretty-radio-b75.notion....

    2025

  30. [38]

    VLM-R1: A stable and generalizable R1-style large vision-language model.arXiv [cs.CV], 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. VLM-R1: A stable and generalizable R1-style large vision-language model.arXiv [cs.CV], 2025. URL http://arxiv.org/a...

    2025 arXiv

  31. [39]

    GRPO-LEAD: A difficulty-aware reinforcement learning approach for concise mathematical reasoning in language models.arXiv [cs.CL], 2025

    Jixiao Zhang and Chunsheng Zuo. GRPO-LEAD: A difficulty-aware reinforcement learning approach for concise mathematical reasoning in language models.arXiv [cs.CL], 2025. URL http://arxiv.org/abs/2504.09696. 14

    2025

  32. [40]

    D1: Scaling reasoning in diffusion large language models via reinforcement learning.arXiv [cs.CL], 2025

    Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. D1: Scaling reasoning in diffusion large language models via reinforcement learning.arXiv [cs.CL], 2025. URL http://arxiv.org/abs/2504.12216

    2025 arXiv

  33. [42]

    Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?arXiv [cs.AI], 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?arXiv [cs.AI], 2025. URLhttp://arxiv.org/abs/2504.13837

    2025 arXiv

  34. [43]

    Assessing diversity collapse in reasoning

    Xingyu Dang, Christina Baek, J Zico Kolter, and Aditi Raghunathan. Assessing diversity collapse in reasoning. InScaling Self-Improving Foundation Models without Human Supervision,

  35. [44]

    Learning to summarize from human feedback

    Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M Ziegler, Ryan Lowe, Chelsea V oss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback. arXiv [cs.CL], 2020. URLhttp://arxiv.org/abs/2009.01325

    2020 arXiv

  36. [45]

    Self-consistency improves chain of thought reasoning in language models.arXiv [cs.CL], 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv [cs.CL], 2022. URLhttp://arxiv.org/abs/2203.11171

    2022 arXiv

  37. [46]

    URLhttps://openreview.net/forum?id=AMiKsHLjQh

  38. [47]

    HybridFlow: A flexible and efficient RLHF framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. HybridFlow: A flexible and efficient RLHF framework. arXiv preprint arXiv: 2409. 19256, 2024

    2024

  39. [48]

    Large language monkeys: Scaling inference compute with repeated sampling, 2024

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling, 2024. URLhttp://arxiv.org/abs/2407.21787

    2024 arXiv

  40. [49]

    Qwen2.5: A party of foundation models, 2024

    Qwen Team. Qwen2.5: A party of foundation models, 2024. URL https://qwenlm.github. io/blog/qwen2.5/

    2024

  41. [50]

    GLoRe: When, where, and how to im- prove LLM reasoning via global and local refinements.arXiv [cs.CL], 2024

    Alex Havrilla, Sharath Raparthy, Christoforus Nalmpantis, Jane Dwivedi-Yu, Maksym Zhu- ravinskyi, Eric Hambro, and Roberta Raileanu. GLoRe: When, where, and how to im- prove LLM reasoning via global and local refinements.arXiv [cs.CL], 2024. URL http: //arxiv.org/abs/2402.10963

    2024 arXiv

  42. [51]

    Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective STaRs.arXiv [cs.CL], 2025

    Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective STaRs.arXiv [cs.CL], 2025. URLhttp://arxiv.org/abs/2503.01307

    2025 arXiv

  43. [52]

    Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhi- fang Sui. Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vo...

    2024 doi

  44. [53]

    Bridging the human-ai knowledge gap: Concept discovery and transfer in alphazero, 2023

    Lisa Schut, Nenad Tomasev, Tom McGrath, Demis Hassabis, Ulrich Paquet, and Been Kim. Bridging the human-ai knowledge gap: Concept discovery and transfer in alphazero, 2023. URL https://arxiv.org/abs/2310.16410

    2023 arXiv

  45. [54]

    To backtrack or not to backtrack: When sequential search limits model reasoning, 2025

    Tian Qin, David Alvarez-Melis, Samy Jelassi, and Eran Malach. To backtrack or not to backtrack: When sequential search limits model reasoning, 2025. URL https://arxiv.org/ abs/2504.07052. 15

    2025

  46. [55]

    Mastering chess and shogi by self-play with a general reinforcement learning algorithm, 2017

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. Mastering chess and shogi by self-play with a general reinforce...

    2017 arXiv

  47. [56]

    Emergence of hidden capabilities: Exploring learning dynamics in concept space, 2024

    Core Francisco Park, Maya Okawa, Andrew Lee, Hidenori Tanaka, and Ekdeep Singh Lubana. Emergence of hidden capabilities: Exploring learning dynamics in concept space, 2024. URL https://arxiv.org/abs/2406.19370

    2024 arXiv

  48. [57]

    Competition dynamics shape algorithmic phases of in-context learning, 2025

    Core Francisco Park, Ekdeep Singh Lubana, Itamar Pres, and Hidenori Tanaka. Competition dynamics shape algorithmic phases of in-context learning, 2025. URL https://arxiv.org/ abs/2412.01003

    2025 arXiv

  49. [58]

    Dick, and Hidenori Tanaka

    Maya Okawa, Ekdeep Singh Lubana, Robert P. Dick, and Hidenori Tanaka. Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task, 2024. URL https://arxiv.org/abs/2310.09336

    2024 arXiv

  50. [59]

    Dick, and Hidenori Tanaka

    Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P. Dick, and Hidenori Tanaka. A percolation model of emergence: Analyzing transformers trained on a formal language, 2024. URL https://arxiv.org/abs/2408.12578

    2024 arXiv

  51. [60]

    Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip H. S. Torr, Amartya Sanyal, and Puneet K. Dokania. What makes and breaks safety fine-tuning? a mechanistic study, 2024. URLhttps://arxiv.org/abs/2407.10264

    2024 arXiv

  52. [61]

    Physics of language models: Part 3.2, knowledge manipula- tion, 2024

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.2, knowledge manipula- tion, 2024. URLhttps://arxiv.org/abs/2309.14402

    2024 arXiv

  53. [62]

    Reinforcement learning for reasoning in large language models with one training example, 2025

    Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. Reinforcement learning for reasoning in large language models with one training example, 202...

    2025 arXiv

  54. [63]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https://arxiv. org/abs/1706.03762

    2023 arXiv

  55. [64]

    Michaud, Ziming Liu, Uzay Girit, and Max Tegmark

    Eric J. Michaud, Ziming Liu, Uzay Girit, and Max Tegmark. The quantization model of neural scaling, 2024. URLhttps://arxiv.org/abs/2303.13506

    2024 arXiv

  56. [65]

    sharpens

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  57. [67]

    Qwen2 technical report.arXiv preprint arXiv:2407

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, and Others. Qwen2 technical report.arXiv preprint arXiv:2407. 10671, 2024

    2024

  58. [69]

    the first two decimal places are 0.27

    Visualize and describe the region formed by the union of the nonagon and the 1-unit neighborhood around its perimeter. 2. Analyze the boundary of this region, identifying straight and curved parts. 3. Calculate the lengths of these boundary segments. 4. Sum these lengths to fi...

  59. [70]

    = 3 + 2 √ 2− √ 3− √ 6, not the original expression. Then the student writes: 1 + 2 √ 2 + 1− √ 3− √ 6 = (2 √ 2 + 1)−( √ 3 + √ 6) which is also incorrect because1+2 √ 2+1− √ 3− √ 6 = 2+2 √ 2− √ 3− √ 6, which is the original expression, but the grouping is not in the formA(1 + √ ...

  60. [71]

    Identify the units digit of8: The units digit of8is8

  61. [72]

    Determine the pattern in the units digits of powers of8: -8 1 = 8(units digit is8) -8 2 = 64(units digit is4) -8 3 = 512(units digit is2) -8 4 = 4096(units digit is6) -8 5 = 32768(units digit is8) -8 6 = 262144(units digit is4) -8 7 = 2097152(units digit is2) -8 8 = 16777216(u...

  62. [73]

    The prime numbers less than 30 are correctly listed as: 2, 3, 5, 7, 11, 13, 17, 19, 23, 29

  63. [74]

    The odd numbers greater than zero are correctly listed as: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27

  64. [75]

    Since 2 is prime but not odd, it should not be in the intersection

    The intersection should be the prime numbers less than 30 that are also odd numbers greater than zero. Since 2 is prime but not odd, it should not be in the intersection

  65. [76]

    The student incorrectly includes 2 in the intersection set, which is a factual mistake about the parity of 2 (2 is even, not odd)

  66. [77]

    The student concludes the intersection has 10 elements, but excluding 2, the intersection has 9 elements: 3, 5, 7, 11, 13, 17, 19, 23, 29

  67. [78]

    This is a basic mathematical fact error (2 is not odd)

  68. [79]

    There is no basic logic mistake; the student’s reasoning is consistent except for the factual error about 2. Final grading: (1) Basic mathematical factual mistake? YES (2 is not odd but included in intersection) (2) Is the mistake elementary or high school? ELEMENTARY (parity ...

  69. [80]

    2 is odd

    This is a logical inconsistency but not a factual mathematical mistake. The sine function’s maximum occurs at π 2 plus multiples of2π, so the student’s equation and solution forCis correct. The confusion is in interpreting the phase shift and the smallest possiblec. There is n...

    2010

  70. [81]

    unknown" token <unk>, beginning of sequence token <bos>, padding token <pad>, prompt/completion separation token “:

    Dividing by 2 is a mistake because rows and columns are different lines and should not be double counted. Also, the student misses other lines with 3 points, such as the lines with slope 1 or -1 that pass through other points (e.g., the middle points in the grid). So the count...

  71. [82]

    An entry TS(s, a)specifies the next state s′ resulting from taking action a in state s

    State Transition Table ( TS): This is a matrix of dimensions NS ×N A. An entry TS(s, a)specifies the next state s′ resulting from taking action a in state s. This ta- ble, ‘state_transitions‘, is initialized by drawing each entry uniformly at random from {0, . . . , NS −1}. 39

  72. [83]

    An entry TP (p, a)specifies the next problem state p′ when action a is taken while the current problem state is p

    Problem State Transition Table (TP ): This is a matrix of dimensions NP ×N A. An entry TP (p, a)specifies the next problem state p′ when action a is taken while the current problem state is p. The table ‘problem_state_transitions’ is constructed such that for any given current...

  73. [84]

    Each row s contains the probability distribution Pα(A|S=s) over actions, generated from Dirichlet(αtrain ·1 NA )

    Action Probability Table for Pre-training (Paction): This table, ‘action_probs’, has dimen- sions NS ×N A. Each row s contains the probability distribution Pα(A|S=s) over actions, generated from Dirichlet(αtrain ·1 NA ). This table is only used for action selection in the ‘tra...

  74. [85]

    Each row a contains the probability distribution Pβ(C|A=a) over contexts, generated from Dirichlet(βtrain ·1 NC )

    Action-Context Probability Table for Pre-training ( Pcontext): This table, ‘ac- tion_context_prob_train’, has dimensions NA ×N C. Each row a contains the probability distribution Pβ(C|A=a) over contexts, generated from Dirichlet(βtrain ·1 NC ). This table is only used for cont...

  75. [86]

    question

    Action-Context Probability Table for RL/Eval: For the ‘train_rl’ and ‘eval’ splits, the context ct for an action at is chosen uniformly. This is represented by ‘ac- tion_context_prob_test’, anN A ×N C matrix where each entry is1/N C. Each individual data instance, comprising a...

  76. [87]

    This k defines the length of the action sequence and consequently the number of state transitions

    Determine Trajectory Length: The number of actions,k, for the current instance is selected by drawing an integer uniformly from the interval [kmin, kmax]. This k defines the length of the action sequence and consequently the number of state transitions. 2.Initialize Sequences:...

  77. [88]

    S <s 0>" Then, for each step j from 0 to k: Append problem state:

    Generate Irrelevant Context Tokens: A sequence of nirr_ctx irrelevant to- kens, (i0, . . . , inirr_ctx−1), is generated by sampling each token uniformly at ran- dom from {0, . . . , ndim −1} . Note: ‘n_dim’ is used here as per the code ‘np.random.randint(0,n_dim,size=num_irr_c...

  78. [2023]

    URLhttp://arxiv.org/abs/2305.20050

    arXiv

  79. [2025]

    URLhttp://arxiv.org/abs/2503.10460

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.