Pith. sign in

REVIEW 4 major objections 5 minor 60 references

System-2 Mathematical Reasoning via Enriched Instruction Tuning

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that enriching human-annotated math solutions with AI-generated reasoning plans and step-level elaborations enables open-source LLMs to match tool-augmented methods on GSM8K and MATH, achieving 84.1% and 32.5% accuracy…

desk verdict Plausible method for enriching math reasoning data, but the central causal claim is undermined by a missing control condition. read the letter →

arxiv 2412.16964 v2 pith:AP3WMRZE submitted 2024-12-22 cs.AI cs.CL

Huanqia Cai , Yijun Yang , Zhifeng Li This is my paper · ORCID
classification cs.AIcs.CL
keywords EnrichedInstructionTuningsystem-2reasoningmathematicalfine-tuningplanstepGSM8KMATH
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that open-source large language models can be taught deliberate, step-by-step mathematical reasoning without external tools or symbolic verifiers, simply by fine-tuning them on instruction data that has been enriched with AI-generated reasoning plans and step-level elaborations. It introduces Enriched Instruction Tuning (EIT), which takes existing human-annotated question-answer pairs and uses GPT-4 to add a high-level plan and fill in missing reasoning contexts, producing smoother and more fine-grained reasoning trajectories. Training LLaMA-2-70B on this enriched data yields 84.1% accuracy on GSM8K and 32.5% on MATH, outperforming prior fine-tuning and prompting baselines and matching tool-augmented methods. The paper also finds that more fine-grained reasoning steps and more enriched data both improve performance, suggesting that data granularity as well as quantity matters.

What carries the argument

The central mechanism is the two-stage prompting pipeline applied to existing human-annotated datasets: ERP generates a high-level plan that decomposes the instruction into simpler objectives, and ERS then expands the original response by filling in the missing contextual and logical steps under the guidance of that plan. The enriched responses preserve the original reasoning pathway and final answers (achieving 99.7% final-answer agreement with the original annotations) while being much more fine-grained, with lower perplexity than original or other augmented datasets. This enriched data is then used for standard supervised fine-tuning, improving the model's ability to produce system-2-like reasoning trajectories at inference time.

What would settle it

Fine-tune LLaMA-2-70B on plain GPT-4-generated chain-of-thought completions of the same human-annotated answers, matched for length and detail but without the explicit 'planning' and 'solution' scaffolding; if this control achieves similar accuracy on GSM8K and MATH (within noise), the causal role of the ERP/ERS structure is falsified.

Watch

Extended reading notes

Core claim

The central claim is that the scarcity of deliberate multi-step reasoning data is a primary limiting factor for mathematical reasoning in LLMs, and that this scarcity can be addressed by enriching existing human-annotated datasets with AI-generated reasoning plans and step-level elaborations. The method, EIT, operates in two stages: Enriching with Reasoning Plan (ERP), where a privileged LLM decomposes the question into a sequence of lower-level objectives, and Enriching with Reasoning Step (ERS), where the same LLM fills in the reasoning contexts and implicit steps that human annotators omitted. The key insight is that human-annotated answers serve as 'meta-knowledge' that guides the LLM to generate more detailed and accurate reasoning trajectories, turning the task from open-ended generation into a fill-in-the-blank elaboration. Fine-tuning LLaMA-2-70B on this enriched data achieves 84.1% on GSM8K and 32.5% on MATH, surpassing state-of-the-art fine-tuning and prompting methods and matching tool-augmented approaches without using any external verifier.

Load-bearing premise

The load-bearing premise is that the observed accuracy gains come from the ERP and ERS enrichment structure itself, rather than from the simple fact that GPT-4 is a better generator of verbose solutions than the original annotators or standard CoT data; the paper does not include a control condition separating these factors.

Editorial extensions

If this is right

  • Fine-tuning on enriched reasoning trajectories can match or exceed the accuracy of tool-augmented methods (e.g., EIT-70B reaches 84.1% on GSM8K versus 83.9% for MathCoder and 84.3% for ToRA without self-consistency), suggesting that external tools may not be necessary for competitive mathematical reasoning.
  • More fine-grained reasoning steps in the training data improve test accuracy, indicating that the granularity of reasoning trajectories is as important as dataset size when training mathematical reasoners.
  • Combining EIT with question-augmentation methods such as MetaMath yields further gains (e.g., +3.6% on MATH and +6.9% on GSM8K over question augmentation alone), showing that response enrichment and question augmentation are complementary.
  • EIT improves performance even with small training sets (7.5k examples) and scales better than existing augmentation methods when more data is added, in contrast to the degradation observed with RFT-augmented data.
  • The lower perplexity of EITMath data correlates with higher downstream accuracy, supporting the claim that data simplicity and learnability are key drivers of reasoning performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The absence of a control condition where GPT-4 is prompted to write plain detailed chain-of-thought completions of the same original responses means the causal attribution of gains to the ERP/ERS scaffolding itself is not fully established; the gains could partly stem from GPT-4's superior generation quality rather than the plan-and-elaboration structure.
  • If the enrichment structure is what matters, then a similar approach could be applied to non-mathematical reasoning domains by replacing the math-specific prompts with domain-appropriate plan-and-elaboration templates, potentially improving commonsense or scientific reasoning as well.
  • The paper's findings suggest that the bottleneck for system-2 reasoning may be not just data quantity but the explicitness of intermediate reasoning steps; this could motivate interactive 'reasoning curriculum' where models are fine-tuned on progressively more detailed trajectories.
  • A direct testable extension would be to compare EIT against fine-tuning on plain GPT-4-generated expanded solutions of identical length, to isolate the effect of the plan-then-fill structure; if the plan structure adds no benefit, the result would collapse to a data-quality effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Enriched Instruction Tuning (EIT), a data-centric method that uses GPT-4 to enrich existing human-annotated math solutions with a high-level reasoning plan (ERP) and step-level elaborations (ERS), then fine-tunes open-source LLaMA-2 models on the resulting EITMath dataset. The authors report 84.1% on GSM8K and 32.5% on MATH for LLaMA-2-70B, surpassing several fine-tuning and prompting baselines and approaching tool-augmented systems. Ablations in Table 3 attribute gains to ERP and ERS separately and in combination, and additional experiments examine scaling, perplexity, and self-consistency.

Significance. If the causal attribution is supported, EIT is a practical and inexpensive route to improving mathematical reasoning in open-weight models without verifiers or search tools, and its claim that finer-grained reasoning trajectories are beneficial is a useful empirical insight. The paper has notable strengths: the prompts for ERP, ERS, and their ablations are given in full in Appendix H; the ablation isolates the contribution of each enrichment component; the benchmark comparisons cover multiple model sizes, MATH subtopics, and self-consistency settings. However, because the main experimental design lacks a control for GPT-4 as the data generator, the central claim that the enrichment structure itself causes the gains is not yet established.

major comments (4)
  1. [Section 3.1 and Table 3] The central causal claim is confounded. Every enriched condition in Table 3 is generated by GPT-4-1106-preview, while the baselines are either original human-annotated responses or standard CoT data; there is no control condition in which GPT-4 is prompted to write a plain, detailed chain of thought directly from the question without being given the human answer and without ERP/ERS scaffolding. Without such a matched control, the gains attributed to ERP and ERS could equally be due to distillation of GPT-4's own reasoning ability or to increased response length/token count, which the paper itself identifies as improving performance in Fig. 3 (middle). I recommend adding an ablation trained on plain GPT-4-generated CoT solutions matched for dataset size and response length.
  2. [Appendix B and Examples H.1, H.2, H.4] The reported 99.7% final-answer agreement is not independent evidence of data quality. The enrichment prompts explicitly instruct GPT-4 not to change the original pathways or modify formulas and results, so high final-answer agreement is enforced by the generation protocol rather than by the quality of the enrichment. To support the claim that EITMath is high-quality, the authors should measure step-level correctness or provide a human evaluation of the added reasoning steps, rather than relying on final-answer agreement with the source response.
  3. [Section 6.2 and Fig. 3 (middle)] The claim that more fine-grained reasoning steps improve performance is conflated with response length and perplexity. The datasets compared in Fig. 3 (middle) differ in average token count, and the paper already shows that lower perplexity and more tokens correlate with better accuracy. Without a control that keeps the amount of new mathematical content fixed while varying granularity, or a control using plain verbose CoT of matched length, the observed gains do not specifically demonstrate that step-level granularity, rather than verbosity or token count, is the operative factor.
  4. [Section 4.2 and Table 2] The scale-up from 7.5k to 70k examples is underspecified. The text says the larger dataset was constructed by sampling GPT-4's output distribution with different temperature coefficients, but it does not report the number of samples, the temperature values, whether the samples were filtered or deduplicated, or how the 70k examples are split between MATH and GSM8K. This matters because the scaling-law claim in Section 6.2 and the comparison to methods trained on much larger datasets depend on the construction of this 70k set.
minor comments (5)
  1. [Table 4 and Section 6.1] The text states a 2.7% improvement over MetaMath-70B on MATH, but Table 4 lists two MetaMath-70B entries (26.6 and 29.8); the comparison should be identified explicitly, including which fine-tuning setup is used.
  2. [Table 6 and Appendix C] The phrases '25.3% and 51.2% improvement' and '22.2% and 13.1% improvement' are ambiguous; the authors should state whether these are absolute percentage-point gains or relative improvements.
  3. [Fig. 3 (right)] The perplexity is computed with an under-finetuned LLaMA-2-7B while the accuracy is reported for a fine-tuned LLaMA-2-70B; the evaluation protocol should be stated precisely, and the claim that lower perplexity 'leads to' better performance should be framed as a correlation.
  4. [Appendix A] The sentence describing 'the work most closely related to ours is Auto-CoT' cites Jin et al. 2024, but that reference is titled 'The Impact of Reasoning Step Length on Large Language Models'; the citation appears mismatched and should be corrected.
  5. [Appendix B and Section 6.4] The comparison of GPT-4's 99.7% accuracy with EIT prompts versus 71.5% without EIT prompts lacks details about the evaluation set, sampling temperature, and number of generations; these details should be reported.

Circularity Check

1 steps flagged · score 2.0 of 10

Local circularity in Appendix B's 99.7% answer-agreement validation; core benchmark gains are empirical and not circular.

  1. self definitional [Appendix B (Quality Analysis of EITMath), with generation prompts in Appendix H.1/H.2; see also Sec. 3.1]
    "we compared the final answers of the generated data with the original answers to calculate accuracy. Our results show an accuracy rate of 99.7% ... "be careful not to change the original pathways or modify formulas and results, as well as special symbol expressions""

    The 99.7% final-answer agreement is presented as evidence that EIT-generated data are high quality, but the ERS/ERP prompts explicitly instruct GPT-4 to preserve the original formulas and results, and Sec. 3.1 states EIT strictly follows the original response without involving additional calculations. Therefore the generated final answers coincide with the human final answers by construction; the figure measures prompt compliance, not independent reasoning correctness. The comparison with 71.5% 'without our EIT prompts' is not matched because in that condition GPT-4 was not given a target answer to preserve.

full rationale

EIT's central result is an empirical benchmark comparison: fine-tuning LLaMA-2 on GPT-4-enriched versions of MATH/GSM8K yields 84.1% and 32.5% (Tables 3 and 4). These accuracies are not derived from fitted parameters, from a load-bearing self-citation, or from the definition of the enrichment; they are external test-set measurements, and Table 3 is a legitimate ablation among trained models. I find no load-bearing self-citation: the only overlapping-author reference (Gao et al. 2024, CoSC-code) appears as a baseline, not as justification of EIT. The genuine circularity is localized to Appendix B: the 99.7% final-answer agreement is used as a data-quality validation, but the ERS/ERP prompts explicitly instruct GPT-4 not to change formulas or results, so the agreement is enforced by construction and measures prompt compliance, not reasoning quality. This does not invalidate the fine-tuning results, but it makes the paper's own quality claim self-definitional. A separate concern that is a confound rather than circularity: all enriched conditions use GPT-4 while baselines use human-annotated or standard CoT data, and there is no matched plain-GPT-4-CoT control; this weakens the causal attribution to ERP/ERS but is not a reduction of the result to its inputs. Score 2 reflects the local circular step while recognizing that the central derivation chain is self-contained empirical evaluation.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method introduces no new scientific entities. Its load-bearing components are assumptions about data quality and the teacher model, plus underspecified training choices.

free parameters (4)
  • learning_rate = 2e-5
    Standard optimizer hyperparameter; not fitted to benchmark, but affects results.
  • number_of_epochs = 3
    Choice of training epochs.
  • training_dataset_size = 70k (main), 7.5k (ablation)
    Dataset scale selected for comparison; the 70k construction relies on sampling GPT-4 with different temperatures.
  • GPT4_sampling_temperature = not reported
    Used to create diverse 70k enriched responses; values omitted, so replication depends on unknown settings.
assumptions (3)
  • domain assumption Human-annotated responses in GSM8K and MATH are correct enough to be used as meta-knowledge for enrichment.
    Section 3.1 states EIT 'strictly follows the original response' and treats it as ground truth; if the original reasoning is wrong, enriched data inherits the error.
  • domain assumption GPT-4, when prompted, enriches responses without changing the final answer.
    Appendix B reports 99.7% final-answer match, but that figure is partly enforced by the prompt instruction not to modify results; independent reasoning quality is not assessed.
  • domain assumption Supervised fine-tuning with next-token prediction is an adequate training paradigm for instilling system-2 reasoning.
    The method follows standard SFT (Eq. 1); no alternative training objective is considered.

how reviews work

0 comments
Cite this review

Pith. "Pith review of System-2 Mathematical Reasoning via Enriched Instruction Tuning." pith.science (2026). https://pith.science/paper/AP3WMRZE

@misc{pith2026241216964,
  author       = {Pith},
  title        = {Pith review of: System-2 Mathematical Reasoning via Enriched Instruction Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AP3WMRZE}},
  note         = {Machine review of arXiv:2412.16964}
}
read the original abstract

Solving complex mathematical problems via system-2 reasoning is a natural human skill, yet it remains a significant challenge for current large language models (LLMs). We identify the scarcity of deliberate multi-step reasoning data as a primary limiting factor. To this end, we introduce Enriched Instruction Tuning (EIT), a method that enriches existing human-annotated mathematical datasets by synergizing human and AI feedback to create fine-grained reasoning trajectories. These datasets are then used to fine-tune open-source LLMs, enhancing their mathematical reasoning abilities without reliance on any symbolic verification program. Concretely, EIT is composed of two critical steps: Enriching with Reasoning Plan (ERP) and Enriching with Reasoning Step (ERS). The former generates a high-level plan that breaks down complex instructions into a sequence of simpler objectives, while ERS fills in reasoning contexts often overlooked by human annotators, creating a smoother reasoning trajectory for LLM fine-tuning. Unlike existing CoT prompting methods that generate reasoning chains only depending on LLM's internal knowledge, our method leverages human-annotated initial answers as ``meta-knowledge'' to help LLMs generate more detailed and precise reasoning processes, leading to a more trustworthy LLM expert for complex mathematical problems. In experiments, EIT achieves an accuracy of 84.1% on GSM8K and 32.5% on MATH, surpassing state-of-the-art fine-tuning and prompting methods, and even matching the performance of tool-augmented methods.

Figures

Figures reproduced from arXiv: 2412.16964 by the authors.

Figure 1
Figure 1. Comparison of LLM’s response generated by four different methods on a randomly selected problem from the MATH [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of Enriched Instruction Tuning (EIT), which first leverages a privileged LLM, e.g., GPT-4, to produce enriched reasoning steps for the existing mathematical instruction dataset through our proposed ERP and ERS prompting methods, and then trains an LLM on this enriched dataset via instruction tuning. Note that the Original Response provided by human annotators overlooks the bold context in Enriched Response,… view at source ↗
Figure 3
Figure 3. left: Scaling up of performance on MATH as adding our EITMath dataset for LLM fine-tuning. Different colored bars represent MetaMathQA combined with Rejection sampling Fine-Tuning (RFT) and EITMath as the training set. middle: Scaling up of performance on MATH as more fine-grained reasoning steps created for fine-tuning. We use average tokens (more is better) of the response to measure its granularity. right: Perple… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 7 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [5]

    Jiang, Jia Deng, Stella Biderman, and Sean Welleck

    Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen Marcus McAleer, Albert Q. Jiang, Jia Deng, Stella Biderman, and Sean Welleck. 2023 b . https://api.semanticscholar.org/CorpusID:264172303 Llemma: An open language model for mathematics . ArXiv, abs/2310.10631

    arXiv 2023

  5. [6]

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954

    arXiv 2024

  6. [7]

    David Brandfonbrener, Sibi Raja, Tarun Prasad, Chloe Loughridge, Jianang Yang, Simon Henniger, William E Byrd, Robert Zinkov, and Nada Amin. 2024. Verified multi-step synthesis using large language models and monte carlo tree search. arXiv preprint arXiv:2402.08147

    arXiv 2024

  7. [8]

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. 2023. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818

    arXiv 2023

  8. [9]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Ma teusz Litwin, ...

    arXiv 2020

Show all 60 references
  1. [10]

    S \'e bastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712

    2023 arXiv

  2. [11]

    Ziru Chen, Michael White, Raymond Mooney, Ali Payani, Yu Su, and Huan Sun. 2024. When is tree search useful for llm planning? it depends on the discriminator. arXiv preprint arXiv:2402.10890

    2024 arXiv

  3. [12]

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023)

    2023

  4. [13]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    2021 arXiv

  5. [14]

    Kahneman Daniel. 2017. Thinking, fast and slow

    2017

  6. [15]

    Daocheng Fu, Xin Li, Licheng Wen, Min Dou, Pinlong Cai, Botian Shi, and Yu Qiao. 2024. Drive like a human: Rethinking autonomous driving with large language models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 910--919

    2024

  7. [16]

    Kuofeng Gao, Huanqia Cai, Qingyao Shuai, Dihong Gong, and Zhifeng Li. 2024. Embedding self-correction as an inherent ability in large language models for enhanced mathematical reasoning. arXiv preprint arXiv:2410.10735

    2024 arXiv

  8. [17]

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, et al. 2023. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.17452

    2023 arXiv

  9. [18]

    Thilo Hagendorff, Sarah Fabi, and Michal Kosinski. 2022. Thinking fast and slow in large language models. arXiv preprint arXiv:2212.05206

    2022 arXiv

  10. [19]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874

    2021 arXiv

  11. [20]

    Yann Hicke, Anmol Agarwal, Qianou Ma, and Paul Denny. 2023. Chata: Towards an intelligent question-answer teaching assistant using open-source llms. arXiv preprint arXiv:2311.02775

    2023 arXiv

  12. [21]

    Fan Huang, Haewoon Kwak, and Jisun An. 2023. Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech. In Companion proceedings of the ACM web conference 2023, pages 294--297

    2023

  13. [22]

    Mingyu Jin, Qinkai Yu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, Mengnan Du, et al. 2024. The impact of reasoning step length on large language models. arXiv preprint arXiv:2401.04925

    2024 arXiv

  14. [23]

    Nitish Joshi, Koushik Kalyanaraman, Zhiting Hu, Kumar Chellapilla, He He, and Li Erran Li. 2023. Improving multi-hop reasoning in llms by learning from rich human feedback. In Neuro-Symbolic Learning and Reasoning in the era of Large Language Models

    2023

  15. [24]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

    2020 arXiv

  16. [25]

    Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  17. [26]

    Yann LeCun. 2022. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62(1):1--62

    2022

  18. [27]

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267

    2023 arXiv

  19. [28]

    Bingbin Liu, Sebastien Bubeck, Ronen Eldan, Janardhan Kulkarni, Yuanzhi Li, Anh Nguyen, Rachel Ward, and Yi Zhang. 2023. Tinygsm: achieving> 80\ arXiv preprint arXiv:2312.09241

    2023 arXiv

  20. [29]

    Pan Lu, Liang Qiu, Wenhao Yu, Sean Welleck, and Kai-Wei Chang. 2022. A survey of deep learning for mathematical reasoning. arXiv preprint arXiv:2212.10535

    2022 arXiv

  21. [30]

    Zimu Lu, Aojun Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, and Hongsheng Li. 2024. Mathgenie: Generating synthetic data with question back-translation for enhancing mathematical reasoning of llms. arXiv preprint arXiv:2402.16352

    2024 arXiv

  22. [31]

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583

    2023 arXiv

  23. [32]

    Max Marion, Ahmet \"U st \"u n, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. 2023. When less is more: Investigating data pruning for pretraining llms at scale. arXiv preprint arXiv:2309.04564

    2023 arXiv

  24. [33]

    Arindam Mitra, Hamed Khanpour, Corby Rosset, and Ahmed Awadallah. 2024. Orca-math: Unlocking the potential of slms in grade school math. arXiv preprint arXiv:2402.14830

    2024 arXiv

  25. [34]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  26. [35]

    Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only. ar...

    2023 arXiv

  27. [36]

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training

    2018

  28. [37]

    Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2021. Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.08633

    2021 arXiv

  29. [38]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2024. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36

    2024

  30. [39]

    MosaicML NLP Team et al. 2023. Introducing mpt-7b: A new standard for open-source, commercially usable llms, 2023. URL www. mosaicml. com/blog/mpt-7b. Accessed, pages 05--05

    2023

  31. [40]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023 a . Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  32. [41]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 b . Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  33. [42]

    Dakuo Wang, Elizabeth Churchill, Pattie Maes, Xiangmin Fan, Ben Shneiderman, Yuanchun Shi, and Qianying Wang. 2020. From human-human collaboration to human-ai collaboration: Designing ai systems that can work together with people. In Extended abstracts of the 2020 CHI conferen...

    2020

  34. [43]

    Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. 2023 a . Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning. arXiv preprint arXiv:2310.03731

    2023 arXiv

  35. [44]

    Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Y Wu, and Zhifang Sui. 2023 b . Math-shepherd: Verify and reinforce llms step-by-step without human annotations. CoRR, abs/2312.08935

    2023 arXiv

  36. [45]

    Xinru Wang, Hannah Kim, Sajjadur Rahman, Kushan Mitra, and Zhengjie Miao. 2024. Human-llm collaborative annotation through effective verification of llm labels. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 1--21

    2024

  37. [46]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  38. [47]

    Xuezhi Wang and Denny Zhou. 2024. Chain-of-thought reasoning without prompting. arXiv preprint arXiv:2402.10200

    2024 arXiv

  39. [48]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824--24837

    2022

  40. [49]

    Jason Weston and Sainbayar Sukhbaatar. 2023. System 2 attention (is something you might need too). arXiv preprint arXiv:2311.11829

    2023 arXiv

  41. [50]

    Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shieh. 2024. Monte carlo tree search boosts reasoning via iterative preference learning. arXiv preprint arXiv:2405.00451

    2024 arXiv

  42. [51]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244

    2023 arXiv

  43. [52]

    Yifan Xu, Xiao Liu, Xinghan Liu, Zhenyu Hou, Yueyan Li, Xiaohan Zhang, Zihan Wang, Aohan Zeng, Zhengxiao Du, Wenyi Zhao, et al. 2024. Chatglm-math: Improving math problem-solving in large language models with a self-critique pipeline. arXiv preprint arXiv:2404.02893

    2024 arXiv

  44. [53]

    Yijun Yang, Tianyi Zhou, Kanxue Li, Dapeng Tao, Lusong Li, Li Shen, Xiaodong He, Jing Jiang, and Yuhui Shi. 2024. Embodied multi-modal agent trained by an llm from a parallel textworld. In CVPR

    2024

  45. [54]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36

    2024

  46. [55]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629

    2022 arXiv

  47. [56]

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12284

    2023 arXiv

  48. [57]

    Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. 2024. Distilling system 2 into system 1. arXiv preprint arXiv:2407.06023

    2024 arXiv

  49. [58]

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. 2023. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825

    2023 arXiv

  50. [59]

    Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2023. Mammoth: Building math generalist models through hybrid instruction tuning. arXiv preprint arXiv:2309.05653

    2023 arXiv

  51. [60]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493

    2022 arXiv

  52. [61]

    Siyu Zhou, Tianyi Zhou, Yijun Yang, Guodong Long, Deheng Ye, Jing Jiang, and Chengqi Zhang. 2024. Wall-e: World alignment by rule learning improves world model-based llm agents. arXiv preprint arXiv:2410.07484

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.