Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Chain-of-Thought Tokens are Computer Program Variables

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Chain-of-thought tokens store and reuse values like program variables

desk verdict A useful empirical study of CoT as variable storage, with strong intervention evidence on explicit tokens and a weaker, probe-based complexity claim. read the letter →

arxiv 2505.04955 v1 pith:ZC4FGFET submitted 2025-05-08 cs.CL cs.AI

Fangwei Zhu , Peiyi Wang , Zhifang Sui This is my paper · ORCID
classification cs.CLcs.AI
keywords chain-of-thoughtprogramvariablesintermediatevalueslatenttokenscausalinterventionmulti-digitmultiplicationdynamicprogramminglinearprobing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the intermediate tokens generated during chain-of-thought reasoning act like variables in a computer program: their main job is to store partial results that later reasoning steps read, rather than to present a readable argument. On two tasks that demand serial computation, multi-digit multiplication and dynamic programming, the authors show that a model fine-tuned to generate only the numeric intermediate values performs about as well as with the full textual steps, and that compressing those values into single latent vectors barely hurts accuracy. They also show that randomly overwriting an intermediate value changes the subsequent tokens and the final answer, exactly as a mutable program variable would. The paper concludes that CoT works because it forces the model to materialize and reuse intermediate values, but that this mechanism has limits: too much compression or too large a computational gap between values breaks the computation. If right, this means CoT can be compressed to just the essential value tokens, and that explicit text reasoning and latent-space 'thinking' may rely on the same underlying variable-storage mechanism.

What carries the argument

The experimental machinery is a controlled comparison across four forms of the same reasoning trace: full textual CoT, a variables-only CoT with all non-numeric tokens removed, a latent CoT where intermediate numbers are encoded as single one-hot vectors fed through a linear input projection with a matching latent output head, and an intervened CoT where one stored value is randomly replaced and the rest of the trace is regenerated. The first three forms isolate whether the value or the form carries the computation; the fourth tests causality. A final linear-probing setup reads the value of the next latent token from the previous token's hidden states at every layer, giving a per-layer measure of when the next variable is actually computed.

What would settle it

For the 5x5 dynamic-programming task with merged latent tokens, replace the linear probes with a nonlinear classifier and also measure final-answer accuracy directly; if the model's final accuracy stays high while linear-probe accuracy falls to near zero, the claimed complexity limit is a probe artifact rather than a real processing bound.

Watch

Extended reading notes

Core claim

The central discovery is that the tokens in a chain of thought store intermediate numerical values that are subsequently read by the model, and the linguistic form of those tokens is not what matters. On both tasks, deleting all non-numeric tokens from the CoT leaves accuracy essentially unchanged, and replacing a group of digit tokens with a single latent token that encodes the same number also preserves performance, as long as the number is small enough. Interventions, randomly changing a carry, a partial product, or a dynamic-programming cell, propagate to later tokens and to the final answer, matching variable-update semantics. The paper also finds that the model sometimes learns shortcuts, such as copying a multiplicand when multiplying by 1, that leave some variables unused; and that when two computations are merged into one step, linear probing cannot recover the value of the next variable, indicating a computational-complexity limit on how much a single token can be asked to compute.

Load-bearing premise

The claimed computational-complexity limit assumes that a linear probe on the previous token's hidden state accurately measures whether the model can compute the next intermediate value; if the value is stored nonlinearly, the probe could report failure even when the model computes it fine.

Editorial extensions

If this is right

  • CoT can be compressed to tokens that store intermediate values without significant accuracy loss, suggesting more efficient inference paths.
  • Because the storage form does not matter, latent-space reasoning and textual chain-of-thought may share the same variable-storage mechanism, connecting explicit and implicit CoT methods.
  • Intermediate values can be edited directly, so a model's output can be steered by rewriting a single variable in its chain of thought, which may enable controllable generation or error correction.
  • The observed computational-complexity limit implies a bound on how much work a single CoT token can encapsulate, which should inform token design and model architecture choices.
  • Shortcut behaviors, where the model ignores variables in easy subproblems, show that generated reasoning traces are not always faithful to the actual computation being performed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the variable view is right, the benefit of CoT on hard problems may come from forcing the model to commit intermediate results to memory, which could explain why longer CoT and even filler tokens help: they add more 'register' space for variable storage.
  • The one-hot latent representation is a minimal test of form independence; a richer continuous latent space might allow even more compression or might fail in different ways, depending on the model's capacity.
  • The computational-complexity limit may be scale-dependent: larger models with more layers or wider hidden states might tolerate more aggressive merging, turning the claimed limit into a capacity bound rather than a fundamental one.
  • Intervention success rates could serve as a diagnostic for which tokens a model actually relies on, complementing the paper's error taxonomy and offering a way to detect unfaithful reasoning.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes that chain-of-thought (CoT) tokens in LLMs function like computer program variables: they store intermediate values that are used in subsequent reasoning steps, and the explicit textual form of storage is not essential. The authors study two synthetic compositional tasks, multi-digit multiplication and dynamic programming, using Qwen-2.5-1.5B fine-tuned with CoT. They report four main findings: (1) CoT is necessary for these tasks; (2) removing non-result tokens from CoT preserves most accuracy; (3) replacing explicit intermediate results with latent one-hot tokens does not substantially hurt performance; and (4) interventions on explicit CoT token values causally affect subsequent tokens and final answers, though sometimes shortcuts prevent the expected effect. The paper also claims a computational complexity limit between CoT tokens, supported by linear probe accuracy on hidden states when adjacent latent tokens are merged. The authors conclude that CoT tokens are like program variables, with implications for CoT compression and understanding latent reasoning.

Significance. If the main claims hold, the paper offers a useful empirical characterization of CoT: intermediate-value storage, not surface form, is what matters, and CoT can be compressed to value-carrying tokens without loss. The intervention experiments on explicit CoT are a concrete step beyond correlational analyses, and the error taxonomy for failed interventions (shortcut, addition, reconstruction, copy, misc) is a useful contribution. The paper also releases code and data, and its claims are framed as falsifiable empirical hypotheses. The central limitation is that the latent-token results stop short of showing that the latent values are actually read and used by the model, which is the load-bearing step for the 'form does not matter' and 'variables' conclusions. The computational-complexity claim in Section 4.2 likewise rests on linear probe accuracy, which cannot by itself distinguish inability to compute from non-linear encoding. These gaps, together with the absence of error bars and the single-model/single-seed setup, currently prevent the strong conclusions from being fully supported.

major comments (4)
  1. [Section 3.3, Figures 4, 15, 16] The latent-token experiments show that replacing intermediate values with one-hot latent vectors preserves accuracy, but they never demonstrate that the model actually reads or uses the latent values. For DP, the CoT in Figure 16 contains only identical-looking <|fim_middle|> tokens; the model could plausibly treat them as pause/scratchpad tokens and recompute the DP recurrence from the input grid at each step, ignoring the latent embeddings entirely. The intervention experiments in Section 4.1 are performed only on explicit text CoT, and the linear probes in Section 4.2 show decodability, not causal use. Without an intervention on the latent embeddings (e.g., flipping a digit in the one-hot vector and observing a corresponding change in downstream computation), the claim that 'storing intermediate results in an alternative latent form will not affect model performance' is not established; the alternative explanation that the latent vectors are causally inert is not ruled out.
  2. [Section 4.2, Equation 5, Figure 8] The computational-complexity-limit conclusion relies entirely on linear probe accuracy on the previous token's hidden state. Low probe accuracy could reflect a non-linear encoding that the model still uses successfully, rather than an inability to compute the next intermediate variable. The paper does not report a non-linear probe baseline, a control on a setting where the value is known to be used, or any intervention that connects probe accuracy to downstream behavior. As stated, the claim that 'there exists a computation complexity limit, over which the LLM can no longer correctly calculate the next intermediate variable' is an overinterpretation of Figure 8; the figure would need to be accompanied by a causal or at least a controlled representational test.
  3. [Section 3.2 and Figure 2] The claim that 'preserving only tokens that store intermediate results would achieve comparable performance' is tested only on multiplication; no compressed-CoT experiment is reported for dynamic programming. Since the paper's abstract and conclusion generalize across both tasks, the missing DP condition leaves the generalization unsubstantiated. Additionally, the latent-form claim already shows a 9% accuracy drop on 4x5 DP (Figure 4b), which complicates the statement that 'storing intermediate results in an alternative latent form will not affect model performance' and should be discussed as a meaningful exception or as evidence of a limit, not just as motivation for Section 4.2.
  4. [Section 3 and all figures] No error bars, confidence intervals, or multiple seeds are reported anywhere in the paper. All experiments use a single fine-tuned model of one size (Qwen-2.5-1.5B), and the conclusions about 'comparable performance' and 'almost identical' accuracy depend on differences that could be within run-to-run variance. At minimum, the authors should report the number of seeds, the variance across seeds, and preferably statistical tests for the key comparisons in Figures 2, 4, and 6. Without this, the empirical support for the central claims is difficult to evaluate.
minor comments (6)
  1. [Section 1] The phrase 'under multinominal complexity' should be 'under polynomial complexity'.
  2. [Section 3.3, first paragraph] The phrase 'to compress consequent number digits' should be 'to compress consecutive number digits'.
  3. [Section 3.2, footnote 1] The footnote describing which tokens are removed is incomplete: it says 'see Appendix C for details,' but Appendix C (Main Experiment Settings) does not contain such details; the actual removal procedure appears in Appendix A. Please fix the cross-reference.
  4. [Section 4.2, Figure 9] The axes of Figure 9 are labeled 'Digit Scale' and 'Accuracy Breakdown,' but the caption and text refer to digit lengths and accuracy values; please clarify the exact quantities plotted and whether 'token accuracy' in this figure refers to the merged latent tokens.
  5. [Appendix E, Table 1] The intervention error breakdown reports 1291 shortcut errors out of 2616 errors, but the category definitions in Section 4.1 mention 'Copy error' and 'Misc error' with different percentages in Figure 6b; please ensure the figure and table are consistent.
  6. [Section 3.3, Equation (1)] The one-hot encoding formula uses 10n dimensions for an n-digit number, and the text says d=20 for multiplication (digit plus carry) and d=50 for DP. Please clarify whether this means DP values are restricted to at most 5 digits, and note that the DP input values are bounded below 100, so the DP latent tokens store substantially larger numbers than the inputs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are supported by empirical accuracy comparisons and causal interventions, not by definitions or fitted predictors.

full rationale

The paper's derivation chain is empirical: it compares model accuracy against golden final answers under full CoT, value-only CoT, latent-token CoT, and intervened CoT. The latent one-hot embedding in Eq. (1) is defined in terms of the intermediate value N, but the claim being tested is whether the model can still solve the task when values appear only in that form; this is not guaranteed by the definition, and indeed the model fails in some compressed DP settings. The intervention experiments in Section 4.1 report a 73.8% success rate with a detailed error taxonomy, so the causal conclusion is not forced by construction. The linear probing in Section 4.2 is a measurement instrument: the inference from probe accuracy to a 'computational complexity limit' is a validity concern (low linear probe accuracy could reflect nonlinear encoding), but the paper does not define task ability as probe accuracy, so this is not a circular reduction. The only self-citation (Zhu et al., 2025) supports a peripheral observation about layer-wise information processing, is redundant with an external citation (Stolfo et al., 2023), and is not a load-bearing premise. No uniqueness theorem imported from the authors, no ansatz smuggled in via citation, and no renaming of a known result as new are present. The paper also explicitly disclaims a theoretical proof in its Limitations section, which is consistent with an empirical, non-circular contribution.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The ledger is dominated by design choices (latent dimension, token-removal rules) and by the generalization assumption that synthetic algorithm-scripted CoT reflects real LLM CoT. The linear-probe measurement assumption is the most fragile unstated premise.

free parameters (2)
  • latent embedding dimension d = 20 (multiplication), 50 (DP)
    Chosen by hand based on maximum number of digits to represent; not fitted to data, but it bounds the capacity of the compressed representation and is a design choice that affects results.
  • set of tokens removed in compressed CoT = words like 'Calculate', 'digit', 'carry', 'Result of', 'Add up partial results:'
    Manual selection of which tokens count as non-result tokens; if the model still uses the structure of these words, the compression result could change. This is a design choice.
assumptions (3)
  • standard math The digit-wise multiplication and grid dynamic programming algorithms (Algorithm 1 and 2) correctly compute the task outputs.
    Section 2.2; the paper uses these algorithms to generate training data and to simulate expected outcomes after interventions.
  • domain assumption The model's fine-tuned behavior on synthetic algorithmically generated CoT traces is representative of how CoT functions generally in LLMs.
    All experiments use Qwen-2.5-1.5B fine-tuned on algorithmically generated traces; the Limitations section acknowledges that real-world datasets are not tested, making this a load-bearing generalization assumption.
  • ad hoc to paper Linear probe accuracy on the previous token's hidden state is a valid measure of the model's ability to compute the next intermediate variable.
    Section 4.2 uses linear probes to infer a computational complexity limit; this is an unproven measurement assumption because low probe accuracy can arise from non-linear encodings even when the model computes correctly.
invented entities (1)
  • Latent token <LAT> and its one-hot embedding
    purpose: Represents intermediate results (e.g., a partial product or DP state) as a single compressed vector instead of multiple text tokens.
    This is an experimental manipulation designed by the authors, not a posited mechanism in the model; it has no falsifiable handle outside the paper. Including it in the ledger marks that the conclusions about latent-form reasoning only apply to this constructed representation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chain-of-Thought Tokens are Computer Program Variables." pith.science (2026). https://pith.science/paper/ZC4FGFET

@misc{pith2026250504955,
  author       = {Pith},
  title        = {Pith review of: Chain-of-Thought Tokens are Computer Program Variables},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZC4FGFET}},
  note         = {Machine review of arXiv:2505.04955}
}
read the original abstract

Chain-of-thoughts (CoT) requires large language models (LLMs) to generate intermediate steps before reaching the final answer, and has been proven effective to help LLMs solve complex reasoning tasks. However, the inner mechanism of CoT still remains largely unclear. In this paper, we empirically study the role of CoT tokens in LLMs on two compositional tasks: multi-digit multiplication and dynamic programming. While CoT is essential for solving these problems, we find that preserving only tokens that store intermediate results would achieve comparable performance. Furthermore, we observe that storing intermediate results in an alternative latent form will not affect model performance. We also randomly intervene some values in CoT, and notice that subsequent CoT tokens and the final answer would change correspondingly. These findings suggest that CoT tokens may function like variables in computer programs but with potential drawbacks like unintended shortcuts and computational complexity limits between tokens. The code and data are available at https://github.com/solitaryzero/CoTs_are_Variables.

Figures

Figures reproduced from arXiv: 2505.04955 by the authors.

Figure 1
Figure 1. Comparison on model accuracy between plain prompting and chain-of-thought prompting. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Model performance when non-result tokens [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The model structure used to reason with latent tokens. We use one-hot vectors as the latent embedding of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Model performances when merging interme [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Examples of a successful intervention (left) and an intervention with a shortcut error (right). [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: (a) Success rate of intervention. When the intervened output is the same as simulated, we view it as a successful intervention. (b) Error breakdown. Shortcut error occupies a large percentage of the errors. DP[i][j] DP[i][j+1] DP[i][j+2] DP[i+1][j] DP[i+1][j+1] DP[i+1]…
Figure 7
Figure 7. Figure 7: Demonstration of the alternative merging strat [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 9
Figure 9. Figure 9: Accuracy breakdown by the scale of target [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Example prompt for the multi-digit multipli [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Example CoT for the multi-digit multiplication task. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Example CoT after compression for the multi-digit multiplication task. [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Example Prompt for the dynamic programming task. [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: Example CoT for the dynamic programming task. [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Example latent CoT for the multi-digit multiplication task. [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: Example latent CoT for the dynamic programming task. [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Internalized Reasoning for Long-Context Visual Document Understanding

    cs.CV 2026-03 unverdicted novelty 7.0 of 10

    Synthetic page-ranked reasoning traces plus low-strength model merging give a 32B VLM 58.3 on MMLongBenchDoc, beating a 235B teacher while cutting output tokens ~12× versus explicit reasoning.

  2. Optimizing Token Consumption in LLMs: A Nano Surge Approach for Code Reasoning Efficiency

    cs.SE 2025-04 conditional novelty 3.0 of 10

    Refactoring smelly Java code and adding context, role, or token-limit prompts cut LLM chain-of-thought token use by roughly 15-50% in this study, but the 'no quality loss' claim rests only on shallow similarity metrics.

Reference graph

Works this paper leans on

32 extracted references · 13 canonical work pages · cited by 2 Pith papers

  1. [1]

    Lijie Chen, Binghui Peng, and Hongxun Wu. 2024. Theoretical limitations of multi-layer transformer. arXiv preprint arXiv:2412.02975

    arXiv 2024

  2. [2]

    Yuntian Deng, Yejin Choi, and Stuart Shieber. 2024. From explicit cot to implicit cot: Learning to internalize cot step by step. arXiv preprint arXiv:2405.14838

    arXiv 2024

  3. [3]

    Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, and 1 others. 2023. Faith and fate: Limits of transformers on compositionality. Advances in Neural Information Processing Systems, 36:70293--70332

    work page 2023

  4. [4]

    Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. 2023. Towards revealing the mystery behind chain of thought: a theoretical perspective. Advances in Neural Information Processing Systems, 36:70757--70798

    2023

  5. [5]

    Angeliki Giannou, Shashank Rajput, Jy-yong Sohn, Kangwook Lee, Jason D Lee, and Dimitris Papailiopoulos. 2023. Looped transformers as programmable computers. In International Conference on Machine Learning, pages 11398--11442. PMLR

    work page 2023

  6. [6]

    Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. 2024. Think before you speak: Training language models with pause tokens. In The Twelfth International Conference on Learning Representations

    work page 2024

  7. [7]

    Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. rstar-math: Small llms can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501.04519

    arXiv 2025

  8. [8]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    arXiv 2025

Show all 32 references
  1. [9]

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769

    2024 arXiv

  2. [10]

    Alexander Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, Eric Hambro, Sainbayar Sukhbaatar, and Roberta Raileanu. 2024. Teaching large language models to reason with reinforcement learning. In AI for Math Workshop@ ICML 2024

    2024

  3. [11]

    Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2022. Decomposed prompting: A modular approach for solving complex tasks. In The Eleventh International Conference on Learning Representations

    2022

  4. [12]

    Zhiyuan Li, Hong Liu, Denny Zhou, and Tengyu Ma. 2024. Chain of thought empowers transformers to solve inherently serial problems. In The Twelfth International Conference on Learning Representations

    2024

  5. [13]

    Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, A...

    2025

  6. [14]

    Bingbin Liu, Jordan T Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang. 2022. Transformers learn shortcuts to automata. In The Eleventh International Conference on Learning Representations

    2022

  7. [15]

    Aman Madaan and Amir Yazdanbakhsh. 2022. Text and patterns: For effective chain of thought, it takes two to tango. arXiv preprint arXiv:2209.07686

    2022 arXiv

  8. [16]

    Jorge P \'e rez, Pablo Barcel \'o , and Javier Marinkovic. 2021. Attention is turing-complete. Journal of Machine Learning Research, 22(75):1--35

    2021

  9. [17]

    Jacob Pfau, William Merrill, and Samuel R Bowman. 2024. Let’s think dot by dot: Hidden computation in transformer language models. In First Conference on Language Modeling

    2024

  10. [18]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  11. [19]

    Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. 2023. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7035--7052

    2023

  12. [20]

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistic...

    2024

  13. [21]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  14. [22]

    Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao, Min-Yen Kan, Junxian He, and Michael Xie. 2023. Self-evaluation guided beam search for reasoning. Advances in Neural Information Processing Systems, 36:41618--41650

    2023

  15. [23]

    Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600

    2025 arXiv

  16. [24]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  17. [25]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809--11822

    2023

  18. [26]

    Longhui Yu, Weisen Jiang, Han Shi, YU Jincheng, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. Metamath: Bootstrap your own mathematical questions for large language models. In The Twelfth International Conference on Learning Representations

    2023

  19. [27]

    Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2023. Mammoth: Building math generalist models through hybrid instruction tuning. In The Twelfth International Conference on Learning Representations

    2023

  20. [28]

    Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, and 1 others. 2024. Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning. arXiv preprint arXiv:2410.02884

    2024 arXiv

  21. [29]

    Denny Zhou, Nathanael Sch \"a rli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, and 1 others. 2022. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference ...

    2022

  22. [30]

    Fangwei Zhu, Damai Dai, and Zhifang Sui. 2025. Language models encode the value of numbers linearly. In Proceedings of the 31st International Conference on Computational Linguistics, pages 693--709

    2025

  23. [31]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  24. [32]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.