Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Chain of Methodologies: Scaling Test Time Computation without Training

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A prompt-only loop that selects a methodology before each reasoning step lifts AIME accuracy by 38.5% and Hotpot F1 by 28.7% over chain-of-thought, without fine-tuning.

desk verdict A novel training-free prompting framework, but the evaluation confounds methodology selection with code execution; the central claim needs a code-controlled baseline. read the letter →

arxiv 2506.06982 v1 pith:OZSDQBSQ submitted 2025-06-08 cs.CL

Cong Liu , Jie Wu , Weigang Wu , Xu Chen , Liang Lin , Wei-Shi Zheng This is my paper
classification cs.CL
keywords ChainofMethodologiestraining-freepromptingmetacognitiontest-timecomputationchain-of-thoughtmathematicalreasoningretrieval-augmentedgenerationmethodologyselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a frozen LLM can run long, structured reasoning on hard problems if the reasoning is organized as a loop: at each step the model first selects a methodology from a user-written list, then produces the next reasoning step under that methodology. CoM, as the framework is called, needs no fine-tuning and uses the same methodology list for mathematics and retrieval-heavy question answering. On AIME math problems and the hard split of HotpotQA, the paper reports accuracy and F1 gains of 38.5% and 28.7% over plain chain-of-thought prompting with a 72B model, and smaller but consistent gains with a 7B model. The paper reads these results as evidence that advanced LLMs already possess metacognitive abilities, the ability to reason about their own reasoning, that prompts can activate.

What carries the argument

The methodology-reasoning loop is the load-bearing mechanism. Each methodology is a Markdown block with three parts: a name, a when field describing the reasoning stage at which it applies, and a what field specifying the approach, criteria, and expected output. The loop alternates two prompts: one that asks the model to choose the next methodology from the list based on the history, and one that asks it to reason under that methodology. A Python interpreter completes the mechanism by executing generated code, which the paper shows is critical: removing it drops AIME accuracy by 44.5% and Hard Hotpot F1 by 40.2%.

What would settle it

Run CoM with the methodology-selection step replaced by a random draw from the same methodology list, holding the reasoning prompt and Python interpreter fixed. If AIME accuracy stays at the reported level, dynamic selection is not load-bearing; if it falls, the model's choice of methodology is what the gain depends on.

Watch

Extended reading notes

Core claim

CoM is a training-free prompting framework that interleaves methodology selection with methodology-guided reasoning. The user provides a list of methodologies, each written in a when-what format: when the methodology applies and what systematic steps it prescribes. At every iteration the model reads the question and the reasoning history, selects one methodology, and then generates the next reasoning step or a Python code block that follows it; a sandboxed interpreter executes any code and substitutes the real output for the model's guess. The paper's central claim is that this justification-before-action loop activates latent metacognitive knowledge in the frozen LLM, producing better zero-shot accuracy on complex math and multi-hop retrieval than single-turn prompting, and outperforming even a fixed, task-specific sequence of the same methodologies on the hardest tasks.

Load-bearing premise

The whole gain depends on the frozen model being able to pick a genuinely useful methodology from the list at each step; if it just repeats the first choice, as the paper observes for some models, the method stops helping.

Editorial extensions

If this is right

  • Users can steer or specialize CoM by editing the methodology list in plain text, with no retraining or prompt-search step.
  • Dynamic methodology selection beats a fixed per-task sequence on the hardest tasks, so the value of the loop grows with task complexity or model capability.
  • The Python interpreter is a major source of the gain, meaning tool use and computation offloading are part of what makes the prompt loop work.
  • Models that cannot reliably select methodologies do not benefit from CoM, so the framework's applicability depends on the model's metacognitive ability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: one can test whether automatically generating the methodology list from the model's own successful trajectories, rather than hand-writing it, compounds the gains and removes the user effort.
  • Editorial inference: because the interpreter accounts for most of the improvement, a controlled comparison that removes only the justification text, keeping the same number of turns, prompts, and tool use, would isolate whether the methodology framing itself or just the extra structured tokens drives the result.
  • Editorial inference: the paper notes self-consistency is orthogonal, so combining CoM with multiple sampled runs and a majority vote is a natural untested extension that could push accuracy further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Chain of Methodologies (CoM), a training-free prompting framework that alternates between selecting a methodology from a user-defined list and generating a reasoning step guided by that methodology, with an optional Python interpreter and search tool. The method is evaluated on mathematical reasoning (AIME, GSM8K, MATH-500, ARC) and multi-hop QA (Hard HotpotQA) across several LLMs, reporting accuracy/F1 improvements over Chain-of-Thought (CoT) and a single-turn methodology-augmented CoT (MCoT). The authors attribute the gains to the activation of latent metacognitive abilities in LLMs via dynamic methodology selection and present methodology-sequence statistics, ablations, and an efficiency analysis.

Significance. If the stated improvements were attributable to the proposed methodology-selection mechanism, the paper would make a useful contribution: it demonstrates a training-free way to elicit structured reasoning with human-defined methodologies, and it provides a clear, extensible prompting recipe. The paper also includes useful ablations and reports results on multiple models and tasks. However, the current experimental design does not isolate the proposed mechanism from the confounding effects of tool access (Python interpreter and search) and from test-set information used to construct the Workflow baseline. As presented, the evidence supports the conclusion that tools help, not that metacognitive selection is the driving factor. The claims about latent metacognitive abilities are therefore not yet substantiated.

major comments (3)
  1. [§4.2, §2.3, Table 5] The headline comparison is confounded by unequal tool access. Section 4.2 states that CoT and MCoT 'do not allow code generation', while CoM includes a Python interpreter (Section 2.3) and, on HotpotQA, a search function with fuzzy matching. The ablation in Table 5 shows that removing the interpreter drops CoM's AIME accuracy from 25.4 to 14.1, below the CoT baseline of 20.15 for the same model, and removing retrieval on Hard Hotpot drops F1 from 0.4174 to 0.22. These results indicate that the reported gains over CoT on AIME and Hard Hotpot are at least largely attributable to code execution and retrieval tools, not to the methodology-selection mechanism that the paper claims to validate. To support the central claim, the authors need to compare CoM against baselines that are given the same tools (e.g., CoT with a Python interpreter, MCoT with a Python interpreter, or a static-workflow baseline with the same code-execution capability).
  2. [§4.2, Table 4] The Workflow baseline is constructed from the most frequent methodology sequences chosen by CoM on the test set (Table 4), and the paper then claims CoM outperforms Workflow. This is circular: the baseline is informed by the test-set behavior of the very method it is meant to be compared against, so the comparison cannot establish that dynamic selection is superior to a fixed sequence. The Workflow sequences should be derived from a validation split or from CoM's selections on training questions, not from the test set. Without this fix, the contrast between CoM and Workflow does not provide independent evidence for the benefit of dynamic methodology selection.
  3. [§4.3, Limitations] The interpretation that the results 'demonstrate the effectiveness of metacognitive abilities in LLMs' is not supported because the experiments do not isolate the selection mechanism. The paper explicitly acknowledges in the Limitations that it assumes the LLM possesses metacognitive abilities, and Appendix A.2 shows that some models (e.g., Yi-1.5-9B, InternLM2.5-7B) fail at selection. However, the positive evidence for the assumption is confounded by the tool-access disparity described above. The observed gains could arise simply from the ability to write and execute correct programs, or from the multi-turn structure, without any metacognitive planning. The authors should add a control that removes the selection component (e.g., random methodology selection, or a fixed sequence with the same tools) while keeping the reasoning loop and tool access otherwise identical, to demonstrate that the dynamic selection itself contributes to the reported improvements.
minor comments (5)
  1. [Abstract] There is a typo: 'reasoning throught' should be 'reasoning through'.
  2. [§4.7] The text says 'Com strikes a good balance' but should read 'CoM strikes a good balance'.
  3. [§A.2] The placeholder 'Table ??' appears in the discussion of Self-Consistency (CoT-SC) results. The actual table is numbered Table 8, but the in-text reference is not resolved.
  4. [Table 2] HotpotQA is cited as '(GLM, 2024)' in Table 2, but the original source is Yang et al. (2018). Please correct the citation.
  5. [Figure 7] The prompt text contains several typos: 'acurate' and 'outpoutfrom' should be 'accurate' and 'output from'.

Circularity Check

1 steps flagged · score 4.0 of 10

One self-referential evaluation element: the Workflow baseline is constructed from CoM's own most frequent methodology sequences on the test set, and CoM's superiority over Workflow is then presented as validating dynamic methodology selection. The headline CoM-vs-CoT comparison is independent, though confounded by unequal tool access.

  1. fitted input called prediction [Section 4.2 (Baselines) and Section 4.3 (Performance Comparison)]
    "Workflow is similar to CoM but uses a fixed methodology sequence per task, derived from the most frequent sequences chosen by CoM (Table 4). ... Compared with the task-specifically optimized Workflow, CoM’s accuracy is 1.7% higher on AIME, and 9.8% higher on Hard Hotpot, demonstrating CoM’s superior flexibility in methodology selection. This highlights the effectiveness of metacognitive abilities in LLMs for choosing appropriate methodology sequences and validates our step-by-step reasoning approach."

    Workflow is not an independent baseline: its per-task sequences are the most frequent methodology sequences that CoM itself selected on the test set (Table 4 lists the top sequences chosen by CoM on AIME). CoM is therefore compared against a static summary of its own behavior. Claiming that outperforming Workflow 'demonstrates CoM’s superior flexibility' and 'validates the effectiveness of metacognitive abilities' is self-referential: the comparison only shows that dynamic selection can beat a compressed version of the same selection process, not that methodology selection per se causes the gain. This does not affect the CoT comparison, which is independent, but it does undermine the paper's specific claim that CoM's flexibility is validated by the Workflow comparison.

full rationale

The paper contains no fitted parameters and no load-bearing self-citation chain: citations to prior metacognition work (Wang and Zhao 2024; Didolkar et al. 2024) are motivational background, not proof of the method. The CoM-vs-CoT headline comparison is not circular by construction. However, the evaluation has one self-referential element: the Workflow baseline is derived from CoM's own most frequent selected methodology sequences on the same test set (Section 4.2, Table 4), and CoM's superiority over Workflow is used to claim validation of dynamic methodology selection. That particular claim reduces to comparing CoM with itself. Separately, the CoM-vs-CoT comparison is confounded: CoM has a Python interpreter and search tool while CoT and MCoT are forbidden from generating code, and Table 5 shows CoM without the interpreter falls below CoT on AIME; this is a correctness/attribution risk, not a circularity. Overall partial circularity in one supporting comparison, with independent content remaining in the main comparison.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen prompt components and on the empirical assumption that LLMs can select methodologies reliably. The method itself has no learned parameters, but the design choices and the metacognitive capability assumption are load-bearing.

free parameters (5)
  • Methodology list (7 definitions)
    The when-what methodology definitions are hand-designed and encode task-specific choices (e.g., writing code in a single block). Not learned, but a key component of the framework.
  • Maximum reasoning iterations K = 8
    Set by hand as the step cap; the reported average number of prompts (Table 7) is below 2*K.
  • Retrieval top-k = 3
    The search function retrieves top-k=3 facts via fuzzy matching; this value affects HotpotQA results.
  • Sampling parameters (temperature, top_k, top_p) = 0.2, 40, 0.95
    LLM API settings used in all experiments; affect output variability.
  • Python interpreter timeout = 1 minute
    Chosen empirically; described in Section 2.3.
assumptions (4)
  • domain assumption The frozen LLM has sufficient metacognitive ability to select appropriate methodologies from the list based on the reasoning history.
    Stated in the Limitations: 'The approach proposed in this paper assumes that the LLM possesses metacognitive abilities.' If false, methodology selection becomes arbitrary and CoM's advantage disappears (Appendix A.2).
  • domain assumption Fuzzy string matching over gold supporting facts approximates real retrieval-augmented generation.
    Section 4.1 simulates RAG by retrieving top-k similar supporting facts with fuzzy matching when the LLM calls search. Real retrievers have different error distributions, so HotpotQA results may not generalize.
  • ad hoc to paper Instructing the LLM to 'accurately simulate' the output of code or search before the interpreter replaces it is a valid prompting technique.
    The methodology prompts (Figures 3 and 8) tell the LLM to guess the output, then the Python interpreter replaces the guessed output with actual stdout. This two-stage interaction is not independently validated.
  • ad hoc to paper Benchmark leakage is the reason CoT outperforms CoM on GSM8K.
    Section 4.3 attributes the GSM8K result to leakage (Xu et al., 2024b) without testing this explanation. This assumption preserves the claim that CoM is competitive on GSM8K.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chain of Methodologies: Scaling Test Time Computation without Training." pith.science (2026). https://pith.science/paper/OZSDQBSQ

@misc{pith2026250606982,
  author       = {Pith},
  title        = {Pith review of: Chain of Methodologies: Scaling Test Time Computation without Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OZSDQBSQ}},
  note         = {Machine review of arXiv:2506.06982}
}
read the original abstract

Large Language Models (LLMs) often struggle with complex reasoning tasks due to insufficient in-depth insights in their training data, which are typically absent in publicly available documents. This paper introduces the Chain of Methodologies (CoM), an innovative and intuitive prompting framework that enhances structured thinking by integrating human methodological insights, enabling LLMs to tackle complex tasks with extended reasoning. CoM leverages the metacognitive abilities of advanced LLMs, activating systematic reasoning throught user-defined methodologies without explicit fine-tuning. Experiments show that CoM surpasses competitive baselines, demonstrating the potential of training-free prompting methods as robust solutions for complex reasoning tasks and bridging the gap toward human-level reasoning through human-like methodological insights.

Figures

Figures reproduced from arXiv: 2506.06982 by the authors.

Figure 1
Figure 1. An Example of our Chain of Methodologies [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The components in our CoM framework and their interactions in the methodology-reasoning itera￾tions. guiding thought processes. We find prompts re￾lated to methodology are ideal candidates for this purpose. Methodology is a critical component of any discipline or field that requires a structured approach to understanding, problem-solving, or conducting research. It provides a framework that ensures tasks are execute… view at source ↗
Figure 3
Figure 3. Two example methodology definitions in our [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Results of Qwen2-72B-Instruct on AIME, GSM8K, and Hard HotpotQA. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Results of DeepSeek-V3 on AIME, GSM8K, and Hard HotpotQA. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Results of Qwen2.5-7B-Instruct, a small LLM, on AIME, GSM8K, MATH, ARC, and Hard HotpotQA. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Our methodology-selection prompt (left) and methodology-based reasoning prompt (right). [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Our list of methodologies [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Results of Yi-1.5-9B-Chat (01.AI: Alex Young, 2024) on AIME, GSM8K, and Hard HotpotQA. pass@1 Acc 5.0 7.5 2.89 8.25 5.47 2.89 AIME pass@1 Acc 50 75 60.96 71.95 85.67 49.89 GSM8K EM F1 Precison Recall 0.0 0.2 0.02 0.07 0.05 0.20 0.02 0.11 0.09 0.18 0.00 0.07 0.05 0.26 0…
Figure 10
Figure 10. Figure 10: Results of InternLM2.5-7B-chat (Cai et al., 2024) on AIME, GSM8K, and Hard HotpotQA. pass@1 Acc 2.5 5.0 7.5 2.68 7.29 2.47 1.82 AIME pass@1 Acc 60 80 59.59 72.78 80.89 77.41 GSM8K EM F1 Precison Recall 0.0 0.2 0.13 0.20 0.33 0.21 0.13 0.20 0.26 0.20 0.00 0.02 0.27 0.0…
Figure 11
Figure 11. Figure 11: Results of GLM-4-9b-chat (GLM, 2024) on AIME, GSM8K, and Hard HotpotQA [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 2 canonical work pages

  1. [1]

    https://cloud.baidu.com/ https://cloud.baidu.com/

    work page

  2. [2]

    https://siliconflow.cn/ https://siliconflow.cn/

  3. [3]

    Chao Li Chengen Huang Ge Zhang Guanwei Zhang Guoyin Wang Heng Li Jiangcheng Zhu Jianqun Chen Jing Chang Kaidong Yu Peng Liu Qiang Liu Shawn Yue Senbin Yang Shiming Yang Wen Xie Wenhao Huang Xiaohui Hu Xiaoyi Ren Xinyao Niu Pengcheng Nie Yanpeng Li Yuchi Xu Yudong Liu Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai 01.AI: Alex Young, Bei Chen. 2024....

    arXiv 2024

  4. [4]

    Anonymous. 2024 a . https://openreview.net/forum?id=z5uVAKwmjf AF low: Automating agentic workflow generation . In The Thirteenth International Conference on Learning Representations (ICLR)

    2024

  5. [5]

    Anonymous. 2024 b . Chain of ideas: Revolutionizing research in idea development with LLM agents. In The Thirteenth International Conference on Learning Representations (ICLR)

    2024

  6. [6]

    Anonymous. 2025. https://openreview.net/forum?id=FSjIrOm1vz Inference scaling for long-context retrieval augmented generation . In The Thirteenth International Conference on Learning Representations (ICLR)

    2025

  7. [7]

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, S...

    arXiv 2024

  8. [8]

    Shulin Cao, Jiajie Zhang, Jiaxin Shi, Xin Lv, Zijun Yao, Qi Tian, Lei Hou, and Juanzi Li. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.835 Probabilistic tree-of-thought reasoning for answering knowledge-intensive complex questions . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12541--12560, Singapore. Associatio...

    doi:10.18653/v1/2023.findings-emnlp.835 2023

Show all 45 references
  1. [9]

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. 2023. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research

    2023

  2. [10]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1

    2018 arXiv

  3. [11]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    2021 arXiv

  4. [12]

    DeepSeek-AI. 2024. Deepseek-v3 technical report. https://arxiv.org/pdf/2412.19437

    2024 arXiv

  5. [13]

    Aniket Rajiv Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy P Lillicrap, Danilo Jimenez Rezende, Yoshua Bengio, Michael Curtis Mozer, and Sanjeev Arora. 2024. https://openreview.net/forum?id=0MsI3bSmmD Metacognitive capabilities of LLM s: An explor...

    2024

  6. [14]

    Marah Abdin et al. 2024. https://arxiv.org/abs/2404.14219 Phi-3 technical report: A highly capable language model locally on your phone . Preprint, arXiv:2404.14219

    2024 arXiv

  7. [15]

    Team GLM. 2024. https://arxiv.org/abs/2406.12793 Chatglm: A family of large language models from glm-130b to glm-4 all tools . Preprint, arXiv:2406.12793

    2024 arXiv

  8. [16]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and J \"u rgen Schmidhuber. 2024. https://openreview.net/forum?id=VtmBAGCN7o Meta GP...

    2024

  9. [17]

    Mingyu Jin, Weidi Luo, Sitao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, and Yongfeng Zhang. 2024. https://arxiv.org/abs/2411.13504 Disentangling memory and reasoning ability in large language models . Preprint, arXiv:2411.13504

    2024 arXiv

  10. [18]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. ICLR 2024

    2023

  11. [19]

    Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023. https://doi.org/10.18653/v1/2023.ijcnlp-main.20 Faithful chain-of-thought reasoning . In Proceedings of the 13th International Joint Conference on Natura...

    2023 doi

  12. [20]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. https://arxiv.o...

    2023 arXiv

  13. [21]

    OpenAI. 2024. https://openai.com/index/learning-to-reason-with-llms/ Learning to Reason with LLMs

    2024

  14. [22]

    Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang. 2024. https://arxiv.org/abs/2408.06195 Mutual reasoning makes smaller llms stronger problem-solvers . In Arxiv

    2024 arXiv

  15. [23]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. https://arxiv.org/abs/2408.03314 Scaling llm test-time compute optimally can be more effective than scaling model parameters . Preprint, arXiv:2408.03314

    2024 arXiv

  16. [24]

    O. Team. 2024. https://github.com/Open-Source-O1/Open-O1 https://github.com/open-source-o1/open-o1

    2024

  17. [25]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. https://doi.org/10.18653/v1/2023.acl-long.557 Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions . In Proceedings of the 61st Annual Meeting of th...

    2023 doi

  18. [26]

    Boshi Wang, Xiang Deng, and Huan Sun. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.174 Iteratively prompt pre-trained language models for chain of thought . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2714--2730, Abu Dh...

    2022 doi

  19. [27]

    Jianing Wang, Qiushi Sun, Xiang Li, and Ming Gao. 2024. https://doi.org/10.18653/v1/2024.acl-long.271 Boosting language models reasoning with chain-of-knowledge prompting . In The 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag...

    2024 doi

  20. [28]

    Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023 a . https://doi.org/10.18653/v1/2023.acl-long.147 Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models . In Proceedings of the 61st Annua...

    2023 doi

  21. [29]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 b . https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Confer...

    2023

  22. [30]

    Yuqing Wang and Yun Zhao. 2024. https://doi.org/10.18653/v1/2024.naacl-long.106 Metacognitive prompting improves understanding in large language models . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human...

    2024 doi

  23. [31]

    Chi, Quoc V Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. https://openreview.net/forum?id=_VjQlMeSB_J Chain of thought prompting elicits reasoning in large language models . In Advances in Neural Information Proc...

    2022

  24. [32]

    Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.720 Faithful logical reasoning via symbolic chain-of-thought . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistic...

    2024 doi

  25. [33]

    Ruijie Xu, Zengzhi Wang, Run-Ze Fan, and Pengfei Liu. 2024 b . https://arxiv.org/abs/2404.18824 Benchmarking benchmark leakage in large language models . arXiv preprint arXiv:2404.18824

    2024 arXiv

  26. [34]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng...

    2024 arXiv

  27. [35]

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2024 b . https://openreview.net/forum?id=Bb4VGOWELI Large language models as optimizers . In The Twelfth International Conference on Learning Representations

    2024

  28. [36]

    Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E Gonzalez, and Bin Cui. 2024 c . Buffer of thoughts: Thought-augmented reasoning with large language models. arXiv preprint arXiv:2406.04271

    2024 arXiv

  29. [37]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA : A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing ( EMNLP )

    2018

  30. [38]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/639a9a172c044fbb64175b5fad42e9a5-Paper-Conference.pdf Star: Bootstrapping reasoning with reasoning . In Advances in Neural Information Processing Systems, vol...

    2022

  31. [39]

    Dan Zhang, Sining Zhoubian, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. Rest-mcts*: Llm self-training via process reward guided tree search. Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS)

    2024

  32. [40]

    Xingyuan Zhang, Philip Becker-Ehmck, Patrick van der Smagt, and Maximilian Karl. 2023 a . https://openreview.net/forum?id=WjlCQxpuxU Action inference by maximising evidence: Zero-shot imitation from observation with world models . In Thirty-seventh Conference on Neural Informa...

    2023

  33. [41]

    Zhebin Zhang, Xinyu Zhang, Yuanhang Ren, Saijiang Shi, Meng Han, Yongkang Wu, Ruofei Lai, and Zhao Cao. 2023 b . https://doi.org/10.18653/v1/2023.emnlp-main.1 IAG : Induction-augmented generation framework for answering reasoning questions . In Proceedings of the 2023 Conferen...

    2023 doi

  34. [42]

    Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. 2024. https://arxiv.org/abs/2411.14405 Marco-o1: Towards open reasoning models for open-ended solutions . Preprint, arXiv:2411.14405

    2024 arXiv

  35. [43]

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023. https://arxiv.org/abs/2205.10625 Least-to-most prompting enables complex reasoning in large language models . In The Eleventh...

    2023 arXiv

  36. [44]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.