Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Knowledge or Reasoning? A Close Look at How LLMs Think Across Domains

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM's step-by-step reasoning can be split into knowledge correctness and reasoning informativeness, and the two respond differently to training.

desk verdict Useful two-axis evaluation lens, but the reasoning metric measures answer-token surprisal, not validated reasoning quality; the SFT/RL claims are plausible but not yet established. read the letter →

arxiv 2506.02126 v1 pith:FFPE7BTQ submitted 2025-06-02 cs.CL

Juncheng Wu , Sheng Liu , Haoqin Tu , Hang Yu , Xiaoke Huang , James Zou , Cihang Xie , Yuyin Zhou This is my paper
classification cs.CL
keywords LLMreasoningevaluationKnowledgeIndexInformationGainstep-wiseperplexitysupervisedfine-tuningreinforcementlearningmedicalmathematical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that final-answer accuracy hides how an LLM actually thinks, and that the thinking trace itself can be scored along two separate axes. It introduces Knowledge Index (KI), the fraction of reasoning steps whose factual content agrees with an external medical knowledge source, and InfoGain, the average reduction in a judge model's perplexity toward the correct answer as each reasoning step is read. Applied to Qwen2.5-7B base and DeepSeek-R1-distilled models trained with supervised fine-tuning (SFT) and reinforcement learning (RL) on medicine and math, the framework shows that SFT raises accuracy and factual knowledge but cuts InfoGain by 38.9% on average, while RL prunes inaccurate or irrelevant knowledge from reasoning paths. The authors conclude that medical tasks lean more on knowledge and math tasks lean more on reasoning, so the two training regimes should be chosen by domain.

What carries the argument

The machinery is a two-metric decomposition of a model's thinking trace. A response is split into successive steps $s_1, \dots, s_t$ by GPT-4o. InfoGain is then computed as $\Delta I = \frac{1}{t}\sum_{i=1}^t(\mathrm{PPL}_{i-1} - \mathrm{PPL}_i)$, where $\mathrm{PPL}_i$ is the perplexity that an untrained Qwen2.5-7B judge assigns to the correct answer $A^*$ given the question and the first $i$ steps; a positive gap means the step concentrates probability mass on the correct answer. Knowledge Index is the average of Boolean judgments $\mathrm{consistency}_i$: for each step GPT-4o extracts its knowledge claim, the claim is looked up in a medical textbook database, and GPT-4o decides whether the step matches the retrieved fact. These two scalar metrics are what let the authors compare SFT and RL training regimes step by step, and they are the basis for the claim that the two capacities are distinct.

What would settle it

Take a set of medical reasoning traces that are logically equivalent but differ in wording, and compute InfoGain for each; if the score moves substantially across paraphrases that preserve the logical content, the metric is tracking style, not reasoning informativeness, and the paper's SFT-versus-RL conclusions lose their support.

Watch

Extended reading notes

Core claim

The paper's central discovery is that reasoning traces can be decomposed into two capacities that respond differently to training: knowledge correctness, measured as Knowledge Index, and reasoning informativeness, measured as InfoGain. In medical fine-tuning the base Qwen2.5-7B outperforms the R1-distilled variant by 14.7 points with SFT alone and by 22.6 points after RL, showing that general reasoning skills distilled from R1 do not transfer to medicine. SFT increases accuracy and knowledge (average KI up 6.2 points on medical tasks) but lowers InfoGain by 38.9% on average, producing verbose or suboptimal reasoning paths; RL restores reasoning quality by steering the model away from steps containing wrong knowledge, raising KI by 12.4 points on average when applied alone. Across five medical benchmarks KI correlates with accuracy at 0.998 on average, about 0.3 higher than InfoGain's correlation, while in math RL yields the largest accuracy gains, supporting the conclusion that knowledge and reasoning are distinct, domain-dependent abilities.

Load-bearing premise

The load-bearing assumption is that the drop in a small untrained language model's confusion about the correct answer from one reasoning step to the next really measures how informative that step is, rather than how the answer is phrased or how familiar the model already is with the topic.

Editorial extensions

If this is right

  • SFT should not be assumed to improve reasoning quality: on these models it raises accuracy and knowledge while cutting InfoGain by 38.9% on average.
  • RL is the better lever for reasoning efficiency: it consistently raises InfoGain in both medicine and math, and in medicine it raises Knowledge Index by 12.4 points on average by pruning wrong-knowledge paths.
  • Accuracy gains in medicine come mostly from knowledge: KI correlates with accuracy at 0.998 average across the five medical benchmarks, about 0.3 higher than InfoGain's correlation.
  • General reasoning distilled from R1 does not transfer to medicine: the Qwen base model beats the R1-distilled variant by 14.7 points with SFT and by 22.6 points after RL in average medical accuracy.
  • Training strategy should differ by domain: SFT matters more for knowledge-intensive medicine, while RL matters more for reasoning-intensive math, where RL-ed accuracy (61.7% average) exceeds SFT-ed accuracy (51.9%).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If InfoGain is a valid measure, it could be used as a training signal: rewarding steps that reduce a judge model's perplexity toward the correct answer should yield more concise, informative reasoning chains than rewarding only final accuracy.
  • The same decomposition could be applied to other structured domains such as legal IRAC reasoning or financial forecasting, where step-level knowledge and inference can be separated; a concrete test would be whether KI and InfoGain correlate with human expert ratings in those domains.
  • A natural stress test is to replace the untrained Qwen2.5-7B judge with models of different size and domain familiarity; if the ranking of training regimes reverses under that substitution, the reported effects are judge-dependent rather than properties of the reasoning itself.
  • The RL 'pruning' behavior suggests RL improves knowledge correctness not by adding facts but by suppressing unreliable paths, which could be verified by tracking which knowledge points disappear from reasoning traces after RL.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes to decompose LLM reasoning traces into two components: knowledge correctness, measured by a Knowledge Index (KI), and reasoning informativeness, measured by an Information Gain (InfoGain) metric defined as the average reduction in perplexity of the correct-answer tokens over successive reasoning steps. The framework is applied to Qwen2.5-7B-base and DeepSeek-R1-Distill-Qwen-7B models after supervised fine-tuning (SFT) and reinforcement learning (RL) on medical and mathematical benchmarks. The main reported findings are that (1) R1-distilled models do not transfer their reasoning ability to medicine, (2) SFT improves accuracy but lowers InfoGain by 38.9% (also stated as 37.1%), and (3) RL improves medical reasoning and knowledge by pruning incorrect or irrelevant knowledge from reasoning paths. The authors conclude that medical tasks are more knowledge-intensive while math tasks are more reasoning-intensive, and that SFT and RL target different capacities.

Significance. If the proposed decomposition is valid, the paper offers a useful lens for understanding post-training effects beyond final-answer accuracy, and the two-domain comparison (medical vs. math) is well motivated. The authors release code, project pages, and detailed prompts in Appendix B.2, which is a concrete reproducibility strength. The paper also includes case studies and a correlation analysis linking the proposed metrics to accuracy. However, the significance is conditional on establishing that the InfoGain metric actually measures reasoning quality rather than answer-token leakage or verbosity, and that the Knowledge Index measures factual correctness rather than GPT-4o self-consistency. As it stands, the headline quantitative claims rest on metrics whose construct validity has not been demonstrated.

major comments (4)
  1. [§3.3, Eqs. (1)–(4)] The InfoGain metric is computed as the reduction in perplexity of the correct-answer tokens A* conditioned on Q and the steps s1:i. Because the score is evaluated on the tokens of A* itself, any step that states, restates, or paraphrases the answer will mechanically reduce PPL even if it contains no new logical inference; conversely, a terse step that supplies the key deduction without mentioning the answer will receive a small ΔPPL. The GPT-4o decomposition in Appendix B.2 does not exclude the final-answer sentence, and Figure 3 explicitly labels a step as "Reasoning + Answer" with step text "The final answer is.....". This means the Section 4.2 conclusion that SFT lowers InfoGain by 37.1% and RL raises it conflates reasoning quality with answer-token leakage, answer placement, and decomposition choices. I ask for a control that removes or masks answer-containing sentences before computing PPL, or a variant that scores steps against the probability of the next reasoning step rather than the final answer, and a report of how the SFT/RL differences survive that control.
  2. [§3.3 (InfoGain validity)] No evidence is presented that the perplexity-gap quantity ΔI tracks independent judgments of reasoning quality. The paper asserts that "a higher ΔI indicates that more information emerges during reasoning, reflecting stronger reasoning capabilities," but this interpretation is not validated against human annotations or an established step-quality benchmark. Table 2 shows only that a different metric (ROSCOE-SA) is insensitive to the differences the authors care about; it does not establish that InfoGain is a faithful measure of reasoning quality. Given that all three headline findings depend on InfoGain, I request a validation study comparing ΔI with human-rated step informativeness or with a known reasoning-quality metric on the same responses. Without this, the central claim that SFT harms reasoning while RL improves it is an assumption rather than a demonstrated result.
  3. [§4.1–4.2, Tables 1 and Figures 6–8] All reported comparisons are point estimates from single training runs, with no error bars, confidence intervals, or significance tests. Several differences used to support the claims are extremely small (e.g., InfoGain 0.157 vs. 0.161 and 0.192 vs. 0.190 in Table 1; KI 63.40 vs. 64.23). The reader cannot assess whether the reported 38.9%/37.1% InfoGain drop, the "RL improves knowledge index by 12.4 points" claim in Figure 8, or the correlation coefficients in Figure 5 are robust to seed variation and to randomness in the GPT-4o decomposition and judgment steps. At minimum, the authors should report variance over multiple random seeds and provide a significance test for the main comparisons; without this, the quantitative strength of the findings is not established.
  4. [§3.4 and Appendix B.2 (KI validation)] The Knowledge Index pipeline uses GPT-4o for knowledge extraction, for query generation, and for the final consistency judgment against retrieved passages, and the manuscript does not report human agreement on KI or a comparison with a curated gold standard. Because the same model produces both the extracted knowledge point and the consistency verdict, KI may measure GPT-4o's self-consistency rather than objective factual correctness. The paper also states that math knowledge is not evaluated ("as the knowledge base of math" is an incomplete sentence in §4.2), so the knowledge side of the decomposition is only tested in the medical domain. This weakens the knowledge component of the framework and the medical-domain claims that depend on KI differences, such as the 6.2% and 12.4% KI gains discussed in Sections 4.1 and 4.2.
minor comments (5)
  1. [Abstract vs. §4.2] The abstract reports an average InfoGain drop of 38.9% after SFT, while Section 4.2 reports an average reduction of 37.1%. These numbers should be reconciled or the discrepancy explained.
  2. [Appendix B.2, Figure 11] The math reasoning decomposition prompt begins "You are a helpful, pattern-following medical assistant," which appears to be a copy-paste error from the medical prompt.
  3. [§4.2, Figure 6] The sentence "We only present the knowledge index metric for medical benchmarks, as the knowledge base of math" is incomplete; please clarify whether math knowledge is omitted and, if so, why.
  4. [§4.2, Figure 6(a)] The text states "the SFT-ed model outperforms both its RL-ed and base counterparts, with average gains of 4.6% and 6.2%, respectively (SFT-ed 49.8% > RL-ed 49.8% > base 48.2%)", but SFT-ed and RL-ed are both listed as 49.8%, which contradicts "outperforms"; the numbers and the comparison should be checked.
  5. [Figure 8 caption] The caption contains a typo: "knowlegde" should be "knowledge".

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: InfoGain and KI are computed from model outputs and external references; the one overlapping-author citation is not load-bearing.

full rationale

The paper's central claims do not reduce to their inputs by construction. InfoGain is defined directly by Eqs. (1)-(4) as the average perplexity reduction of the correct answer over adjacent reasoning steps, computed with an untrained Qwen2.5-7B judge; Knowledge Index is computed by GPT-4o-based extraction, retrieval from an external medical database, and consistency judgment. The SFT and RL findings (accuracy gains, InfoGain drops, KI changes) are measured outcomes of these metrics applied to model outputs, not parameters fitted to the conclusions. No fitted value is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. The only notable self-reference is the citation of Chen et al. [8], which shares authors with this paper, to support the interpretation that 'RL introduces little new knowledge to the model'; however, this premise is also supported by the independent citation [10] and by the paper's own case study in Fig. 9, so the self-citation is not load-bearing. A separate concern, that Eq. (1) scores P(A*|Q, s1:i) and may conflate answer-token leakage or verbosity with reasoning quality, is a construct-validity limitation of the proposed metric rather than a circular derivation; it does not make the findings equivalent to the metric's definition. Accordingly, the analysis is self-contained against external benchmarks and no circular step is exhibited.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of two proxy metrics, the faithfulness of GPT-4o-based decomposition and verification, and the representativeness of a narrow model family. There are no fitted free parameters and no invented physical or conceptual entities beyond the two evaluation metrics themselves.

assumptions (4)
  • domain assumption The untrained Qwen2.5-7B model's probability assigned to the correct answer after each reasoning step is a valid measure of that step's reasoning informativeness.
    Used to define InfoGain in Section 3.3, Equation (1) and Equation (2); the reasoning-quality axis rests entirely on this proxy.
  • domain assumption GPT-4o can faithfully decompose reasoning chains into logical steps without omitting or distorting the model's actual reasoning.
    Response decomposition in Section 3.2 relies on GPT-4o with domain-specific prompts, and all downstream metrics operate on these automatically generated steps.
  • domain assumption The external medical knowledge database and GPT-4o consistency judgments provide reliable ground truth for knowledge correctness.
    Knowledge Index in Section 3.4 depends on retrieval plus GPT-4o Boolean consistency checks, with no human validation reported.
  • domain assumption Single training runs on 7B-parameter Qwen models are sufficient to support cross-domain generalizations about SFT and RL.
    The empirical findings claim effects across medical and math domains based on one model family and no repeated seeds or statistical tests.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge or Reasoning? A Close Look at How LLMs Think Across Domains." pith.science (2026). https://pith.science/paper/FFPE7BTQ

@misc{pith2026250602126,
  author       = {Pith},
  title        = {Pith review of: Knowledge or Reasoning? A Close Look at How LLMs Think Across Domains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FFPE7BTQ}},
  note         = {Machine review of arXiv:2506.02126}
}
read the original abstract

Recent advances in reasoning-enhanced Large Language Models such as OpenAI-o1/3 and DeepSeek-R1 have significantly improved performance on complex tasks. However, the quality and transparency of their internal reasoning processes remain underexplored. This work moves beyond the final-answer accuracy and investigates step-by-step reasoning in the medical and mathematical domains by explicitly decomposing the thinking trajectories into two parts: knowledge and reasoning. Specifically, we introduce a fine-grained evaluation framework that judges: (1) the correctness of knowledge used (measured by Knowledge Index (KI)) and (2) the quality of reasoning (measured by Information Gain (InfoGain)). Using this framework, we study R1-distilled and base Qwen models trained with supervised fine-tuning (SFT) and/or reinforcement learning (RL) in the medical and math domains. Three intriguing findings emerge: (1) The general reasoning abilities in R1-distilled models do not transfer effectively to the medical domain through either SFT or RL. (2) SFT raises final-answer accuracy in both domains, but often at the cost of reasoning quality: InfoGain drops by 38.9% on average compared with untrained models; In the medical domain, however, SFT remains crucial because domain knowledge is indispensable. (3) RL enhances medical reasoning by pruning inaccurate or irrelevant knowledge from reasoning paths, thereby improving both reasoning accuracy and knowledge correctness.

Figures

Figures reproduced from arXiv: 2506.02126 by the authors.

Figure 1
Figure 1. Reasoning and Knowledge are Different Evaluation Aspects for LLMs. A reasoning step may effectively reduce uncertainty toward the final answer despite relying on incorrect knowledge (e.g., Step 3), or it may present factually correct but irrelevant/redundant knowledge that hinders reasoning efficiency (e.g., Step 4). Accuracy alone fails to capture these nuances. We introduce two complementary metrics that separatel… view at source ↗
Figure 2
Figure 2. Evaluation Pipeline. (a) We decompose the model’s reasoning into reasoning steps using gpt4o[27] , then evaluate the (b) information gain and (c) knowledge index of each reasoning step. have focused on enhancing the reasoning abilities of LLMs by producing high-quality training datasets [40, 17, 39] or crafting comprehensive reward mechanisms [8, 4]. However, these efforts are directed exclusively towards boosting t… view at source ↗
Figure 3
Figure 3. Example of reasoning decomposition. Every stage of reasoning corresponds to a logical step si , accompanied by the specific knowledge point it contains (ki). In this section, we investigate the roles of reasoning and knowledge in model responses by first decomposing them into separate steps, and then introducing two novel metrics to evaluate the knowledge and reasoning abilities embedded in those responses. 3.2 Resp… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Comparison of SFT Qwen-Base and Qwen-R1. While the medical knowledge of all reasoning steps in Qwen-R1 + SFT is correct, the ignorance of considering more appropriate treatment for the specified disease results in an incorrect answer. 3.4 Knowledge Index As for accessi…
Figure 5
Figure 5. Figure 5: Correlations between the pro￾posed two metrics and accuracy. Different tasks require different levels of knowledge and/or reasoning capabilities in LLMs. Trained Qwen-Base Outperforms its R1-distilled Counterpart. From the overall accuracy results, we observe that Qwen…
Figure 6
Figure 6. Figure 6: Comparison between medical and math domain. (a) In mathematics, RL enhances accuracy most, whereas in the medical field, SFT provides a greater enhancement in overall accuracy; (b) Across both fields, RL is more adept at boosting information gain, whereas SFT results i…
Figure 7
Figure 7. Figure 7: Comparison of Models w and wo SFT We compare both factual grounding and reasoning efficiency. SFT adds redundant reasoning steps, reducing per-step information gain and inference efficiency. 47.02 55.33 53.67 44.57 + 9.44 + 11.89 + 11.57 + 16.51 40 45 50 55 60 65 70 Me…
Figure 8
Figure 8. Figure 8: RL improves knowledge Index. In the medical domain, applying RL to base model before or post SFT consistently improves the knowlegde index [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Comparison of models w/ and w/o RL. We compare the knowledge correctness of the models’ reasoning steps. RL improves the knowledge correctness by guiding the model to select reasoning paths that have fewer knowledge errors. To further investigate, we conduct a case stu…
Figure 10
Figure 10. Figure 10: Medical Reasoning Decomposition Prompt Full prompt employed to decompose the model’s reasoning into reasoning steps using gpt4o. Prompt for Math Reasoning Decomposition messages = [ """ You are a helpful, pattern-following medical assistant. Given a paragraph of reaas…
Figure 11
Figure 11. Figure 11: Math Reasoning Decomposition Prompt Full prompt employed to decompose the model’s reasoning into reasoning steps using gpt4o. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Medical Knowledge Retrieve Prompt Full prompt employed to identify the knowledge pertinent to the reasoning step using gpt4o. Prompt for Planning Consistent messages = [ """ You are a helpful, pattern-following medical assistant. Given two lists of planning lists, you…
Figure 13
Figure 13. Figure 13: Planning Consistency Evaluation Prompt Full prompt employed to evaluate whether the reasoning step is consistent with the retrieved facts using gpt4o. C More experiment results C.1 Data Filtering for Medical Reasoning Previous studies in general reasoning domains, suc…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Prescriptive Scaling Reveals the Evolution of Language Model Capabilities

    cs.LG 2026-02 conditional novelty 6.0 of 10

    For most benchmarks, the best achievable post-training accuracy follows a stable sigmoid curve in pre-training compute; math reasoning is the exception, with a boundary that keeps rising over time.

  2. Kernel-Based Sparse Additive Nonlinear Model Structure Detection through a Linearization Approach

    eess.SY 2025-08 unverdicted novelty 6.0 of 10

    The paper uses an LPV linearization and sparse RKHS estimators to detect the additive structure of continuous-time nonlinear models.

Reference graph

Works this paper leans on

54 extracted references · 20 canonical work pages · cited by 2 Pith papers

  1. [1]

    https:// artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions, 2024

    American invitational mathematics examination (aime) 2024 problems. https:// artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions, 2024. Accessed 7 May 2025

    work page 2024

  2. [2]

    https://www.vals.ai/benchmarks/ math500-03-24-2025, 2025

    Math500 benchmark (2025-03-24 release). https://www.vals.ai/benchmarks/ math500-03-24-2025, 2025. Accessed 7 May 2025

    work page 2025

  3. [3]

    https://www.maa.org/math-competitions, 2025

    Usamo 2025 problems and solutions. https://www.maa.org/math-competitions, 2025. Accessed 7 May 2025

    work page 2025

  4. [4]

    L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    arXiv 2025

  5. [5]

    Large language models for mathematical reasoning: Progresses and challenges.arXiv preprint arXiv:2402.00157, 2024

    Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges.arXiv preprint arXiv:2402.00157, 2024

    arXiv 2024

  6. [6]

    Llama-nemotron: Efficient reasoning models.arXiv preprint arXiv:2505.00949, 2025

    Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, et al. Llama-nemotron: Efficient reasoning models.arXiv preprint arXiv:2505.00949, 2025

    arXiv 2025

  7. [7]

    Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, andet al.Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877–1901, 2020

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, andet al.Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877–1901, 2020

    work page 1901

  8. [8]

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468, 2025

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468, 2025

    arXiv 2025

Show all 54 references
  1. [9]

    Huatuogpt-o1, towards medical complex reasoning with llms.arXiv preprint arXiv:2412.18925, 2024

    Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-o1, towards medical complex reasoning with llms.arXiv preprint arXiv:2412.18925, 2024

    2024 arXiv

  2. [10]

    Sft memorizes, rl generalizes: A comparative study of foundation model post-training.arXiv preprint arXiv:2501.17161, 2025

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training.arXiv preprint arXiv:2501.17161, 2025

    2025 arXiv

  3. [11]

    Deepseek -r1-distill-qwen-7b

    DeepSeek AI. Deepseek -r1-distill-qwen-7b. https://huggingface.co/deepseek-ai/ DeepSeek-R1-Distill-Qwen-7B, 2025. HuggingFace model card, accessed 7 May 2025

    2025

  4. [12]

    Rlhf workflow: From reward modeling to online rlhf

    Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. Rlhf workflow: From reward modeling to online rlhf. Transactions on Machine Learning Research, 2024. Technical Report. 10

    2024

  5. [13]

    Large language model influence on diagnostic reasoning: a randomized clinical trial.JAMA Network Open, 7(10):e2440969– e2440969, 2024

    Ethan Goh, Robert Gallo, Jason Hom, Eric Strong, Yingjie Weng, Hannah Kerman, Joséphine A Cool, Zahir Kanjee, Andrew S Parsons, Neera Ahuja, et al. Large language model influence on diagnostic reasoning: a randomized clinical trial.JAMA Network Open, 7(10):e2440969– e2440969, 2024

    2024

  6. [14]

    Roscoe: A suite of metrics for scoring step-by-step reasoning.arXiv preprint arXiv:2212.07919, 2022

    Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. Roscoe: A suite of metrics for scoring step-by-step reasoning.arXiv preprint arXiv:2212.07919, 2022

    2022 arXiv

  7. [15]

    Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models

    Neel Guha, Julian Nyarko, Daniel Ho, Christopher Ré, Adam Chilton, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, et al. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in Neural...

    2023

  8. [16]

    Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874, 2021

    Dan Hendrycks, Steven Basart, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  9. [17]

    m1: Unleash the potential of test-time scaling for medical reasoning with large language models.arXiv preprint arXiv:2504.00869, 2025

    Xiaoke Huang, Juncheng Wu, Hui Liu, Xianfeng Tang, and Yuyin Zhou. m1: Unleash the potential of test-time scaling for medical reasoning with large language models.arXiv preprint arXiv:2504.00869, 2025

    2025

  10. [18]

    Am-thinking-v1: Advancing the frontier of reasoning at 32b scale.arXiv preprint, 2025

    Yunjie Ji, Xiaoyu Tian, Sitong Zhao, et al. Am-thinking-v1: Advancing the frontier of reasoning at 32b scale.arXiv preprint, 2025

    2025

  11. [19]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams.arXiv preprint arXiv:2009.13081, 2020

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams.arXiv preprint arXiv:2009.13081, 2020

    2009 arXiv

  12. [20]

    Cohen, and Xinghua Lu

    Qiao Jin, Bhavdeep Dhingra, Zhengping Liu, William W. Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. InEMNLP-IJCNLP, pages 2567–2577. Association for Computational Linguistics, 2019

    2019

  13. [21]

    Is that your final answer? test-time scaling improves selective question answering.arXiv preprint, 2025

    William Jurayj, Jeffrey Cheng, and Benjamin Van Durme. Is that your final answer? test-time scaling improves selective question answering.arXiv preprint, 2025

    2025

  14. [22]

    Solving quantitative reasoning problems with language models.arXiv preprint arXiv:2206.14858, 2022

    Aitor Lewkowycz, Ethan Du, Klaus Siniscalchi, Jason Wei, Xuezhi Wang, et al. Solving quantitative reasoning problems with language models.arXiv preprint arXiv:2206.14858, 2022

    2022 arXiv

  15. [23]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint, 2025

    Wenfeng Liang, DeepSeek-AI, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint, 2025

    2025

  16. [24]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019

    2019

  17. [25]

    s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  18. [26]

    Capa- bilities of gpt-4 on medical challenge problems.arXiv preprint arXiv:2303.13375, 2023

    Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. Capa- bilities of gpt-4 on medical challenge problems.arXiv preprint arXiv:2303.13375, 2023

    2023 arXiv

  19. [27]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    OpenAI. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  20. [28]

    Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. InProceedings of the Conference on Health, Inference, and Learning, pages 248–260. PMLR, 2022

    2022

  21. [29]

    Llm evaluators recognize and favor their own generations.Advances in Neural Information Processing Systems, 37:68772–68802, 2024

    Arjun Panickssery, Samuel Bowman, and Shi Feng. Llm evaluators recognize and favor their own generations.Advances in Neural Information Processing Systems, 37:68772–68802, 2024

    2024

  22. [30]

    Receval: Evaluating reasoning chains via correctness and informativeness.arXiv preprint arXiv:2304.10703, 2023

    Archiki Prasad, Swarnadeep Saha, Xiang Zhou, and Mohit Bansal. Receval: Evaluating reasoning chains via correctness and informativeness.arXiv preprint arXiv:2304.10703, 2023. 11

    2023 arXiv

  23. [31]

    ZeRO: Memory opti- mizations toward training trillion parameter models

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. ZeRO: Memory opti- mizations toward training trillion parameter models. InProceedings of SC20: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2020

    2020

  24. [32]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. InarXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  25. [33]

    Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning.arXiv preprint arXiv:2504.13914, 2025

    ByteDance Seed. Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning.arXiv preprint arXiv:2504.13914, 2025

    2025

  26. [34]

    Chain of logic: Rule-based reasoning with large language models.arXiv preprint arXiv:2402.10400, 2024

    Sergio Servantez, Joe Barrow, Kristian Hammond, and Rajiv Jain. Chain of logic: Rule-based reasoning with large language models.arXiv preprint arXiv:2402.10400, 2024

    2024 arXiv

  27. [35]

    Benchmarking large language models for math reasoning tasks.arXiv preprint arXiv:2408.10839, 2024

    Kathrin Seßler, Yao Rong, Emek Gözlüklü, and Enkelejda Kasneci. Benchmarking large language models for math reasoning tasks.arXiv preprint arXiv:2408.10839, 2024

    2024 arXiv

  28. [36]

    Kimi k1.5: Scaling Reinforcement Learning with LLMs.arXiv preprint arXiv:2501.12599, 2025

    Kimi Team. Kimi k1.5: Scaling Reinforcement Learning with LLMs.arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  29. [37]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024

    2024

  30. [38]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark.arXiv preprint arXiv:2406.01574, 2024

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark.arXiv preprint arXiv:2406.01574, 2024

    2024 arXiv

  31. [39]

    Star-1: Safer alignment of reasoning llms with 1k data.arXiv preprint arXiv:2504.01903, 2025

    Zijun Wang, Haoqin Tu, Yuhan Wang, Juncheng Wu, Jieru Mei, Brian R Bartoldson, Bhavya Kailkhura, and Cihang Xie. Star-1: Safer alignment of reasoning llms with 1k data.arXiv preprint arXiv:2504.01903, 2025

    2025

  32. [40]

    Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs.arXiv preprint arXiv:2504.00993, 2025

    Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyunjin Cho, Chang-In Choi, et al. Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs.arXiv preprint arXiv:2504.00993, 2025

    2025 arXiv

  33. [41]

    A preliminary study of o1 in medicine: Are we closer to an ai doctor?arXiv preprint arXiv:2409.15277, 2024

    Yunfei Xie, Juncheng Wu, Haoqin Tu, Siwei Yang, Bingchen Zhao, Yongshuo Zong, Qiao Jin, Cihang Xie, and Yuyin Zhou. A preliminary study of o1 in medicine: Are we closer to an ai doctor?arXiv preprint arXiv:2409.15277, 2024

    2024 arXiv

  34. [42]

    Rcot: Detecting and rectifying factual inconsistency in reasoning by reversing chain-of-thought.arXiv preprint arXiv:2305.11499, 2023

    Tianci Xue, Ziqi Wang, Zhenhailong Wang, Chi Han, Pengfei Yu, and Heng Ji. Rcot: Detecting and rectifying factual inconsistency in reasoning by reversing chain-of-thought.arXiv preprint arXiv:2305.11499, 2023

    2023 arXiv

  35. [43]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, et al. Qwen3 technical report, 2025

    2025

  36. [44]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement.arXiv preprint arXiv:2409.12122, 2024

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via se...

    2024 arXiv

  37. [45]

    Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    2025 arXiv

  38. [46]

    Temporal data meets llm–explainable financial time series forecasting.arXiv preprint arXiv:2306.11025, 2023

    Xinli Yu, Zheng Chen, Yuan Ling, Shujing Dong, Zongyi Liu, and Yanbin Lu. Temporal data meets llm–explainable financial time series forecasting.arXiv preprint arXiv:2306.11025, 2023

    2023 arXiv

  39. [47]

    Online-dpo -r1: Unlocking effective reasoning without the ppo overhead

    Hanning Zhang, Jiarui Yao, Chenlu Ye, Wei Xiong, and Tong Zhang. Online-dpo -r1: Unlocking effective reasoning without the ppo overhead. https://www.notion.so/ Online-DPO-R1-1908b9a70e7b80c3bc83f4cf04b2f175. Project blog post, Feb. 2025

    2025

  40. [48]

    Verify-and-edit: A knowledge-enhanced chain-of-thought framework.arXiv preprint arXiv:2305.03268, 2023

    Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, and Lidong Bing. Verify-and-edit: A knowledge-enhanced chain-of-thought framework.arXiv preprint arXiv:2305.03268, 2023

    2023 arXiv

  41. [49]

    Medxpertqa: Benchmarking expert-level medical reasoning and understanding.arXiv preprint arXiv:2501.18362, 2025

    Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu, et al. Medxpertqa: Benchmarking expert-level medical reasoning and understanding.arXiv preprint arXiv:2501.18362, 2025. 12 Technical Appendices and Supplementary Material Technical appendices with additional results, fi...

    2025 arXiv

  42. [51]

    planning

    "planning" : a brief description of the main idea of the step. "planning" parts from all steps should form a coherent and logical sequence

  43. [52]

    action" : a detailed description of the actions taken in this step

    "action" : a detailed description of the actions taken in this step. "action" is taken based on the "planning" part of the step, and should contain specific medical knowledge or procedures

  44. [53]

    step_text

    "step_text" : sentence(s) from the input reasoning paragraph that corresponds to this step. ### Output Format: Strictly follow the JSON structure below. ```json {{"Steps: [ {{"id" : 1, "planning" : "Planning for step 1", "action" : "Action for step 1", "step_text" : "Correspon...

  45. [54]

    "id" : a unique number for the step

  46. [55]

    query" :

    "step" : coresponding sentence(s) from the input reasoning paragraph, each reasoning step should describe a logical step in the question solution process. ### Output Format: Strictly follow the structure below, and do not add any other content. ```text ###step: Corresponding s...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.