The reviewed record of science sign in
Pith

arxiv: 2607.05199 · v1 · pith:5GAD77AG · submitted 2026-07-06 · cs.AI

Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models

Raj Jaiswal , Dhruv Jain , Rishabh Dhawan , Sree Krishna Uppalapati , Shin'ichi Satoh , Tanuja Ganu , Rajiv Ratn Shah This is my paper

Reviewed by Pith2026-07-07 23:46 UTCglm-5.2pith:5GAD77AGopen to challenge →

classification cs.AI
keywords errorsphysicsreasoningstep-levelacrosserrorfailurefeedback
0
0 comments X

The pith

Step-level error rewards cut physics reasoning failures in small language models by up to 33%

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes a training method for small language models (under 4 billion parameters) that aims to fix physics reasoning at the exact step where it first goes wrong, rather than scoring the entire solution as a single unit. The central mechanism is a step-level reward r = e_first/(n+1), where e_first is the index of the first reasoning error and n is the total number of steps: solutions that stay correct longer receive higher reward, and earlier failures are penalized more heavily. During training, an external verifier (a large language model) compares the small model's solution against a ground-truth solution, identifies the first erroneous step, classifies the error into one of three types (problem miscomprehension, conceptual misapplication, or calculation error), and generates structured feedback routed through a type-specific channel. The small model then produces a revised solution conditioned on that feedback, and a policy gradient update with KL regularization adjusts the model's weights to favor the revision. The model never sees correct solutions as generation targets — it learns only from feedback on its own mistakes. The framework requires no annotated step-level preference data and uses the external verifier only at training time, not at inference. Across four open-source models and five physics benchmarks, the authors report accuracy gains of 17–20 percentage points over chain-of-thought prompting and 10–16 points over the strongest baseline, with calculation errors dropping from 56.9% to 23.5% and miscomprehension errors from 22.3% to 12.0% in the best cases. Conceptual misapplication errors, however, remain above 42% across all conditions and are identified as the hardest unresolved failure mode.

Core claim

The paper's central claim is that pinpointing the first reasoning error in a multi-step physics solution — and conditioning feedback on the error type — lets a small language model learn to revise its own reasoning more effectively than whole-response preference optimization or supervised fine-tuning, without requiring annotated step-level data or an inference-time verifier. The reward formula r = e_first/(n+1) is the load-bearing object: it converts the position of the first error into a scalar training signal, making earlier errors costlier than later ones. The three-channel feedback architecture (re-reading the problem for miscomprehension, retrieving correct physical laws for conceptual,

What carries the argument

step-level reward r = e_first/(n+1), error-type-conditioned feedback channels, policy gradient with KL regularization against a frozen SFT reference policy

If this is right

  • If the step-level reward mechanism generalizes, the same identify-first-error-and-condition-feedback approach could apply to other sequential reasoning domains — chemistry, multi-step arithmetic, logical proof — wherever an external verifier can locate the first deviation from a reference solution during training.
  • The persistence of conceptual misapplication errors above 42% suggests that retrieval-augmented feedback supplying the correct formula is insufficient for teaching correct application; a feedback channel that demonstrates principle application, not just retrieval, may be needed.
  • The framework's dependence on a large external verifier at training time but not at inference means the cost of verifier quality is paid entirely during training, making the deployed model lightweight — but any systematic verifier errors are baked into the policy and invisible at evaluation time.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reward formula r = e_first/(n+1) implicitly assumes that error position is the most informative feature for training signal quality. An alternative — weighting by error type severity or by downstream propagation distance — could be tested by ablating the reward shape while holding feedback constant.
  • The framework's training loop skips samples where the first attempt is already correct (r > threshold). If a small model's initial accuracy is low, most training samples enter the revision loop; if initial accuracy is high, the effective training set shrinks. This suggests the method may have a natural ceiling tied to the model's first-attempt accuracy distribution.
  • The coupling of verifier and feedback generator in the same large model (GPT-4o) means that error-type classification accuracy and feedback quality are not independent variables. A controlled study using a high-accuracy classifier with a low-quality feedback generator (or vice versa) would isolate which component drives the gains.
  • The observation that DPO increases conceptual misapplication while the proposed framework reduces it suggests that whole-response preference signals may actively distort conceptual reasoning in small models, and that step-level localization is not merely an incremental improvement but may prevent a specific failure mode introduced by response-level training.

Load-bearing premise

The entire reward signal and feedback routing depend on a single large language model (GPT-4o) correctly identifying and classifying the first reasoning error against a ground-truth solution during training. If that verifier systematically misclassifies error types or misidentifies which step first goes wrong, the reward is wrong, feedback is sent to the wrong correction channel, and the policy gradient update optimizes against a corrupted signal — with no way to detect the腐败

What would settle it

Run the framework with a verifier whose error-classification accuracy is independently measured and deliberately degraded (e.g., by injecting controlled misclassification rates). If accuracy gains degrade proportionally to verifier error rate, the framework's success is verifier-dependent; if gains persist, the feedback signal is robust to misclassification.

Figures

Figures reproduced from arXiv: 2607.05199 by Dhruv Jain, Rajiv Ratn Shah, Raj Jaiswal, Rishabh Dhawan, Shin'ichi Satoh, Sree Krishna Uppalapati, Tanuja Ganu.

Figure 1
Figure 1. Figure 1: Step-Level Reward Penalizes Earlier Failures More Heavily; Feedback Targets the Precise Point of Failure. πθ generates y1, receives error-type-conditioned feedback p1 from GPT-4o, and revises to y2; reward r = efirst/(n + 1) drives policy gradient update. Discussed §3. generates an initial solution y1, an external verifier (GPT-4o) identifies the first reasoning error against ground truth y ∗ and computes … view at source ↗
Figure 2
Figure 2. Figure 2: Targeted Feedback at the First Error Step Enables the Model to Reason Toward a Correct Solution. Each column shows one Baseline responses on the same problem; red annotations mark the first error step and type, green checkmarks in <final_answer> indicate correct quantities. Structured feedback on the identified error step progressively corrects reasoning across revision passes. Discussed in §5.3 & §6.1 cor… view at source ↗
Figure 3
Figure 3. Figure 3: Common Topic Distributions Confirm Data similarity Across Training and Benchmarks. Topic distribution (%) across the SFT training corpus and evaluation benchmarks; the same five domains appear in both but at substantially different proportions. 4.2 Models We evaluate four open-source language models: Qwen 2.5 1.5B Instruct (Qwen Team, 2024), LLaMA 3.2 (1B and 3.2 3B) Instruct (Grattafiori et al., 2024), an… view at source ↗
Figure 4
Figure 4. Figure 4: Accuracy Improves With Model Size; Gains Are Larger on Easy Than Hard Benchmarks. CoT accuracy (%) across models (1B–72B and GPT-4o); across SciEval, MMLU-High, MMLU-College (easy) and JEEBench, PhysicsQA (hard). Discussed in §5.1. 4.4 Evaluation Final answer accuracy is the primary metric across all five benchmarks. We employ a three-step shadow protocol: Step 1 verifies answer correct￾ness via string mat… view at source ↗
Figure 5
Figure 5. Figure 5: The physics formula sheet used as the RAG knowledge base. The corpus is strictly limited to formulas, [PITH_FULL_IMAGE:figures/full_fig_p026_5.png] view at source ↗
read the original abstract

Physics reasoning fails structurally in small language models: an error at any step propagates forward, corrupting every inference that follows. Limited domain knowledge, hallucination under multi-step derivation, and distributional sensitivity compound this failure. We propose a step-level reward framework that identifies the first reasoning error, generates targeted structured feedback, and trains the model to revise its solution via policy gradient with KL regularization, without exposing it to ground truth solutions as generation targets. Unlike annotation-dependent step-level methods, no preference data construction is required and the external verifier operates exclusively at training time. Across five physics benchmarks, our framework delivers accuracy gains of 17-20% over CoT prompting and 10-16% over the strongest baseline, reduces calculation errors from 56.9% to 23.5%, and reduces miscomprehension errors from 22.3% to 12.0% in the best observed cases. Conceptual errors reduce from 89.7% to 68.7%, yet persist as the hardest failure mode across all conditions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 8 minor

Summary. The paper proposes a step-level reward framework for physics reasoning in small language models (SLMs). The method operates in three stages: (1) SFT warm-up on structured XML-tagged chain-of-thought solutions, (2) a step-level reward mechanism where GPT-4o identifies the first reasoning error against ground truth and computes a position-dependent reward r = e_first/(n+1), and (3) error-type-conditioned structured feedback generation (MC, CM, CE channels) followed by policy gradient updates with KL regularization. The framework is evaluated on four SLMs (1B–3.8B) across five physics benchmarks, reporting accuracy gains of 17–20% over CoT and 10–16% over the strongest baseline, with error-type-specific reductions in calculation and miscomprehension errors. The paper includes ablations on feedback channels, a comparison table with prior work across five dimensions, and an honest limitations section.

Significance. The paper addresses a genuine gap: step-level reward methods for reasoning typically require annotated preference data, and this work proposes a mechanism that avoids that requirement while operating in a domain (physics) where SLMs struggle structurally. The structured feedback routing by error type is a reasonable design choice, and the error taxonomy (MC/CM/CE) is well-motivated. The authors provide code listings, full hyperparameter tables, training prompt templates, and dataset samples, supporting reproducibility. The limitations section is notably candid about several threats to validity. However, the central empirical claims rest on uncharacterized variance and an unvalidated training-time verifier, which together weaken the strength of the evidence presented.

major comments (3)
  1. §8; Tables 3–4: All results are from single training runs (seed 42, §B). No multi-seed validation or variance estimates are reported. Given that the RL training loop involves stochastic generation, GPT-4o verifier queries, and feedback routing, the reported accuracy differences—some as small as 2–3 percentage points between conditions—cannot be distinguished from run-to-run noise. This is load-bearing for the claim of consistent gains across models and benchmarks. At minimum, 3 seeds with mean ± std for the primary accuracy tables would be needed to support the comparative claims in Tables 3–4 and the ablation in Table 7.
  2. §3.2, §3.3, Algorithm 1 lines 4–5, 11–12: The entire reward signal and feedback routing depend on GPT-4o correctly identifying the first error step (e_first) and classifying the error type (c1 ∈ {MC, CM, CE}) against ground truth solutions. The human evaluation (Cohen's Kappa 0.75–0.89, Table 6) validates the evaluation pipeline (Step 3 error analysis in Appendix F), not the training-time verifier. No accuracy metric for GPT-4o's verifier role is reported. The paper acknowledges this in §8: 'systematic GPT-4o classification errors would corrupt both the reward signal and feedback conditioning y2, without being detectable from accuracy metrics alone.' This is a correctness-risk concern: if GPT-4o systematically misclassifies error types or misidentifies the first error step, the reward r = e_first/(n+1) is wrong and feedback is routed to the wrong channel, making the policy gradientupdate
  3. Appendix B, Listing 2, line 15: The reward computation uses regex `r'(?:#+ )?Step :d+: '` to count total steps (total_steps), but the SFT format (Appendix A) uses seven XML tags (<problem_analysis>, <principle>, etc.) without 'Step N:' headers. If the regex does not match, total_steps defaults to 1 (line 20: `total_steps = len(matches) if matches else 1`), making reward = e_first/2 regardless of solution length. This would collapse the position-dependent penalty—the core mechanism distinguishing this work from response-level reward methods—into a near-binary signal. The authors should either confirm that the model generates 'Step N:' headers during training (contradicting the XML format in Appendix A) or report the frequency of regex match failures and their impact on the reward distribution.
minor comments (8)
  1. §3.4, Eq. (2): The loss expression mixes log-probability terms and KL divergence without clear grouping. Parentheses around the REINFORCE term vs. the KL term would improve readability.
  2. Table 1: The caption says 'Cell shading indicates per-row rank' but no shading is visible in the text rendering. If shading is present in the PDF, this is fine; otherwise, rank should be indicated numerically.
  3. §4.4: The three-step evaluation protocol is described in the main text but the Step 3 prompt is in Appendix F.2. A forward reference would help readers locate it.
  4. Table 5: The caption references 'Benchmarks' generically but the text (§5.3) specifies PhysicsQA only. The caption should state 'PhysicsQA' for precision.
  5. §6.1: The phrase 'the only model where this channel consistently shows negative values' refers to Qwen 2.5 1.5B for Concept feedback, but the prior sentence discusses Statement feedback for LLaMA 3.2 3B. The transition is abrupt and could confuse readers.
  6. Appendix B vs. Appendix A: The SFT configuration appendix is labeled 'B SFT Training Configuration' but appears after 'A Training Dataset Samples.' The lettering is inconsistent with standard ordering (A, B, C...).
  7. §8: 'parse failures default to minimal reward and their frequency is not reported' — this is acknowledged but the default value is not specified. Is it 0? This should be stated explicitly.
  8. References: Lai et al. 2024a and 2024b are identical citations (same arXiv ID). One should be removed.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The three major comments are all valid concerns. We agree that (1) multi-seed validation is needed to support the comparative claims, (2) the training-time GPT-4o verifier accuracy should be directly measured and reported, and (3) the regex-based step-counting mechanism has a format mismatch with the XML-tagged SFT format that must be investigated and corrected. We describe our planned revisions for each point below.

read point-by-point responses

  1. Referee: §8; Tables 3–4: All results are from single training runs (seed 42, §B). No multi-seed validation or variance estimates are reported. Given that the RL training loop involves stochastic generation, GPT-4o verifier queries, and feedback routing, the reported accuracy differences—some as small as 2–3 percentage points between conditions—cannot be distinguished from run-to-run noise. This is load-bearing for the claim of consistent gains across models and benchmarks. At minimum, 3 seeds with mean ± std for the primary accuracy tables would be needed to support the comparative claims in Tables 3–4 and the ablation in Table 7.

    Authors: The referee is correct. Single-seed results without variance estimates are insufficient to support the comparative claims, particularly for differences in the 2–3 percentage point range. We acknowledge this limitation explicitly in §8 of the current manuscript but agree that acknowledgment alone is not adequate for the central empirical claims of the paper. We will rerun all four models across all five benchmarks with at least 3 seeds (42, 123, 456) and report mean ± standard deviation for Tables 3, 4, and 7. We will also report whether the gains over the strongest baseline remain statistically significant (paired t-test or Wilcoxon signed-rank, as appropriate). If any model-benchmark comparison fails to show significance after multi-seed validation, we will qualify the corresponding claim accordingly rather than presenting it as a consistent gain. We note that the largest gains (e.g., LLaMA 3.2 3B on JEEBench: 30.21% → 57.34%, a 27-point improvement) are unlikely to be attributable to seed variance, but we agree that the smaller differences require validation. The revised manuscript will present multi-seed results as the primary tables, with the current single-seed numbers available as supplementary material. revision: yes

  2. Referee: §3.2, §3.3, Algorithm 1 lines 4–5, 11–12: The entire reward signal and feedback routing depend on GPT-4o correctly identifying the first error step (e_first) and classifying the error type (c1 ∈ {MC, CM, CE}) against ground truth solutions. The human evaluation (Cohen's Kappa 0.75–0.89, Table 6) validates the evaluation pipeline (Step 3 error analysis in Appendix F), not the training-time verifier. No accuracy metric for GPT-4o's verifier role is reported. The paper acknowledges this in §8: 'systematic GPT-4o classification errors would corrupt both the reward signal and feedback conditioning y2, without being detectable from accuracy metrics alone.' This is a correctness-risk concern: if GPT-4o systematically misclassifies error types or misidentifies the first error step, the reward r = e_first/(n+1) is wrong and feedback is routed to the wrong channel, making the policy gradientupdate

    Authors: The referee correctly identifies a gap in our validation: the Cohen's Kappa scores in Table 6 measure inter-annotator agreement for the evaluation pipeline (Appendix F, Step 3), not the training-time GPT-4o verifier. These are different systems—the evaluation pipeline uses a separate LLM judge with a different prompt (Appendix F.2), while the training-time verifier uses the prompt in Appendix A (Listing 1). We agree that the training-time verifier's accuracy in (a) identifying the first error step and (b) classifying the error type must be measured directly. In the revision, we will conduct a human evaluation of the training-time verifier on a random sample of at least 200 model-generated solutions from the training loop. For each sample, two human annotators (from the same pool used for Table 6) will independently identify the first error step and error type. We will report: (i) exact-match accuracy for the first error step index, (ii) accuracy for error type classification, and (iii) Cohen's Kappa between GPT-4o and human annotators for both dimensions. We will also report the distribution of reward values (r = e_first/(n+1)) as actually computed during training, which will show whether the reward signal has meaningful variance or is collapsed. If the verifier accuracy is low in certain error categories, we will report this transparently and discuss its implications for the reward signal quality. We acknowledge that we cannot fully eliminate the correctness-risk concern—the framework's dependence on GPT-4o is a fundamental design choice, not a validation gap that can be closed entirely. However, providing direct accuracy metrics for the training-time verifier will allow readers to assess the magnitude of this risk. revision: yes

  3. Referee: Appendix B, Listing 2, line 15: The reward computation uses regex `r'(?:#+ )?Step :d+: '` to count total steps (total_steps), but the SFT format (Appendix A) uses seven XML tags (<problem_analysis>, <principle>, etc.) without 'Step N:' headers. If the regex does not match, total_steps defaults to 1 (line 20: `total_steps = len(matches) if matches else 1`), making reward = e_first/2 regardless of solution length. This would collapse the position-dependent penalty—the core mechanism distinguishing this work from response-level reward methods—into a near-binary signal. The authors should either confirm that the model generates 'Step N:' headers during training (contradicting the XML format in Appendix A) or report the frequency of regex match failures and their impact on the reward distribution.

    Authors: The referee has identified a genuine inconsistency in our code that we must investigate and address honestly. The SFT training format (Appendix A) uses seven XML tags without 'Step N:' headers, yet the reward computation in Listing 2 (Appendix B) uses a regex pattern that matches 'Step N:' headers. There are two possible explanations: (1) the model generates 'Step N:' headers during the RL training loop despite being trained on XML-tagged format (models sometimes introduce formatting variations during generation), in which case the regex matches but the step count may not correspond to the intended XML-tag-based steps; or (2) the regex does not match, total_steps defaults to 1, and the reward collapses to e_first/2, which would significantly weaken the position-dependent penalty mechanism. We cannot determine which scenario occurred without re-examining the training logs, which we will do in the revision. Specifically, we will: (i) inspect a sample of model-generated solutions from the training loop to determine whether 'Step N:' headers appear, (ii) compute the regex match rate across all training samples, (iii) report the actual distribution of total_steps values and reward values as computed during training, and (iv) if the regex is failing, correct the step-counting mechanism to parse XML tags instead (e.g., counting <substitution> and <calculation> sub-blocks as steps) and rerun the experiments with the corrected reward computation. If the correction changes the results, we will report both the original and corrected numbers transparently. If the regex was matching due to model-generated 'Step N:' headers, we will document this and clarify the relationship between the XML-tagged SFT format and the actual generation format during RL training. We agree with the ref's revision: no

Circularity Check

1 steps flagged

No significant circularity; one minor self-citation for the error taxonomy that is not load-bearing for the framework's core derivation.

specific steps
  1. self citation load bearing [§3.2, first paragraph]
    "Physics reasoning failures exhibit three recurring patterns (Jaiswal et al., 2024): Problem Miscomprehension (MC), misidentified objective or misread quantities; Conceptual Misapplication (CM), wrong governing law or principle applied outside validity conditions; and Calculation Error (CE), correct setup but arithmetic or algebraic error in execution."

    The three-category error taxonomy (MC, CM, CE) that structures the reward and feedback routing is cited to Jaiswal et al. (2024), which includes overlapping authors with the present paper. However, this citation is for a descriptive taxonomy, not a mathematical theorem or uniqueness result. The taxonomy is a labeling convention for error types; it does not constrain or define the reward computation or the policy gradient. The reward r = e_first/(n+1) and the feedback routing are constructed independently of this taxonomy — the taxonomy only determines which feedback channel is activated. This is a minor self-citation that does not make the central claim circular.

full rationale

The paper's central derivation chain is: (1) GPT-4o identifies the first error step e_first against ground truth y* (§3.2, Algorithm 1 line 4), (2) reward r = e_first/(n+1) is computed from e_first and total steps n (Eq. 1), (3) GPT-4o generates structured feedback p1 conditioned on the error tuple (§3.3), (4) the model generates a revised solution y2, (5) GPT-4o evaluates y2 to produce r2 (Algorithm 1 line 11), (6) policy gradient update uses r2 (Eq. 2). None of these steps reduce to their inputs by construction. The reward r is a function of e_first and n, both of which are independently determined (e_first by GPT-4o's comparison against y*, n by the solution length). The feedback p1 is generated from the error tuple but is not defined as a function of the reward or the model's output in a way that would make the prediction circular. The policy gradient objective (Eq. 2) is a standard REINFORCE-with-KL formulation where r2 is an externally computed scalar. The reader's concern about GPT-4o serving as both verifier and feedback generator is a correctness/validity concern (whether GPT-4o's classifications are accurate), not a circularity concern (the steps are not definitionally equivalent to their inputs). The paper acknowledges this coupling as a limitation in §8. The one self-citation (Jaiswal et al., 2024) for the error taxonomy is descriptive and not load-bearing for the mathematical derivation. The regex step-counting concern is an implementation correctness issue, not circularity. Score 2 reflects the minor self-citation for the taxonomy, which does not undermine the independence of the core derivation.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical entities, particles, forces, or dimensions. The step-level reward formula r=e_first/(n+1) is a heuristic design choice, not a postulated entity. The error taxonomy (MC, CM, CE) is inherited from prior work (Jaiswal et al. 2024). The free parameters are standard RL training hyperparameters (β, τ, learning rate, batch size, epochs, LoRA rank, temperature) — none are fitted to the evaluation benchmarks, though the lack of multi-seed validation means we cannot assess sensitivity to these choices.

free parameters (7)
  • β (KL penalty) = 0.1
    Controls KL regularization strength in Eq. 2; chosen value stated in §3.4, no sensitivity analysis beyond Appendix A mention
  • τ (early stopping threshold) = 0.9
    Determines which samples are skipped in training loop; §3.3 states value, sensitivity analysis referenced to Appendix A
  • Learning rate (Stage 2 RL) = 5×10⁻⁶
    Stated in Table 8; no justification provided for this specific value
  • Batch size (Stage 2 RL) = 5
    Stated in Table 8; no justification provided
  • Stage 2 epochs = 60
    Stated in Table 8; no justification for epoch count
  • LoRA rank = 16
    Stated in Table 8 for Stage 2; SFT uses rank 32 (Appendix B)
  • Temperature = 1.0
    Stated in Table 8; no sensitivity analysis
axioms (5)
  • domain assumption GPT-4o can reliably identify and classify the first reasoning error in a physics solution against ground truth
    §3.2 Verifier: the entire reward mechanism depends on this; no independent validation of GPT-4o's accuracy as a physics error classifier is provided
  • domain assumption The three-category error taxonomy (MC, CM, CE) with priority ordering is sufficient to capture physics reasoning failures
    §3.2: classification follows MC, CM, CE priority order; the taxonomy is from Jaiswal et al. 2024 (self-cited)
  • ad hoc to paper Step-level reward proportional to error position (r=e_first/(n+1)) is a meaningful training signal for reasoning quality
    §3.2 Eq. 1: the specific functional form is introduced without theoretical justification; alternative forms (e.g., exponential decay, binary) are not compared
  • domain assumption Regex-based step counting correctly identifies the total number of reasoning steps
    Appendix B Listing 2: total_steps is computed via regex pattern matching on 'Step N:' format; non-standard formatting would produce incorrect reward signals (acknowledged in §8)
  • domain assumption Training on 2,494 JEE physics problems generalizes to five evaluation benchmarks with different distributions
    Figure 3 shows topic distribution overlap but different proportions; §8 notes formal semantic deduplication was not performed

pith-pipeline@v1.1.0-glm · 36074 in / 3642 out tokens · 217997 ms · 2026-07-07T23:46:39.881031+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 45 canonical work pages · 6 internal anchors

  1. [1]

    Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models

    Have LLMs advanced enough? A challeng- ing problem solving benchmark for large language models.arXiv preprint arXiv:2305.15074. Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. 2023. A general theoret- ical paradigm to understand learning from human preferences.Preprint, arXiv:2310.12036. ...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  2. [2]

    Training Verifiers to Solve Math Word Problems

    Step-level value preference optimization for mathematical reasoning. InFindings of the Associa- tion for Computational Linguistics: EMNLP 2024. Paul F Christiano, Jan Leike, Tom Brown, Miljan Mar- tic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences.Ad- vances in Neural Information Processing Systems, 30. Karl Cobbe,...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  3. [3]

    Measuring Massive Multitask Language Understanding

    Measuring massive multitask language under- standing.arXiv preprint arXiv:2009.03300. Namgyu Ho, Laura Schmid, and Se-Young Yun. 2022. Large language models are reasoning teachers.arXiv preprint arXiv:2212.10071. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adap- tation ...

    work page internal anchor Pith review Pith/arXiv arXiv 2009

  4. [4]

    Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback

    Step-kto: Optimizing mathematical reason- ing through stepwise binary feedback.Preprint, arXiv:2501.10799. Elita Lobo, Chirag Agarwal, and Himabindu Lakkaraju

    work page internal anchor Pith review Pith/arXiv arXiv

  5. [5]

    On the Impact of Fine-Tuning on Chain-of-Thought Reasoning

    On the impact of fine-tuning on chain-of- thought reasoning.arXiv preprint arXiv:2411.15382. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian- guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. Wiz- ardMath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  6. [6]

    LMFCA-Net: A Lightweight Model for Multi-Channel Speech Enhancement with Efficient Narrow-Band and Cross-Band Attention

    Direct preference optimization: Your lan- guage model is secretly a reward model.Advances in Neural Information Processing Systems, 36:53728– 53741. Nils Reimers and Iryna Gurevych. 2019. Sentence- BERT: Sentence embeddings using siamese BERT- networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pages 3982–39...

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  7. [7]

    Discussed in §3

    * reward2_tensor - beta2 * kl_div 9).mean() E Training Hyperparameters Hyperparameter Value Batch size 5 Stage 1 epochs (SFT) 10 Stage 2 epochs (RL) 60 Learning rate 5×10 −6 KL penaltyβ 0.1 Early stoppingτ 0.9 Max prompt length 512 tokens Max attempt length 1,240 tokens LoRA rank 16 LoRAα 32 Temperature 1.0 Table 8: Training hyperparameters used across al...

    work page 2022

  8. [8]

    m1, m2 separated by r: m1 m2 C r m2r m1+m2 m1r m1+m2

    work page

  9. [9]

    Triangle (CM≡Centroid)yc = h 3 C h 3 h

    work page

  10. [10]

    Semicircular ring: yc = 2r π C 2r πr

    work page

  11. [11]

    Semicircular disc: yc = 4r 3π C 4r 3πr

    work page

  12. [12]

    Hemispherical shell: yc = r 2 Cr r 2

    work page

  13. [13]

    Solid Hemisphere: yc = 3r 8 Cr 3r 8

    work page

  14. [14]

    Cone: the height of CM from the base is h/4 for the solid cone and h/3 for the hollow cone. Motion of the CM: M =∑mi ⃗ vcm = ∑mi⃗ vi M , ⃗ pcm =M⃗ vcm, ⃗ acm = ⃗Fext M Impulse: ⃗J = ∫⃗F dt = ∆⃗ p Collision: m1 m2 v1 v2 Before collision After collision m1 m2 v′ 1 v′ 2 Momentum conservation:m1v1+m2v2 =m1v′ 1+m2v′ 2 Elastic Collision: 1 2m1v12+1 2m2v22 = 1 2...

    work page

  15. [15]

    Boundary conditions: y = 0 at x = 0 and at x =L

    work page

  16. [16]

    Allowed Freq.: L =nλ 2, ν= n 2L √ T µ, n= 1, 2, 3,

    work page

  17. [17]

    Fundamental/1st harmonics: ν0 = 1 2L √ T µ

    work page

  18. [18]

    1st overtone/2nd harmonics: ν1 = 2 2L √ T µ

    work page

  19. [19]

    2nd overtone/3rd harmonics: ν2 = 3 2L √ T µ

    work page

  20. [20]

    String fixed at one end: L N A N A λ/2

    All harmonics are present. String fixed at one end: L N A N A λ/2

    work page

  21. [21]

    Boundary conditions: y = 0 at x = 0

    work page

  22. [22]

    Allowed Freq.: L = (2n + 1)λ 4, ν= 2n+1 4L √ T µ, n = 0, 1, 2,

    work page

  23. [23]

    Fundamental/1st harmonics: ν0 = 1 4L √ T µ

    work page

  24. [24]

    1st overtone/3rd harmonics: ν1 = 3 4L √ T µ

    work page

  25. [25]

    2nd overtone/5th harmonics: ν2 = 5 4L √ T µ

    work page

  26. [26]

    Sonometer: ν∝1 L, ν∝ √ T, ν∝1√µ

    Only odd harmonics are present. Sonometer: ν∝1 L, ν∝ √ T, ν∝1√µ. ν= n 2L √ T µ 2.3: Sound Waves Displacement wave: s =s0 sinω(t−x/v) Pressure wave: p =p0 cosω(t−x/v), p0 = (Bω/v)s0 Speed of sound waves: vliquid = √ B ρ, v solid = √ Y ρ, v gas = √ γP ρ Intensity: I = 2π2B v s02ν2 = p0 2v 2B = p0 2 2ρv Standing longitudinal waves: p1 =p0 sinω(t−x/v), p 2 =p...

    work page

  27. [27]

    Boundary condition: y = 0 at x = 0

    work page

  28. [28]

    Allowed freq.: L = (2n + 1)λ 4, ν= (2n + 1) v 4L, n = 0, 1, 2,

    work page

  29. [29]

    Fundamental/1st harmonics: ν0 = v 4L

    work page

  30. [30]

    1st overtone/3rd harmonics: ν1 = 3ν0 = 3v 4L Figure 5 continued (Page 4). 3 Optics 3.1: Reflection of Light Laws of reflection: normal incident reflectedi r (i) Incident ray, reflected ray, and normal lie in the same plane (ii)∠i =∠r Plane mirror: d d (i) the image and the object are equidistant from mir- ror (ii) virtual image of real object Spherical Mirror...

    work page

  31. [31]

    Mirror equation: 1 v + 1 u = 1 f

    work page

  32. [32]

    Two thin lenses separated by distance d: 1 F = 1 f1 + 1 f2 −d f1f2 f1 f2 d 3.3: Optical Instruments Simple microscope: m =D/f in normal adjustment

    Magnification: m =−v u 3.2: Refraction of Light Refractive index: µ= speed of light in vacuum speed of light in medium = c v Snell’s Law: sini sinr = µ2 µ1 µ1 µ2 incident refracted reflected i r Apparent depth: µ= real depth apparent depth = d d′ O Id d′ Critical angle: θc = sin−1 1 µ θc µ Deviation by a prism: µ δ i i′ A r r′ δ=i +i′−A, general result µ= s...

    work page

  33. [33]

    Magnification in normal adjustment: m = v u D fe

    work page

  34. [34]

    Resolving power: R = 1 ∆d = 2µsinθ λ Astronomical telescope: fo fe

    work page

  35. [35]

    In normal adjustment: m =−fo fe , L =fo +fe

    work page

  36. [36]

    Resolving power: R = 1 ∆θ= 1 1.22λ 3.4: Dispersion Cauchy’s equation:µ=µ0 + A λ2, A> 0 Dispersion by prism with small A and i:

    work page

  37. [37]

    Mean deviation: δy = (µy−1)A

    work page

  38. [38]

    4 Heat and Thermodynamics 4.1: Heat and Temperature Temp

    Angular dispersion: θ= (µv−µr)A Dispersive power: ω= µv−µr µy−1 ≈θ δy (if A and i small) Dispersion without deviation: µ µ′ A A′ (µy−1)A + (µ′ y−1)A′= 0 Deviation without dispersion: (µv−µr)A = (µ′ v−µ′ r)A′ Figure 5 continued (Page 6). 4 Heat and Thermodynamics 4.1: Heat and Temperature Temp. scales: F = 32 + 9 5C, K =C + 273.16 Ideal gas equation: pV =n...

    work page

  39. [39]

    Problem Comprehension:

    work page

  40. [40]

    Does the solution attempt to address the correct objective asked in the question?

    work page

  41. [41]

    Are the correct values, variables, and notations from the question being used?

    work page

  42. [42]

    Concept Application:

    work page

  43. [43]

    Check the solution against the relevant physics concepts and formulas required

    work page

  44. [44]

    Important: Do not verify mathematical reasoning or calculations

    Verify whether the correct physics concepts and formulas are applied. Important: Do not verify mathematical reasoning or calculations. Focus only on whether the correct physics concepts are applied

    work page

  45. [45]

    Calculation Accuracy: Check each step and verify all mathematical calculations, including arithmetic, algebraic manipulation, substitutions, integration, differentiation, fractions, exponents, and numerical approximations. Output format(strictly follow; no extra text): Problem Miscomprehension Flag: [Yes/No] Concept Error Flag: [Yes/No] Calculation Error ...

    work page 2000