{"id":"9f727364-9a59-4965-a9d6-3b2891765b4e","arxiv_id":"2608.03673","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A distillation framework that localizes the first verifiably wrong step in a student's causal reasoning trajectory and repairs only the affected suffix via curriculum-guided reinforcement learning.","lead":"This paper introduces CausalOPD, a training method that makes small language models reason in causal chains by having a knowledge-based teacher locate and fix the student's first wrong reasoning step. The method improved reasoning-path correctness by 23 percentage points across industrial, clinical, and legal tasks, and the small models often beat larger proprietary models on path quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RLWR headline may overstate the reduction in wrong reasoning: unresolved transitions are excluded from the main RLWR metric and receive zero penalty in training, so the 15.7%->4.4% claim is only established if strict RLWR (counting unresolved) is also low.","rationale":"The reader's CONDITIONAL verdict is well aligned with my read: the paper has strong internal evidence, including five baselines, ablations, cross-system transfer, significance tests, and a human audit that recovers the method ordering. My stress-test identifies one additional load-bearing concern that is more specific than the reader's weakest-assumption statement. The reader pointed to the modeling assumption that causal reasoning decomposes into the Evidence/Mechanism/Conclusion order and that the tri-state verifier reliably detects first wrong steps. The paper is actually quite explicit that it targets a narrowed definition of causal reasoning, and Table 2 reports 96.92-98.47% exact first-wrong-step agreement against independently constructed gold chains, so I do not see the stage-ordering assumption as the weakest point. The unresolved-state asymmetry is more concrete and touches the headline RLWR claim directly: unresolved transitions are neither penalized in Eq. (5) nor counted in the main RLWR definition, while the strict variant is only in an absent appendix. The Path metric's conservatism is real evidence that the method is not merely making outputs unverifiable, but the RLWR reduction as stated is not fully supported without the strict number. A single table of strict RLWR and unresolved rates would settle the issue. Because the reader's verdict is already CONDITIONAL and this concern is a condition on accepting the headline RLWR claim, I recommend no change to the verdict, hence UNCHANGED.","tokens_in":12771,"tokens_out":7233,"duration_ms":71879,"concrete_test":"On the held-out test sets, recompute RLWR with the stricter definition: a correct conclusion counts as wrong-reasoning if its trajectory contains at least one confirmed violation OR at least one unresolved transition, using the same gold-chain-conditioned frozen scorer and the method-blinded human audit. Also report the unresolved-transition rate per method and per domain. If CausalOPD's strict RLWR remains close to 4.4% and is substantially below the strict RLWR of trajectory SFT and sequence-level OPD, the concern is resolved. If strict RLWR jumps materially (e.g., above 10% or near baseline levels), the headline 15.7%->4.4% reduction is partly an artifact of verifier abstention, and the central claim should be restated to distinguish reducing violations from reducing wrong reasoning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's headline 'reduces the right-label-wrong-reasoning rate from 15.7% to 4.4%' is one of the two central quantitative claims. Section 4.1 defines RLWR as the fraction of correct conclusions whose trajectory contains at least one confirmed violation; unresolved transitions are not counted. The stricter variant counting unresolved transitions is deferred to the appendix, which was not available in the reviewed material. This matters because the training objective in Eq. (5) assigns R_nv = +1 for verified, -1 for violated, and 0 for unresolved, and Section 3.3 explicitly excludes trajectories with unresolved prefixes from the main localized objective and states that unresolved transitions 'receive no positive credit.' Thus a student can reduce confirmed-violation RLWR by making problematic transitions unresolved, e.g. by omitting a required field, citing unavailable evidence, or using an ambiguous reference, without incurring a training penalty and without being counted as wrong reasoning in the headline metric. The paper's Path metric is conservative because it counts unresolved transitions as failures, and the large Path gain mitigates this concern, but the RLWR claim specifically is not established unless strict RLWR, counting unresolved as wrong reasoning, is reported and remains low. The paper's own acknowledgments, 'the unresolved rate is bounded' in the curriculum scheduler and the existence of a stricter RLWR variant, show this is a live failure mode rather than a hypothetical.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CausalOPD, a curriculum-based online process distillation method for step-dependent causal reasoning. It uses a knowledge-augmented teacher that verifies each student-generated transition as verified, violated, or unresolved, identifies the earliest verifiable violation (the first wrong step), preserves the verified prefix, and applies short-horizon reinforcement learning to the affected suffix. Training proceeds through cold-start SFT on teacher-revised trajectories, correction-state SFT, and a causal-stage curriculum advancing from evidence to mechanism to conclusion. Experiments on industrial AHU fault diagnosis, DDXPlus respiratory diagnosis, and MSLR insider trading compare an 8B Qwen3 student against five baselines and two proprietary LLMs, reporting large gains in strict path correctness (83.16% average vs. 59.72% for sequence-level OPD), a reduction in the right-label-wrong-reasoning (RLWR) rate from 15.7% to 4.4%, and retained cross-system industrial transfer.","tokens_in":13015,"tokens_out":5938,"duration_ms":58649,"significance":"If the reported results hold, CausalOPD is a meaningful advance in process supervision for structured causal reasoning, and the paper is unusually careful in its internal validation: five controlled baselines, component ablations with matched budgets, Holm-corrected pairwise tests, a human audit with 90.8-92.4% agreement and Kendall tau = 1.0, validation of the verifier against independently constructed gold chains, and cross-system industrial evaluation. The tri-state verifier and typed transition representation are a principled response to coverage limitations. The main unresolved risk is that the headline RLWR metric excludes 'unresolved' transitions while the training objective assigns them zero reward, so the claim about reducing wrong reasoning is not established for the strict definition; the Path metric, which counts unresolved transitions as failures, mitigates but does not eliminate this concern.","major_comments":[{"comment":"The headline RLWR reduction (from 15.7% to 4.4%) is not established for the stated quantity 'wrong reasoning'. Section 4.1 defines RLWR using only confirmed violations, with a stricter variant counting unresolved transitions deferred to the appendix. Meanwhile Eq. (5) gives unresolved transitions zero reward (R_nv = 0) and Eq. (3) excludes trajectories with unresolved prefixes from the localized objective. A student can therefore lower confirmed-violation RLWR by making problematic transitions unresolved (for example, by omitting a required field, citing ambiguous evidence, or exploiting a coverage gap) without incurring a training penalty and without the failure being counted. The Path metric is conservative and the large Path gains mitigate this concern, but they do not by themselves validate the specific RLWR claim. The authors must report the strict RLWR (counting unresolved transitions as wrong reasoning) for the final model and for all baselines, and should discuss why the zero-reward treatment of unresolved transitions does not encourage this form of evasion.","section":"Abstract, §4.1, Eq. (5), §3.3"},{"comment":"The framework's central premise is that valid causal reasoning trajectories obey the monotonic stage ordering Evidence ≺ Mechanism ≺ Conclusion, with g(h) ⪯ g(h+1), and the curriculum in Section 3.4 depends on this ordering. The main text does not provide per-domain evidence that the gold chains or the accepted student trajectories actually respect this ordering, nor does it report the trajectory of the unresolved rate over training rounds, even though the curriculum scheduler uses a bounded unresolved rate as a condition for advancement. Since the reverse-order ablation (Section 4.4) shows the ordering matters, the authors should report, per domain, the fraction of gold and accepted trajectories satisfying the monotonicity assumption and the unresolved-rate trend; otherwise the central premise is asserted rather than empirically demonstrated.","section":"§3.1 and §3.4"}],"minor_comments":[{"comment":"Figure 1 contains many abbreviations and symbols (M2, E1, Q1-Q3, h*, c*, rho*, g(h*)) that are not fully explained in the caption; a legend or expanded caption would make the pipeline much easier to follow.","section":"Figure 1"},{"comment":"The curriculum advancement thresholds, the rehearsal decay ratio, and the unresolved-rate bound are all deferred to the technical appendix; stating at least the concrete threshold values used in the experiments would make the scheduler reproducible from the main text.","section":"§3.4"},{"comment":"The sentence describing the industrial test set ('The industrial test set (3,942 cases) adds two cross-system sets...') is ambiguous about whether the 3,942 cases already include the cross-system sets or are additional to them; please clarify the exact composition.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is close: the method, baselines, ablations, and human audit are strong, and the central Path-correctness claim is well supported. The blocking issue is the RLWR metric: as defined and rewarded, unresolved transitions can mask violations, so the headline 15.7% to 4.4% reduction is not yet a claim about wrong reasoning in the strict sense. If the appendix provides strict RLWR numbers that remain low and explains the zero-reward treatment of unresolved transitions, I would be willing to support acceptance. Please ensure the technical appendix is included in the review version, since several load-bearing details are currently only referenced."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jian, here's my read of CausalOPD.\n\nThe genuinely new piece is the first-wrong-step boundary defined by a knowledge-grounded tri-state verifier (verified/violated/unresolved) applied to typed causal transitions, paired with a curriculum that goes evidence->mechanism->conclusion and only optimizes the suffix after the verified prefix. That combination is not present in the OPD, PRM, GLoRe, R3, or KARD lines—they either use learned process scores or outcome-level feedback. The paper also does the empirical work right: five baselines, ablations that isolate each component, fresh-rollout versus stale-pool comparisons, a weak-teacher replacement study, cross-system transfer, and a human audit that reproduces the method ordering with Kendall tau 1.0. The 23.4-point Path gain over sequence-level OPD and the 83.16% average Path are impressive if they hold up.\n\nThe internal evidence is strong enough that I'd trust the qualitative conclusion: localizing the first verifiable violation and repairing the suffix works better than full-trajectory or sequence-level supervision. The ablation showing random-step localization recovers only 41% of the gain is a good sign that it's the boundary, not just shorter optimization, that matters.\n\nNow the soft spots, in proportion. The RLWR claim in the abstract is real but not fully established as stated. The metric counts only confirmed violations; unresolved transitions are excluded, and training gives them zero reward. So the 15.7%->4.4% drop could be partly achieved by pushing problematic transitions into 'unresolved' rather than fixing them. The Path metric is conservative and counts unresolved as failures, so the large Path gains mitigate this substantially—but the headline claim specifically needs the strict RLWR variant, which is deferred to an appendix we don't have. That's a reportable gap, not a fatal flaw.\n\nThe other soft spot is verification: no code/data, the teacher is proprietary, and thresholds/reward coefficients/pseudocode are in the appendix. The weak-teacher replacement helps, and the gold-chain audit partly anchors the evaluation, but exact reproduction is currently impossible.\n\nThe stage ordering assumption (Evidence<Mechanism<Conclusion with monotonic progression) is a real modeling commitment. If real reasoning doesn't respect that order, both the boundary and the curriculum rest on a false premise. The paper acknowledges this by scoping to step-dependent causal attribution and noting future work on less structured domains.\n\nBottom line: this is a serious empirical paper with a novel, plausible mechanism and unusually thorough internal controls. It deserves a full peer review. The referee should require the strict RLWR numbers and the appendix details before sign-off. I'd be comfortable citing it after those are public.\n\nRecommendation: engage with it—send to a competent reviewer focusing on the RLWR metric's vulnerability to the unresolved class and the verifier's independence from the training reward.","headline":"A well-engineered distillation recipe for causal chain reasoning with strong internal evidence; the RLWR headline needs the strict variant reported, but the Path gains and human audit carry the paper.","tokens_in":13608,"tokens_out":2496,"would_cite":false,"duration_ms":21112,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that distilling causal reasoning should supervise the student at its first verifiably wrong step, and reports a 23.4-point path-correctness gain over sequence-level on-policy distillation with the…","keywords":["causal chain reasoning","knowledge distillation","on-policy distillation","process supervision","first-wrong-step localization","curriculum learning","reinforcement learning","reasoning path correctness"],"falsifier":"Run CausalOPD on a new benchmark whose independently written reference reasoning chains contain a substantial fraction of cases in which the earliest wrong step occurs before the evidence stage is complete, so the fixed Evidence-to-Mechanism-to-Conclusion curriculum is mis-ordered for those cases; if the path-correctness advantage over sequence-level OPD survives such mis-ordering, the claimed mechanism is not the causal-stage curriculum, and the gain would have to come from localization or verifier coverage instead.","tokens_in":12505,"feed_emoji":"🧠","tokens_out":10345,"duration_ms":82491,"temperature":0.7,"pith_summary":"The paper argues that when causal reasoning is distilled from a large teacher into a small student, the right place to supervise is not the final answer or the whole chain but the earliest transition that can be shown to contradict domain knowledge. It presents CausalOPD, in which a knowledge-augmented teacher verifies the student's own rollouts, finds that first wrong step, and uses short-horizon reinforcement learning to repair only the suffix from that step onward, while a curriculum schedules repairs from evidence-level to mechanism-level to conclusion-level errors. The authors report that this raises average path correctness by 23.4 percentage points over sequence-level on-policy distillation, lowers the right-label-wrong-reasoning rate from 15.7% to 4.4%, and lets 8B students beat both evaluated proprietary models on path correctness in all three domains.","feed_headline":"Fixing the first wrong causal step lifts path correctness 23.4 points","feed_subtitle":"A knowledge-grounded curriculum trains 8B students that beat larger proprietary models on reasoning paths.","key_machinery":"The load-bearing object is the first-wrong-step state $s^* = ((x, \\mu_x), z_{<h^*}, c^*, \\rho^*, g(h^*))$, which packages the verified prefix, the violated constraint, the implicated entities, and the causal stage. It is produced by a tri-state evaluator $V_K$ that returns verified, violated, or unresolved for each typed transition $z_h = (g_h, e_h, r_h, d_h)$, where $e_h$ is the cited evidence, $r_h$ is the invoked rule or relation, and $d_h$ is the resulting decision. Typed transitions make constraints inspectable, and the unresolved state prevents insufficient knowledge from becoming false supervision. The same boundary drives the curriculum: when the incidence of first wrong steps at the active stage falls below a threshold, training advances from evidence to mechanism to conclusion, following the order in which errors propagate.","core_discovery":"The paper's central claim is that a single knowledge-verified boundary, the first wrong step, can coordinate all three decisions that process-level distillation leaves open: what to repair, where optimization begins, and when a correction is trained. Concretely, the verifier $V_K(z_h \\mid x, z_{<h}, \\mu_x)$ labels each transition as verified, violated, or unresolved against explicit causal rules, entity relations, and structural constraints; $h^* = \\min\\{h : V_K(z_h \\mid x, z_{<h}, \\mu_x) = \\text{violated}\\}$ is the first wrong step. The violated constraint $c^*$ says what is wrong, its position $h^*$ sets the optimization horizon, and its stage $g(h^*)$, drawn from Evidence, Mechanism, or Conclusion, schedules the curriculum. The paper claims this localization, together with verified-prefix retention and fresh on-policy rollouts, is what converts process rewards into targeted repair: the ablations attribute the largest losses to removing first-wrong-step localization, teacher revision, and the causal stage ordering.","pith_inferences":["The authors do not claim this, but the same first-violation curriculum could be probed outside LLM distillation: any sequential diagnosis or repair process with causally ordered stages should show a similar benefit if the boundary is placed at the earliest verifiable violation.","A further implication left implicit is that the gain should degrade as verifier coverage drops; a direct experiment would deliberately remove constraint templates and check whether the path-correctness advantage tracks verifier coverage rather than the curriculum stage itself.","The paper evaluates on domains where typed causal stages are supplied, so an untested extension is whether the method can infer stages from data and still retain most of the gain, which would determine how far it reaches beyond curated knowledge bases."],"forward_implications":["If CausalOPD works as reported, process-level distillation of causal reasoning should be evaluated by path correctness rather than conclusion accuracy alone, because correct conclusions reached through invalid steps are detected and repaired.","Short-horizon optimization from the first wrong step should replace full-trajectory reinforcement learning for step-dependent causal tasks, since full-trajectory process RL recovered only 38.8% of the path-correctness gain.","A causal-stage curriculum that repairs evidence errors before mechanism and conclusion errors should yield better final competence than stage-agnostic or reverse-order scheduling, since reversing the order cost 16.4 percentage points in the reported comparisons.","On-policy distillation should rebuild correction pools from fresh rollouts every round rather than reusing a stale pool, since the budget-matched stale pool lagged by 9.3 percentage points and plateaued after round three.","Domain-specific 8B students can surpass larger proprietary models on reasoning-path correctness in industrial, clinical, and legal settings, supporting local deployment under privacy or latency constraints."],"supporting_citations":[{"why":"Foundational on-policy distillation method that CausalOPD extends and that the sequence-level OPD baseline instantiates.","marker":"Agarwal et al. 2024"},{"why":"MiniLLM, the on-policy distillation baseline the paper compares against.","marker":"Gu et al. 2024"},{"why":"Establishes process- versus outcome-based feedback for step-level credit assignment, the supervision family CausalOPD refines.","marker":"Uesato et al. 2022"},{"why":"Provides step-by-step verification ('Let's Verify Step by Step') that supports the process-supervision premise.","marker":"Lightman et al. 2024"},{"why":"Math-Shepherd supplies continuation-based process reward estimation, a PRM baseline that does not use explicit knowledge constraints.","marker":"Wang et al. 2024"},{"why":"GLoRe is the closest localization baseline: it conditions refinement on a predicted first-error location, which CausalOPD replaces with verifier-grounded localization.","marker":"Havrilla et al. 2024"},{"why":"R3 starts reinforcement learning from states along correct demonstrations, another localization-adjacent method that CausalOPD differs from by verifying the student's own transitions.","marker":"Xi et al. 2024"},{"why":"Supplies the industrial air-handling-unit fault diagnosis cases used for training and cross-system testing.","marker":"Granderson et al. 2022"},{"why":"Supplies the clinical diagnostic reasoning benchmark used for training and testing.","marker":"Fansi Tchango et al. 2022"},{"why":"Supplies the legal judgment reasoning benchmark used for training and testing.","marker":"Yu et al. 2025"}],"fun_headline_variants":["First wrong step is the fix: CausalOPD lifts path correctness by 23.4 pts","Target first wrong step to cut wrong-reasoning from 15.7% to 4.4%","CausalOPD: pinpoint the first wrong step to lift reasoning paths by 23.4 pts","Fix the first wrong causal step, not the whole chain: 23.4 pt boost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method stands on the assumption that the target reasoning really does proceed in the fixed order Evidence, then Mechanism, then Conclusion, and that the available domain knowledge is complete enough to mark the earliest step that breaks that order; if a real case violates that order or the knowledge misses a step, the first-wrong-step boundary and the curriculum rest on a false premise.","fun_headline_variants_meta":{"raw":{"variants":["First wrong step is the fix: CausalOPD lifts path correctness by 23.4 pts","Target first wrong step to cut wrong-reasoning from 15.7% to 4.4%","CausalOPD: pinpoint the first wrong step to lift reasoning paths by 23.4 pts","Fix the first wrong causal step, not the whole chain: 23.4 pt boost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000765,"raw_usage":{"total_tokens":3419,"prompt_tokens":996,"completion_tokens":2423,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":2320}},"tokens_in":612,"tokens_out":2423,"duration_ms":15340,"temperature":1.0,"reasoning_tokens":2320,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:47:42.771421+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CausalOPD on a new benchmark whose independently written reference reasoning chains contain a substantial fraction of cases in which the earliest wrong step occurs before the evidence stage is complete, so the fixed Evidence-to-Mechanism-to-Conclusion curriculum is mis-ordered for those cases; if the path-correctness advantage over sequence-level OPD survives such mis-ordering, the claimed mechanism is not the causal-stage curriculum, and the gain would have to come from localization or verifier coverage instead.","supporting_citations":[{"cited_title":"C.; Nalmpantis, C.; Dwivedi-Yu, J.; Zhuravinskyi, M.; Hambro, E.; and Raileanu, R","cited_arxiv_id":null,"evidence_quote":"GLoRe is the closest localization baseline: it conditions refinement on a predicted first-error location, which CausalOPD replaces with verifier-grounded localization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the clinical diagnostic reasoning benchmark used for training and testing."}],"review_version":2}