{"id":"9d8d395e-94b7-4726-98aa-6a2fb5aa8f23","arxiv_id":"2607.16955","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A coverage-adaptive on-policy distillation framework with dense partial-credit rewards raises a 0.5B student's GSM8K pass@1 from 48.7% to 69.8% (1.5B teacher) and to 72.1% (3B teacher).","lead":"CADENCE distills a 0.5B math model from a 1.5B or 3B teacher, improving GSM8K accuracy from a 48.7% pretrained baseline to 69.8-72.1% using on-policy distillation with adaptive KL mixing and dense partial-credit rewards. It claims to beat matched label-using baselines by about 4 points, all on a single Mac Studio.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"192-token GSM8K cap may inflate CADENCE's +4.4 margin; LAP's brevity bias could exploit truncation that §4.3 never isolates.","rationale":"The central claim rests on empirical numbers, not theory; the theory itself is honestly scoped (per-token surrogates, Proposition 1). The most fragile link is the evaluation harness, because it was modified after observing under-performance, and the modification is defended by reproducing teacher scores, not by establishing that student scores are unbiased. The reader's concern about LAP interacting with the harness is on point, but I make it concrete: §4.3 retains a 192-token cap for GSM8K with the assertion 'where sufficient,' yet the paper's own data shows the teacher improved 74.5→82.1 when moving to the corrected protocol, confounding extractor and truncation effects. LAP explicitly trades off length for reward, producing a 10-token median reduction vs DRIFT+binary; if the 192-token cap truncates any non-trivial fraction of longer correct responses, the pass@1 gap is distorted. The 'genuine compression' analysis in §5.4 checks a subset (<180 tokens) but does not measure truncation frequency, so it does not answer this. The proposed test directly isolates the cap effect by holding the extractor fixed and expanding the cap; if the margin holds, the concern is resolved. I therefore keep the reader's CONDITIONAL verdict (no change), adding this specific verification as a condition.","tokens_in":13887,"tokens_out":6257,"duration_ms":55800,"concrete_test":"Re-run the full Experiment 1 evaluation (all methods in Table 1) on GSM8K with a 512-token generation cap, using the same Qwen2.5-Math–matched answer extractor, and compare each method's pass@1 against the reported 192-token numbers. Report the proportion of truncated responses per method and the change in the CADENCE vs DRIFT+binary margin. If the margin shrinks by more than ~1 point or any method moves by >1 point, the headline numbers are confounded with truncation. Also run the same check on the teacher to verify the 'sufficient' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline margin (+4.4±0.7 over DRIFT+binary) depends on the corrected evaluation protocol in §4.3, which keeps a 192-token generation cap on GSM8K. The paper's own before/after numbers conflate two protocol changes: under the original 192-token cap plus regex extractor, the 1.5B teacher scored 74.5% on GSM8K; under the corrected protocol (512-token MATH-500 cap, 192-token GSM8K cap, matched extractor) it scores 82.1%. The improvement is attributed to the extractor and to truncation on MATH-500, but no experiment isolates whether the 192-token cap truncates a material fraction of GSM8K responses. This matters because CADENCE's LAP component explicitly rewards brevity: Table 3 shows CADENCE's correct-rollout median length is 115 tokens, vs 125 for DRIFT+binary and 148 for pretrained. If some correct responses in longer-generating methods are cut off at 192 tokens, pass@1 for those methods is suppressed, while CADENCE's shorter responses evade truncation. That would inflate the claimed +4.4 margin and the 'teacher gap closed' numbers. The paper's RLD 'genuine compression' analysis (untruncated <180 subset) does not test the truncation rate under the 192-token cap, so the 'sufficient' claim for GSM8K is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CADENCE, a framework for on-policy knowledge distillation that combines a per-token KL surrogate mixture (DRIFT) with six additional components (COVA, FTB, CCD, LAP, EMR, BSD) and two stabilizers (TFW, KTR). The central empirical claim is that CADENCE distills a 0.5B student from a 1.5B teacher to 69.8 ± 0.5% GSM8K pass@1 (closing 63.2% of the teacher–student gap) and to 72.1 ± 0.4% with a 3B teacher (76.2% closed), outperforming the strongest matched-compute label-using baseline (DRIFT+binary reward) by +4.4 ± 0.7 points on GSM8K and +3.4 ± 0.8 on MATH-500. The paper also presents diagnostic metrics, an ablation study, and honest scope limitations, and it acknowledges an earlier evaluation-harness issue that was corrected in the present version.","tokens_in":14340,"tokens_out":6754,"duration_ms":68859,"significance":"If the empirical results hold, CADENCE would be a meaningful advance in making on-policy distillation of reasoning abilities practical on modest hardware, with small-model math reasoning gains that are nontrivial relative to the strongest baseline. The paper's reporting practices are genuinely strong in several respects: five seeds with reported standard deviations, hyperparameters selected on a held-out validation split, honest statements about the surrogate nature of DRIFT (Remark 1), an explicit description of the evaluation-harness change (§4.3), and an ablation that deliberately discusses component interactions rather than claiming independence. These transparency features are commendable and should be credited. The main risk to the central claim is an evaluation-protocol interaction that may inflate the headline margin, as detailed in the major comments.","major_comments":[{"comment":"The corrected evaluation protocol retains a 192-token generation cap for GSM8K, but the paper never quantifies how often responses are truncated under this cap for any method. The analysis in §4.3 attributes the earlier teacher under-performance to MATH-500 truncation and to the answer extractor; GSM8K is declared 'where sufficient' without supporting evidence. This is load-bearing because LAP (Eq. 15) explicitly rewards shorter correct rollouts, and Table 3 reports RLD medians of 115 tokens for CADENCE versus 125 for DRIFT+binary and 148 for the pretrained student. If a material fraction of correct responses from longer-generating methods are cut at 192 tokens, their pass@1 is suppressed while CADENCE's shorter responses evade truncation, inflating the headline +4.4 ± 0.7 margin and the gap-closure percentages. The RLD 'genuine compression' analysis in §5.4 (untruncated <180 subset) doe","section":"§4.3, Eq. (15), Table 3"},{"comment":"Baseline tuning is not documented. §4.5 lists STaR/RFT, GKD+GRPO, DRIFT+binary, and the label-free baselines, but gives no hyperparameters, optimization schedules, or validation-selection procedure for these methods. 'Matched-compute' is defined only through n_g=4 rollouts; if the baselines were not tuned to the same degree as CADENCE (e.g., same validation split, same number of steps, same LoRA configuration), the reported +4.4-point margin over DRIFT+binary could be overstated. Please provide, for every baseline, the learning rate, LoRA rank, number of training steps, sampling temperature, and the validation-selection process, and confirm that the same validation split as used for CADENCE hyperparameters was used for baseline selection. In particular, the binary-reward variant of DRIFT should be tuned to the same precision as CADENCE's components.","section":"§4.5, Table 1"},{"comment":"The unbiasedness claim in Proposition 1 is not established by the 'substitute; direct' proof. At β=0, Eq. (5) uses per-token advantages A_t = -k_t with a leave-one-out baseline (Eq. 8) taken over positions t' ≠ t. Because future tokens depend on the current token through the autoregressive chain, the baseline terms A_{t'} for t'>t are generally correlated with x_t, so the baseline is not independent of the current action. Moreover, the simple per-token score function A_t ∇ log πθ(x_t) omits the cross-terms that appear in ∇ E[f_t] = E[f_t Σ_{t'≤t} ∇ log πθ(x_{t'})] and the cumulant terms from later positions. Thus the claim that Eq. (5) 'yields the REINFORCE Monte-Carlo estimator of ∇J_rev' is stronger than the derivation supports. Either provide a correct derivation with explicit assumptions (e.g., reward-to-go formulation, or a baseline independent of the prefix), or restate the claim a","section":"Proposition 1, Eq. (8)"}],"minor_comments":[{"comment":"The phrase '192 retained for GSM8K, where sufficient' is an unsupported assertion. Please either cite the truncation statistics or soften the wording to reflect that adequacy is assumed rather than demonstrated.","section":"§4.3"},{"comment":"H_ref is described as 'fixed' but is 'chosen based on measured per-token teacher entropy distribution during initial training runs.' This is a validation-selected hyperparameter and should be described as such for consistency with the paper's claim that all hyperparameters were selected on the validation split.","section":"§3.3.2"},{"comment":"The caption mentions 'Shaded rows' for the six novel components, but the shading is not visible in the provided text. Please make the row grouping explicit in a column or with bold/italic formatting.","section":"Table 4"},{"comment":"The figure caption confusingly juxtaposes the 'original 192-token protocol' headline result with the corrected numbers. It would be clearer to provide a side-by-side table of the same methods under both protocols, including the corrected teacher reference, so the reader can directly see the protocol's impact on student and baseline scores.","section":"Figure 3"},{"comment":"The paper interchangeably uses s_t and x_{1:t} for the conditioning history. Please unify notation to avoid ambiguity in equations (2), (3), and (9).","section":"Notation, §3.2"}],"recommendation":"major_revision","confidential_remarks":"The authors are unusually transparent about the evaluation-harness change, which mitigates but does not eliminate the reproducibility risk introduced by changing the protocol after observing teacher under-performance. The lack of code release is a further limitation; I would encourage the editor to request code and raw truncation/length statistics as part of the revision. The paper fits the journal's scope and the central idea is interesting, but the two load-bearing concerns (GSM8K truncation and baseline tuning) must be addressed before the empirical claims can be fully trusted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real method paper, not a toy. The DRIFT mixture of per-token forward/reverse KL surrogates with coverage-adaptive scheduling, plus the six components (COVA, FTB, CCD, LAP, EMR, BSD), is a genuinely new combination. Existing work uses fixed interpolation, reverse/forward KL, or entropy-weighted tokens; the coverage-adaptive gate and the numerical-proximity partial credit are not in the cited prior work. The authors are unusually honest about what their theory does and does not claim. Proposition 1 is explicitly a per-token surrogate, not sequence-level KL, and the discussion lists the limits: COVA is asymmetric, CCD's partial credit is modest, and no teacher-size scaling claim is made. That honesty is a point in their favor.\n\nThe empirical reporting is better than average: five seeds, standard deviations, validation-selected hyperparameters, and a clear account of the corrected evaluation protocol in §4.3. They explain why the earlier 192-token cap and regex extractor under-measured teachers, and they reproduce published teacher scores under the corrected protocol. That is good practice.\n\nBut the central numeric claim—+4.4 over DRIFT+binary, 63-76% gap closure—has a soft spot the paper does not fully close. The correction kept the 192-token cap on GSM8K, calling it 'sufficient,' but no experiment isolates how many correct responses actually get truncated at 192 tokens. The stress-test concern is legitimate: LAP explicitly rewards brevity, and CADENCE's correct rollouts are shorter (median 115 vs 125 for DRIFT+binary, 148 for pretrained). If a material fraction of longer correct responses from baselines are cut off, their pass@1 is suppressed and CADENCE's margin is inflated. The RLD 'genuine compression' analysis only looks at responses under 180 tokens; it does not measure truncation rate at 192. So the 'sufficient' claim for GSM8K is unsupported. This is not a deal-breaker for the framework, but the headline numbers should be treated as conditional until this is measured.\n\nTwo more concerns, minor in comparison: baseline tuning is not documented, and no code or data is released. Without those, the +4.4 margin can't be independently verified.\n\nThis paper is for people working on on-policy distillation or efficient reasoning models. The components are individually incremental but the integration is thoughtful, and the diagnostics (SAG, FTA, KLPE, CNI, RLD) are useful even if their definitions are a bit ad hoc. It deserves a serious referee. I would send it to peer review, but ask for code/data and a direct measurement of GSM8K truncation rates before accepting the headline numbers at face value.","headline":"A transparent, well-structured distillation paper with a plausible framework, but the headline margin rests on an evaluation protocol change and an untested GSM8K generation cap.","tokens_in":14782,"tokens_out":4611,"would_cite":true,"duration_ms":44821,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 0.5B student reaches 69.8% GSM8K by fixing three on-policy distillation failure modes.","keywords":["on-policy distillation","knowledge distillation","mathematical reasoning","KL divergence","reward sparsity","small language models","calibration","GSM8K"],"falsifier":"Re-run the exact CADENCE training with the corrected protocol but switch the reward's numerical proximity to a binary correctness gate (or vice versa), and compare pass@1; if the +4.4 margin over the binary baseline vanishes or reverses, the partial-credit mechanism is not the driver. Alternatively, evaluate with a different answer extractor or a longer generation cap on MATH-500; if the student's advantage shrinks materially, the result is protocol-bound.","tokens_in":13792,"feed_emoji":"🧮","tokens_out":4160,"duration_ms":40122,"temperature":0.7,"pith_summary":"This paper argues that on-policy distillation of mathematical reasoning into a small student fails for three specific, addressable reasons: a cold start in which the student assigns near-zero probability to the teacher's preferred tokens, a divergence schedule that ignores whether the student has actually learned a prompt, and a binary reward that throws away information from partially correct attempts. To fix these, it introduces CADENCE, which schedules a per-token mixture of forward and reverse KL surrogates, adapts the schedule to measured coverage, and adds dense partial credit for numerically close answers. On GSM8K, CADENCE lifts a 0.5B student from 48.7% to 69.8% with a 1.5B teacher (63.2% of the teacher gap) and to 72.1% with a 3B teacher (76.2% gap), beating the strongest matched-compute baseline by +4.4 points. The paper also reports improved calibration, shorter correct responses, and diagnostics showing the components interact positively. A sympathetic reader would care because the result suggests strong reasoning can be distilled into small models on modest hardware without sacrificing transparency.","feed_headline":"Tiny 0.5B student reaches 69.8% GSM8K via coverage-adaptive distillation","feed_subtitle":"New on-policy distillation adds partial credit for near-miss answers and adapts KL scheduling to student coverage—closing two-thirds of the","key_machinery":"The central object is the DRIFT per-token advantage: at each generated token, a self-normalized importance-weighted forward-KL surrogate is mixed with a reverse-KL log-ratio surrogate by a continuously annealed beta, with a coverage-adaptive gate (COVA). This surrogate advantage is used in a policy-gradient update, extended by a dense reward (CCD) that gives partial credit for numerically close incorrect answers, an entropy boost (FTB), a brevity term (LAP), an entropy-matching regularizer (EMR), a bootstrapped self-distillation phase (BSD), plus teacher-forced warmup and a KL trust region.","core_discovery":"The central claim is that the three failure modes can be fixed simultaneously by a unified objective: DRIFT, a per-token convex mixture of forward-KL and reverse-KL surrogates evaluated on student-sampled trajectories, extended with six components. In particular, COVA modulates the beta schedule by measured student coverage over teacher-preferred tokens; CCD adds numerical-proximity partial credit so incorrect-but-close traces receive nonzero reward; FTB concentrates gradient at high-entropy forking tokens; LAP biases toward brevity; EMR matches student-teacher entropy at forking tokens for calibration; and BSD adds a bootstrapped self-distillation phase. The paper demonstrates that these do","pith_inferences":["If the partial-credit mechanism generalizes, similar numerical-proximity rewards could be applied to code execution outputs or other verifiable domains.","The coverage-adaptive schedule suggests that time-only schedules are systematically suboptimal; a testable extension is per-prompt or per-cluster beta adaptation.","Since all experiments used LoRA on a 0.5B base, the framework's benefit for full fine-tuning or larger students is untested; a natural follow-up is scaling the student.","The evaluation-harness correction (512-token cap, matched answer extractor) is itself a caution for the field: small-model distillation results are sensitive to evaluation protocol."],"forward_implications":["Small (0.5B) models can reach within a few points of a 1.5B math-tuned teacher on GSM8K (69.8 vs 82.1) using only a single desktop machine.","Dense partial-credit rewards for near-miss answers materially improve distillation over binary rewards (+4.4 points over the strongest binary baseline).","The student's calibration improves substantially (ECE from 0.164 to 0.078) without directly optimizing calibration on the test set.","Response lengths shrink by about 26 tokens on correct rollouts, indicating the method compresses reasoning traces rather than merely learning to be verbose.","The component ablation shows superadditive interactions: removing any single component costs accuracy, but removing all six costs only 6.7 points, implying the components reinforce each other."],"fun_headline_variants":["CADENCE: Partial credit for near-answers boosts distilled model to 69.8%","On-policy KL mix with coverage scheduler lifts 0.5B to 69.8% GSM8K","Near-miss rewards in CADENCE push 0.5B student to 69.8% GSM8K","Coverage-adaptive distillation: 0.5B model closes 63% of teacher gap","Tiny student gets partial credit: 0.5B model hits 69.8% GSM8K"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"All headline numbers depend on the corrected evaluation protocol (512-token cap on MATH-500 and a matched answer extractor); if this protocol interacts with the partial-credit or brevity rewards in a way that inflates the student's measured score, the claimed gap-closure margins would shrink.","fun_headline_variants_meta":{"raw":{"variants":["CADENCE: Partial credit for near-answers boosts distilled model to 69.8%","On-policy KL mix with coverage scheduler lifts 0.5B to 69.8% GSM8K","Near-miss rewards in CADENCE push 0.5B student to 69.8% GSM8K","Coverage-adaptive distillation: 0.5B model closes 63% of teacher gap","Tiny student gets partial credit: 0.5B model hits 69.8% GSM8K"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001973,"raw_usage":{"total_tokens":7653,"prompt_tokens":964,"completion_tokens":6689,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":708,"completion_tokens_details":{"reasoning_tokens":6568}},"tokens_in":708,"tokens_out":6689,"duration_ms":49138,"temperature":1.0,"reasoning_tokens":6568,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T19:25:40.600095+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the exact CADENCE training with the corrected protocol but switch the reward's numerical proximity to a binary correctness gate (or vice versa), and compare pass@1; if the +4.4 margin over the binary baseline vanishes or reverses, the partial-credit mechanism is not the driver. Alternatively, evaluate with a different answer extractor or a longer generation cap on MATH-500; if the student's advantage shrinks materially, the result is protocol-bound.","supporting_citations":[],"review_version":1}