REVIEW 3 major objections 4 minor 54 references
SR-OPSD: Self-Referenced On-Policy Self-Distillation
T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A token-level identity turns on-policy self-distillation into a Rényi projection from a reference-anchored self-teacher target, and the resulting objective stabilizes training and beats KL, JSD, and GRPO baselines on science, math, and…
desk verdict A plausible self-distillation recipe that pairs a frozen-reference anchor with Rényi projection, backed by honest algebra and an honest ablation, but whose SOTA claim rests on single-seed runs and benchmark-tuned hyperparameters. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the normalized geometric target $\pi^\star_\alpha(\cdot|h^{\mathrm{sf}},h^{\mathrm{ref}}) \propto \mathrm{sg}[\pi_{\bar\theta}(\cdot|h^{\mathrm{sf}})]^\alpha \pi_{\mathrm{ref}}(\cdot|h^{\mathrm{ref}})^{1-\alpha}$, a log-linear interpolation between the EMA self-teacher and a frozen reference policy. The projection machinery is the Rényi divergence $D_\rho(p\|q) = \frac{1}{\rho-1}\log \sum_a p(a)^\rho q(a)^{1-\rho}$, which converges to KL divergence as $\rho\to 1$. Proposition 3.2 shows that SR-OPSD's token objective equals $-D_\rho(\pi^\star_\alpha \| \pi_\theta)$ minus a constant, and Proposition 3.3 turns this into an implementable logit gradient whose update is the student distribution minus a power-tempered reweighted distribution; $\alpha$ sets where the target sits and $\rho$ sets how the student is projected toward it.
What would settle it
Record the student's token distribution and logits at sampled prefixes during an SR-OPSD run and compare the measured logit changes with the identity $\partial D_\rho/\partial z_\theta(a|h) = \pi_\theta(a|h) - \widetilde{\pi}_{\alpha,\rho,\theta}(a|h,h^{\mathrm{sf}},h^{\mathrm{ref}})$; a discrepancy that grows with context-distribution shift would show the coupled dynamics are not governed by the fixed-context characterization.
Extended reading notes
Core claim
At any fixed student-generated prefix, the SR-OPSD token objective is exactly the forward Rényi divergence from a normalized geometric interpolation between the self-teacher and a reference policy to the student, up to a log-normalizer constant that is fixed in the inner update. The target is $\pi^\star_\alpha(a) \propto \mathrm{sg}[\pi_{\bar\theta}(a|h^{\mathrm{sf}})]^\alpha \pi_{\mathrm{ref}}(a|h^{\mathrm{ref}})^{1-\alpha}$, so $\alpha$ places the target on a log-linear path from the reference policy at $\alpha=0$ to the self-teacher at $\alpha=1$. The Rényi order $\rho$ controls the projection geometry: the logit gradient is shown to be $\pi_\theta(a|h) - \widetilde{\pi}_{\alpha,\rho,\theta}(a|h,h^{\mathrm{sf}},h^{\mathrm{ref}})$, where the second term reweights the student distribution by the target-to-student density ratio raised to the power $\rho$, thereby tempering extreme ratios. The paper then reports that this anchored Rényi objective improves on or matches GRPO and the KL/JSD self-distillation baselines across SciKnowEval, five mathematics benchmarks, and LiveCodeBench v6, while also keeping policy entropy more stable during long training runs.
Load-bearing premise
The load-bearing premise is that the pointwise, fixed-context equivalence between SR-OPSD and a Rényi projection continues to describe the actual training loop, where the rollout context distribution and the self-teacher both evolve; if the coupled dynamics escape that pointwise analysis, the theoretical justification no longer applies.
Editorial extensions
If this is right
- With $\alpha=1$ the target is the self-teacher and with $\alpha=0$ it is the reference, so the method contains standard self-distillation and pure reference matching as endpoints of one family.
- As $\rho\to 1$ the Rényi projection recovers the KL limit, making reverse-KL on-policy self-distillation a boundary case of the same objective.
- The gradient identity shows $\rho$ acts as a power temper on the target-to-student density ratio: lower $\rho$ suppresses extreme token probabilities, which is the mechanism behind the claimed training stability.
- In the mathematics benchmark, SR-OPSD's mean Pass@64 across the five benchmarks is 78.4 versus 75.2 for GRPO, and it exceeds the forward-KL OPSD baseline by 11.5 points and the reverse-KL SDPO baseline by 8.0 points in mean Pass accuracy.
- The pointwise equivalence holds only for the frozen inner update; the global coupled objective is not a single Rényi minimization because the normalizer depends on the changing on-policy context distribution.
Reading between the lines
- The where/how separation suggests a design principle that likely transfers beyond self-distillation: any moving target distribution, such as a reward-scaled policy or an iteratively refined preference model, could be anchored to a frozen reference and approached with a tunable divergence to prevent drift.
- Because $\alpha$ and $\rho$ enter the gradient only through the product $\alpha\rho$, the two knobs are not independent in practice; a testable prediction is that performance depends mainly on $\alpha\rho$, producing a ridge rather than two separate effects in the ablation landscape.
- A natural extension the paper leaves open is adaptive $\rho$: raising the order when the self-teacher is reliable and lowering it when density ratios become extreme could improve on the fixed $\rho=0.95$ used in the experiments.
- The fixed-context caveat implies the strongest gains should appear when context shift is modest, so a stress test with rapidly shifting prompt distributions or much longer rollouts could map where the reference anchor stops helping.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SR-OPSD, a self-referenced on-policy self-distillation method for LLM post-training. The method constructs a token-level target as a normalized geometric interpolation between an EMA self-teacher and a frozen reference policy, and projects the student toward this target using a Rényi divergence of order ρ. The main theoretical results (Propositions 3.1–3.3) are token-level variational characterizations for fixed contexts and frozen targets: maximizing the proposed objective is equivalent to minimizing a forward Rényi divergence to the interpolated target, and the logit gradient takes the form πθ(a|h) − π̃α,ρ,θ(a|h, h_sf, h_ref). The experimental sections report results on SciKnowEval, five mathematical reasoning benchmarks, and LiveCodeBench v6 across Qwen3 model scales, plus an ablation on SciKnowEval Physics. The authors explicitly state that the theory applies only to fixed rollout contexts and does not establish convergence of the fully coupled student–self-teacher dynamics.
Significance. If the empirical claims are confirmed, the paper makes a useful conceptual contribution: it separates where the adaptive distillation target is placed (α) from how the student is projected toward it (ρ), and it provides explicit, apparently correct algebraic derivations for the resulting objectives and gradients. The proof of Proposition 3.2 in Appendix C is detailed and the fixed-context scope is stated honestly rather than hidden. The paper also evaluates across several domains and model scales and states that the code is open-sourced. However, the significance of the headline 'state-of-the-art or competitive performance' claim is currently limited by the lack of multi-seed experiments and by hyperparameter selection that rests on a single ablation domain.
major comments (3)
- [§4.1–§4.3, Tables 1–3] The central empirical claim that SR-OPSD is state-of-the-art or competitive rests entirely on single-run point estimates. None of the tables reports standard deviations, seed counts, or significance tests. The claimed margins are often comparable to plausible seed noise: for example, Table 1 shows Qwen3-8B Chemistry at 5h as 80.2 versus SDPO 80.5, and OLMo Biology at 15h as 51.1 versus 52.0; Table 3 shows Qwen3-0.6B SR-OPSD at 18.0 versus SDPO 18.7 and GRPO 18.5, a gap of at most 0.7 points. Without multi-seed results, confidence intervals, or a clearly stated evaluation protocol for variance, the 'state-of-the-art' conclusion is not established.
- [§4.4, Tables 4 and 8] The hyperparameters α=0.9 and ρ=0.95 are fixed across all tasks after an ablation only on SciKnowEval Physics. The ablation itself shows that adding self-reference degrades JSD (79.4→77.2) and forward KL (79.1→76.3), so the benefit appears only for the Rényi objective with a ρ selected on a single domain. Since the same ρ is then applied to mathematical reasoning and code generation, the transferability of this selection is load-bearing for the paper's main experimental conclusion. Please report sensitivity of the main results to ρ (and α) on at least one additional task, or justify the selection with a pre-registered rule; otherwise the conclusion that the Rényi combination is what drives the gains is not empirically supported.
- [§3 and Algorithm 1] The theoretical characterizations in Propositions 3.1–3.3 are explicitly pointwise and assume fixed contexts and frozen target components, a limitation the paper correctly states. However, the introduction and abstract motivate SR-OPSD by instability in the coupled student–self-teacher dynamics, and Algorithm 1 applies the local objective inside an evolving rollout loop. Since the paper does not provide any bound or monotonicity result for the coupled system, the theoretical section should either state more directly that it does not by itself justify the stability claims, or add a formal statement of what the local equivalence licenses for Algorithm 1. This is a scope concern rather than a correctness error, but it affects how readers can interpret the theoretical contribution.
minor comments (4)
- [§4.4 and Appendix D.3] Table 4 lists SR-OPSD (ρ=0.95) Avg@16 as 81.1, while Table 8 in Appendix D.3 reports 81.3 for the same configuration; please reconcile the discrepancy and state whether the two tables correspond to different checkpoints or evaluation runs.
- [Table 1] The base-model rows in Table 1 (e.g., 'Qwen3-8B 41.2 59.2 30.8 58.9 57.5') have fewer columns than the trained rows; please clarify whether these are baseline numbers at a single time point and format the table so that the column alignment is unambiguous.
- [Equation (2)] Equation (2) is introduced as 'the corresponding token-level objective' immediately after Proposition 3.2, but the relationship between Eq. (2) and the functional Rα,ρ in Proposition 3.2 is not stated explicitly; a sentence noting that Eq. (2) is the fixed-context equivalent up to the constant term would help the reader follow the derivation.
- [Conclusion and Abstract] The conclusion contains 'R’enyi' with a nonstandard apostrophe, while the abstract uses 'R\'enyi'; please normalize the spelling of 'Rényi' throughout.
Circularity Check
No significant circularity: the theoretical identities are algebraic characterizations of an explicitly constructed objective, and the empirical claims rest on external benchmarks rather than on fitted inputs relabeled as predictions.
full rationale
The paper's derivation chain is self-contained and not circular. Propositions 3.1–3.3 are algebraic identity results: Proposition 3.2 defines the token-level functional R_alpha,rho, and the pseudo-reward r_alpha, so that the equality J_alpha,rho = -D_rho(pi*_alpha || pi_theta) - const follows by direct manipulation of the definitions; the geometric target pi*_alpha is explicitly constructed ('At position t, define pi*_alpha ...'), not inferred from data. Proposition 3.3 computes the logit gradient of the Rényi divergence from this frozen target, and the derivation is standard calculus. The paper itself states the scope limitation in the Conclusion: 'Our theoretical characterization concerns token-level optimization at fixed rollout contexts and does not establish global convergence of the fully coupled student–self-teacher dynamics.' That limitation is disclosed and does not conceal a circular step. On the empirical side, alpha=0.9, rho=0.95, and beta=0.05 are fixed and reported; rho is selected through an ablation on SciKnowEval Physics, which is a disclosed hyperparameter choice rather than a fitted parameter being presented as a prediction. The reported performance is compared against external baselines (GRPO, SDPO, OPSD) on independent benchmarks, so the central SOTA claim is not forced by construction. There are no load-bearing self-citations: the cited OPSD, SDPO, and Rényi-divergence results come from other research groups, and no uniqueness theorem or prior claim by the present authors is invoked to forbid alternative designs. Accordingly, no circular step meeting the evidentiary standard is present.
Assumptions & free parameters
free parameters (5)
- alpha (self-reference coefficient) =
0.90 in all main experiments
- rho (Rényi order) =
0.95 in all main experiments; 0.50, 0.70, 0.95 in ablation
- beta (EMA teacher update rate) =
0.05 in all experiments
- Top-K candidate set size =
100 in most experiments; 100 with no tail bucket in math
- Distillation IS clip =
2.0
assumptions (3)
- domain assumption The fixed-context token-level variational equivalence (Propositions 3.1 and 3.2) remains a valid guide for the full coupled student-self-teacher training loop.
- standard math All token-level distributions have strictly positive support over the vocabulary or candidate set, so Rényi divergences and log-ratios are well-defined.
- domain assumption The frozen initial policy theta_0 is a useful reference anchor for the post-training task.
Cite this review
Pith. "Pith review of SR-OPSD: Self-Referenced On-Policy Self-Distillation." pith.science (2026). https://pith.science/paper/46EPMDWD
@misc{pith2026260809745,
author = {Pith},
title = {Pith review of: SR-OPSD: Self-Referenced On-Policy Self-Distillation},
year = {2026},
howpublished = {\url{https://pith.science/paper/46EPMDWD}},
note = {Machine review of arXiv:2608.09745}
}
read the original abstract
On-policy self-distillation (OPSD) converts feedback into dense token-level supervision on trajectories generated by the policy to be optimized, providing a useful complement to reinforcement learning with sparse outcome rewards. However, the self-teacher policy used in OPSD is typically a stop-gradient or exponential-moving-average copy of the policy conditioned on additional context information, and thus co-evolves with both the student policy and its on-policy context distribution. Directly matching such a moving target with a fixed projection objective can lead to unstable optimization or excessive distributional concentration. This nature of OPSD motivates the proposed \emph{Self-Referenced On-Policy Self-Distillation (SR-OPSD)}. At fixed student-generated contexts, a token-level variational characterization identifies the effective distillation target as a geometric interpolation between the self-teacher policy and a reference policy. Meanwhile, we use the R\'enyi divergence family to generalize the projection geometry. This formulation separates \emph{where} the adaptive target is placed from \emph{how} the student is projected toward it: the interpolation coefficient controls underlying target, while the R\'enyi order controls the projection geometry and its sensitivity to token-level density ratios. Extensive experiments across scientific evaluation, mathematical reasoning, and coding generation tasks with multiple large language models show that SR-OPSD achieves the state-of-the-art or competitive performance across various settings.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:1503.02531 , year=
Distilling the knowledge in a neural network , author=. arXiv preprint arXiv:1503.02531 , year=
-
[2]
International Conference on Learning Representations , year=
Finetuned Language Models are Zero-Shot Learners , author=. International Conference on Learning Representations , year=
-
[3]
International Conference on Learning Representations , year=
Multitask Prompted Training Enables Zero-Shot Task Generalization , author=. International Conference on Learning Representations , year=
-
[4]
The Twelfth International Conference on Learning Representations , year=
What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning , author=. The Twelfth International Conference on Learning Representations , year=
-
[5]
Proceedings of the 41st International Conference on Machine Learning , pages =
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , editor =
work page 2024
-
[6]
Training language models to follow instructions with human feedback , url =
Ouyang, Long and Wu, Jeffrey and Jiang, Xu and Almeida, Diogo and Wainwright, Carroll and Mishkin, Pamela and Zhang, Chong and Agarwal, Sandhini and Slama, Katarina and Ray, Alex and Schulman, John and Hilton, Jacob and Kelton, Fraser and Miller, Luke and Simens, Maddie and Askell, Amanda and Welinder, Peter and Christiano, Paul F and Leike, Jan and Lowe,...
-
[7]
Proceedings of the 41st International Conference on Machine Learning , pages =
Model Alignment as Prospect Theoretic Optimization , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , editor =
work page 2024
-
[8]
arXiv preprint arXiv:2604.00626 , year=
A survey of on-policy distillation for large language models , author=. arXiv preprint arXiv:2604.00626 , year=
Show all 54 references
-
[9]
Advances in neural information processing systems , volume=
Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=
-
[10]
Advances in neural information processing systems , volume=
Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=
-
[11]
The Twelfth International Conference on Learning Representations , year=
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author=. The Twelfth International Conference on Learning Representations , year=
-
[12]
arXiv preprint arXiv:2603.05433 , year=
Crisp: Compressed reasoning via iterative self-policy distillation , author=. arXiv preprint arXiv:2603.05433 , year=
-
[13]
Thinking Machines Lab: Connectionism , year =
Kevin Lu and Thinking Machines Lab , title =. Thinking Machines Lab: Connectionism , year =
-
[14]
International Conference on Learning Representations , volume=
Distillation of Large Language Models via Concrete Score Matching , author=. International Conference on Learning Representations , volume=
-
[15]
arXiv preprint arXiv:2605.07865 , year=
KL for a KL: On-Policy Distillation with Control Variate Baseline , author=. arXiv preprint arXiv:2605.07865 , year=
-
[16]
f-Divergence Minimization for Sequence-Level Knowledge Distillation
Wen, Yuqiao and Li, Zichao and Du, Wenyu and Mou, Lili. f-Divergence Minimization for Sequence-Level Knowledge Distillation. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023. doi:10.18653/v1/2023.acl-long.605
2023 doi
-
[17]
arXiv preprint arXiv:2605.08741 , year=
Training with Harnesses: On-Policy Harness Self-Distillation for Complex Reasoning , author=. arXiv preprint arXiv:2605.08741 , year=
-
[18]
arXiv preprint arXiv:2512.13961 , year=
Olmo 3 , author=. arXiv preprint arXiv:2512.13961 , year=
-
[19]
arXiv preprint arXiv:2406.09098 , year=
Sciknoweval: Evaluating multi-level scientific knowledge of large language models , author=. arXiv preprint arXiv:2406.09098 , year=
-
[20]
arXiv preprint arXiv:2505.09388 , year=
Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=
-
[21]
2026 , eprint=
SimCT: Recovering Lost Supervision for Cross-Tokenizer On-Policy Distillation , author=. 2026 , eprint=
2026
-
[22]
The Fourteenth International Conference on Learning Representations , year=
Distillation of Large Language Models via Concrete Score Matching , author=. The Fourteenth International Conference on Learning Representations , year=
-
[23]
NeurIPS 2022 Workshop on Score-Based Methods , year=
Towards Healing the Blindness of Score Matching , author=. NeurIPS 2022 Workshop on Score-Based Methods , year=
2022
-
[24]
2021 , eprint=
Blindness of score-based methods to isolated components and mixing proportions , author=. 2021 , eprint=
2021
-
[25]
The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=
Universal Cross-Tokenizer Distillation via Approximate Likelihood Matching , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=
-
[26]
The Fourteenth International Conference on Learning Representations , year=
Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation , author=. The Fourteenth International Conference on Learning Representations , year=
-
[27]
arXiv preprint arXiv:2602.12125 , year=
Learning beyond teacher: Generalized on-policy distillation with reward extrapolation , author=. arXiv preprint arXiv:2602.12125 , year=
-
[28]
Li, Yingzhen and Turner, Richard , booktitle =. R\'
-
[29]
Reinforcement Learning from Rich Feedback with Distributional
Rishabh Agrawal and Jacob Fein-Ashley and Paria Rashidinejad , booktitle=. Reinforcement Learning from Rich Feedback with Distributional. 2026 , url=
2026
-
[30]
Proceedings of The 33rd International Conference on Machine Learning , pages =
Black-Box Alpha Divergence Minimization , author =. Proceedings of The 33rd International Conference on Machine Learning , pages =. 2016 , editor =
2016
-
[31]
2026 , eprint=
Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation , author=. 2026 , eprint=
2026
-
[32]
2026 , eprint=
GATES: Self-Distillation under Privileged Context with Consensus Gating , author=. 2026 , eprint=
2026
-
[33]
2026 , eprint =
Stein, Alex and Huang, Furong and Goldstein, Tom , journal =. 2026 , eprint =
2026
-
[34]
arXiv preprint arXiv:2605.11458 , year =
Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning , author =. arXiv preprint arXiv:2605.11458 , year =. 2605.11458 , archivePrefix =
-
[35]
2026 , eprint =
Wang, Jiaxuan and Ouyang, Xuan and Chen, Zhiyu and Hu, Yulan and Pan, Zheng and Li, Xin and Guo, Lan-Zhe , journal =. 2026 , eprint =
2026
-
[36]
arXiv preprint arXiv:2605.11182 , year =
The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes , author =. arXiv preprint arXiv:2605.11182 , year =. 2605.11182 , archivePrefix =
-
[37]
Proceedings of the 31st International Conference on Computational Linguistics , pages=
Rethinking kullback-leibler divergence in knowledge distillation for large language models , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=
-
[38]
arXiv preprint arXiv:2605.13255 , year =
Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning , author =. arXiv preprint arXiv:2605.13255 , year =. 2605.13255 , archivePrefix =
-
[39]
arXiv preprint arXiv:2602.04942 , year =
Privileged Information Distillation for Language Models , author =. arXiv preprint arXiv:2602.04942 , year =. 2602.04942 , archivePrefix =
-
[40]
Proceedings of the 32nd International Conference on Machine Learning , pages =
Trust Region Policy Optimization , author =. Proceedings of the 32nd International Conference on Machine Learning , pages =. 2015 , editor =
2015
-
[41]
arXiv preprint arXiv:1707.06347 , year=
Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=
-
[42]
arXiv preprint arXiv:2402.03300 , year=
Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=
-
[43]
2025 , url=
Qiying Yu and Zheng Zhang and Ruofei Zhu and Yufeng Yuan and Xiaochen Zuo and YuYue and Weinan Dai and Tiantian Fan and Gaohong Liu and Juncai Liu and LingJun Liu and Xin Liu and Haibin Lin and Zhiqi Lin and Bole Ma and Guangming Sheng and Yuxuan Tong and Chi Zhang and Mofan Z...
2025
-
[44]
arXiv preprint arXiv:2306.08543 , year=
Minillm: On-policy distillation of large language models , author=. arXiv preprint arXiv:2306.08543 , year=
-
[45]
International Conference on Learning Representations , year=
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author=. International Conference on Learning Representations , year=
-
[46]
arXiv preprint arXiv:2602.12275 , year=
On-policy context distillation for language models , author=. arXiv preprint arXiv:2602.12275 , year=
-
[47]
Forty-third International Conference on Machine Learning , year=
Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. Forty-third International Conference on Machine Learning , year=
-
[48]
Forty-third International Conference on Machine Learning , year=
Reinforcement Learning via Self-Distillation , author=. Forty-third International Conference on Machine Learning , year=
-
[49]
Van Erven, Tim and Harremos, Peter , journal=. R. 2014 , publisher=
2014
-
[50]
arXiv preprint arXiv:2603.07079 , year=
Entropy-aware on-policy distillation of language models , author=. arXiv preprint arXiv:2603.07079 , year=
-
[51]
arXiv preprint arXiv:2605.21606 , year=
When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning , author=. arXiv preprint arXiv:2605.21606 , year=
-
[52]
arXiv preprint arXiv:2606.03532 , year=
When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation , author=. arXiv preprint arXiv:2606.03532 , year=
-
[53]
arXiv preprint arXiv:2604.08527 , year=
Demystifying opd: Length inflation and stabilization strategies for large language models , author=. arXiv preprint arXiv:2604.08527 , year=
-
[54]
arXiv preprint arXiv:2605.05040 , year=
Preference-Based Self-Distillation: Beyond KL Matching via Reward Regularization , author=. arXiv preprint arXiv:2605.05040 , year=
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.