Pith. sign in

REVIEW 4 major objections 5 minor 17 references

A vision-language model can supervise itself by comparing its own next-token predictions on a real image versus a content-erased blank image, removing the need for an external teacher, privileged answers, or visual evidence hints.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 07:03 UTC pith:XIKEFMB7

load-bearing objection Genuinely new self-distillation signal with clean math, but empirical evidence is thinner than the claims and the mechanism isn't fully proven—worth reviewing, not desk-rejecting. the 4 major comments →

arxiv 2607.21556 v1 pith:XIKEFMB7 submitted 2026-07-23 cs.CV cs.AI

Visual Contrastive Self-Distillation

Yijun Liang , Yunjie Tian , Yijiang Li , Yuqi Jia , Furong Huang , Tianyi Zhou , Di Fu This is my paper
classification cs.CV cs.AI
keywords self-distillationon-policy distillationvision-language modelscontrastive conditioningvisual groundingknowledge distillationEMA teacherinput conditioning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether on-policy self-distillation — where a model trains on its own outputs without an external teacher — can get the teacher–student asymmetry it needs from the input alone. The answer is yes: at every step of a student-generated answer, an EMA copy of the model predicts the next token twice, once with the original image and once with a content-erased black control. The log-probability difference between the two predictions highlights tokens whose likelihood is specifically increased by the image's instance-level content. The paper then sharpens the teacher's original-image distribution by upweighting those visually grounded tokens within a plausibility support and distills the result back into the student with forward KL. Across three scales of two vision-language model families, this consistently beats both the base models and a matched answer-hint OPSD baseline on a seven-benchmark aggregate.

Core claim

The paper's central claim is that the teacher–student asymmetry required for on-policy self-distillation can be built purely from input conditioning, without privileged answers, visual evidence, reasoning traces, or an external teacher. At each student-generated prefix, the EMA teacher produces two next-token distributions — one under the original image, one under a content-erased control (a same-size black image) — and their token-wise log-probability difference highlights candidates whose likelihood is specifically increased by instance-level visual content. This contrast sharpens the original-image distribution within a plausibility support, and the resulting full-distribution target is d

What carries the argument

The central object is the conditioning contrast Δ_t(v) = log p_phi(v | P, I, y_<t) − log p_phi(v | P, I_ctrl, y_<t), the token-wise log-probability difference between the EMA teacher's prediction conditioned on the original image and on a content-erased control at the same student-generated prefix. This contrast acts as an implicit visual-evidence reward. The target is q*_t(v) ∝ p_phi(v|P,I,y_<t) exp(α Δ_t(v)) over a plausibility support S_t(β), so the original-image distribution provides the anchor and the contrast sharpens relative preferences. Remark 1 shows this target is exactly the closed-form solution of a one-step KL-regularized policy update with the contrast as reward. Forward KL d

Load-bearing premise

The load-bearing premise is that the EMA teacher's distribution under a content-erased black image is a faithful surrogate for its prediction with no instance-specific visual information, so that the log-probability difference isolates visual grounding rather than low-level artifacts or unnatural-token shifts.

What would settle it

Construct control images that erase content but vary low-level statistics (e.g., a solid non-black color, uniform noise at different amplitudes, or an inverted image), and check whether VCSD's aggregate gains disappear or reverse; alternatively, compute the contrast on a dataset with human-annotated visually grounded tokens and test whether positive-contrast tokens actually align with the annotated evidence.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Self-distillation for vision-language models no longer needs any external signal: no answer hints, no reasoning traces, no visual-evidence crops, and no extra inference-time passes.
  • The contrast objective shifts the student's probability mass toward tokens whose likelihood is specifically increased by instance-level visual content, so training encourages visual grounding rather than reliance on language priors.
  • Because the target is built from the model's own paired predictions, the same recipe applies across model families and scales and improves over both the base model and matched OPSD baselines.
  • The plausibility support is load-bearing: removing it leads to progressive degradation over longer self-distillation, suggesting the anchor prevents recursive distortion of the target.
  • Ablations indicate the method is insensitive to exactly how content is erased — black, Gaussian noise, Gaussian blur, and no-image controls give similar accuracy — implying content removal itself is the operative mechanism.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the conditional-PMI interpretation is correct, VCSD is effectively self-distilling toward higher pointwise mutual information between each token and the observed image; a direct test would be whether the contrast scores align with human-annotated visual evidence tokens.
  • The insensitivity to the control construction suggests a general recipe: any input intervention that removes content while preserving the interface could supply the same asymmetry, potentially extending this idea to pure-language models by erasing content words from the prompt.
  • A stress test for the method would be to adversarially choose a control image that changes low-level statistics in a particular way; if performance collapses, the contrast is partly exploiting low-level cues rather than clean content removal.
  • Since the method adds no inference cost and uses only the model itself, it could plausibly be combined with other self-improvement signals as a complementary visual-grounding component in later training stages.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Visual Contrastive Self-Distillation (VCSD), an on-policy self-distillation objective for vision-language models. At each student-generated prefix, an EMA teacher is evaluated twice—once with the original image and once with a content-erased control—and the token-wise log-probability contrast is used to sharpen the teacher's original-image distribution within a plausibility support. The resulting full-distribution target is distilled into the student via forward KL. Experiments on ViRL39K with Qwen3-VL and Qwen3.5 models from 2B to 9B report consistent aggregate accuracy gains over the base models and over an answer-hint OPSD baseline on seven vision-language benchmarks.

Significance. If the empirical claims hold, VCSD is a conceptually simple and practically attractive contribution: it removes the need for privileged answers, visual evidence signals, external teachers, or verifiable rewards while producing a self-distillation signal from purely input-conditioning contrast. The mathematical reinterpretation in Section 3.5 and Appendix A (Remark 1) is correct and cleanly proved: the contrast-shaped target is exactly the KL-regularized policy-update optimum. The paper also includes useful ablations of plausibility support, contrastive strength, distillation divergence, and control-image construction. However, the empirical evidence as reported is not yet sufficient to establish the central claims: the main results are single point estimates without variance, hyperparameters are selected on the evaluation benchmarks, the OPSD baseline is under-specified and underperforms on Qwen3.5, and the content-erasure mechanism underlying the PMI interpretation is not directly validated.

major comments (4)
  1. [§4.2, Table 1] The paper reports single point estimates with no error bars, multiple seeds, or significance tests. Several aggregate gains are modest (e.g., +1.86 on Qwen3-VL-4B, +2.83 on Qwen3.5-4B), and on individual benchmarks VCSD is below the base or OPSD (e.g., BLINK on Qwen3-VL-4B; V* on Qwen3.5-9B). Without variance estimates or repeated runs, the central claim of 'consistently outperforms' is not statistically supported. Since training is stochastic, I request at least three seeds on a representative scale (e.g., 2B and 4B) and error bars on the main table, or an explicit significance analysis.
  2. [§4.1, §4.4, Figure 3] The hyperparameters α and β are selected by ablations on the same seven evaluation benchmarks, and the final reported numbers use the best settings from those ablations. This makes the headline results selected numbers rather than out-of-sample predictions. The authors should either tune on a separate validation split or demonstrate robustness across a range of α and β on a held-out set. In addition, the OPSD baseline is described only as 'published answer-hint OPSD' (Section 4.1); its hyperparameters, rollout count, EMA settings, and tuning procedure are not given. Given that OPSD is below the base model on Qwen3.5 (Table 1), it is not clear that the comparison is 'matched' in terms of training effort and hyperparameter tuning.
  3. [§3.5, Eq. (16); §4.6, Table 3] The PMI interpretation in Eq. (15)–(16) rests on the assumption that p_phi(v|J_ctrl, H_t) approximates the model's prediction without instance-specific visual information. Table 3 tests black, Gaussian-noise, Gaussian-blur, and no-image controls, but all of these simultaneously remove content and alter low-level image statistics. The similar aggregate accuracies across these controls are consistent with content removal being the key factor, but they are also consistent with any paired-input distribution shift being sufficient. To validate the stated mechanism, the authors should add a control that preserves natural image statistics while removing instance-level content—for example, a random different natural image or a patch-shuffled version of the original image. Without such a comparison, the claimed content-erasure mechanism and the PMI framing are underdetermined.
  4. [§4.6, Table 3] The paper states that the four control constructions achieve 'similar' aggregate accuracy (67.14, 67.04, 66.36, 66.24). With single runs and no error bars, it is impossible to tell whether these differences are meaningful. A claim of insensitivity to the control construction needs confidence intervals or repeated runs; otherwise the robustness conclusion is not well supported.
minor comments (5)
  1. [§3.3, Eq. (8)–(9)] Equation (8) uses e∆, while the log-score identity in Eq. (9) is stated 'for non-termination tokens.' For terminal tokens, e∆ = 0, so the unnormalized log score is simply log p_J. Consider defining r_t(v) = e∆_t(v) explicitly to avoid the subtle discontinuity between the two cases.
  2. [§4.7, Figure 3(c)] The y-axis label 'Drift reduced by anchor (%)' is not defined. State explicitly whether this is the relative reduction in the fraction of rollouts containing non-target-language tokens, and how it is computed.
  3. [§4.1] The training budget is described as '90 optimization steps' with batch 32 and n=8 rollouts. This is unusually short; please clarify how many epochs over ViRL39K this corresponds to, and whether the same budget was used for every baseline and variant.
  4. [§4.1] Please specify whether the OPSD baseline and VCSD use the same T_KD, same EMA decay, same rollout count n, and same response-position weighting. The phrase 'matched OPSD' should be substantiated with a table of shared settings.
  5. [Abstract / Introduction] There are small typographical spacing errors: 'we proposeVisual' and 'asVisual Contrastive Self-Distillation' in the introduction. Please proofread.

Circularity Check

1 steps flagged

No significant circularity: the central empirical claim is self-contained and benchmark-validated; one theoretical remark is a tautological restatement but is not load-bearing.

specific steps
  1. self definitional [Section 3.5 (Remark 1; Eqs. 8, 12–14)]
    "Remark 1. For each student-generated prefix, the contrast-shaped target in Equation 8 is the unique solution to the following optimization problem: q⋆t = arg max q∈Π(St(β)) { αEv∼q [r_vis_t(v)] − D_KL(q∥¯pJ φ,t) }. ... Because the support-normalization constant in ¯pJ φ,t cancels between the numerator and denominator, Equation 14 is exactly equivalent to the target defined in Equation 8."

    The 'reward' r_vis_t is defined in Eq. 12 as the log-ratio ∆_t(v) = log p^J_φ,t(v) − log p^0_φ,t(v), and Eq. 8 already defines q* by exponentiating α∆ against p^J. Remark 1 then proves that q* is the KL-regularized policy maximizing this same reward. This is a mathematical identity: for any reward r, a distribution of the form pbar·exp(αr)/Z is the unique KL-regularized optimum. The theorem restates the construction rather than deriving it from independent principles or predicting benchmark gains. It is a post-hoc interpretation, not the load-bearing evidence for VCSD.

full rationale

The central claim of the paper is empirical: VCSD, which constructs its self-distillation target from the EMA teacher's own conditional distributions under original and content-erased images, improves seven external benchmarks over matched OPSD baselines. That claim is judged against external benchmarks (BLINK, MMStar, MathVista, V*Bench, HRBench4K/8K, HallusionBench), so it does not reduce to the paper's definitions or to a fitted constant. The method is self-distillation by design, so the use of the same model family as teacher and student is not circular. The only definitional coincidence is Remark 1: the contrast-shaped target is, by construction, the KL-regularized optimum for a reward defined as the same log-ratio used to build the target. The paper explicitly frames this as an interpretation and the empirical sections, including ablations on control-image construction, plausible-support threshold, contrastive strength, and distillation divergence, carry the argument. No load-bearing self-citation chain is present: the HallusionBench citation includes a co-author but is used as an external evaluation benchmark, not as a theoretical premise, and no uniqueness theorem is imported from the authors' prior work. Thus the score is low: one tautological interpretive remark with no impact on the empirical validity of the method.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

VCSD introduces no new physical or mathematical entities; it reuses the existing model and image inputs. The main free parameters are α and β, both tuned on the evaluation suite. The content-erased control is a configuration, not an invented entity. The assumptions listed are standard in self-distillation and are stated explicitly or implicit in the method.

free parameters (2)
  • α (contrastive strength) = 1.0 (default)
    Chosen by ablation on Qwen3-VL-2B using the same seven benchmarks; Figure 3(b) shows performance peaks around α=1.
  • β (plausibility threshold) = 0.1 (default)
    Chosen by ablation on Qwen3-VL-2B; Figure 3(a) shows β=0.1 stabilizes long-horizon training versus β=0.
axioms (4)
  • domain assumption A black, same-size image is a valid content-erased control (C(J)).
    Section 3.2 assumes the black image removes instance-specific visual content while preserving interface, resolution, and visual-token count. The ablation in Table 3 supports content-removal robustness, but the assumption is not externally verified.
  • domain assumption The content-erased prediction p_phi(v|J_ctrl,H_t) approximates the prediction without instance-specific visual information, so Δ approximates conditional PMI.
    Section 3.5 states this as a 'when' condition, not a proven fact. The method's interpretation depends on it, but the empirical gains do not strictly require the PMI interpretation to hold.
  • domain assumption The EMA teacher provides a stable, stronger learning signal than the student.
    Section 3.1 assumes EMA averaging produces a teacher whose contrast is informative. The anchor ablation (Figure 3c) suggests drift control, but the core assumption is standard in self-distillation.
  • domain assumption The plausibility support S_t(β) restricts target shaping to candidates that are plausible under the original image.
    Section 3.3 introduces this relative-threshold support to prevent low-probability tokens from dominating. The ablation in Figure 3(a) supports its necessity, but the specific threshold is tuned on test benchmarks.

pith-pipeline@v1.3.0-alltime-deepseek · 13572 in / 10865 out tokens · 107794 ms · 2026-08-01T07:03:22.012235+00:00 · methodology

0 comments
read the original abstract

On-policy self-distillation (OPSD) is promising as it removes the external teacher required by on-policy distillation (OPD), yet it still needs asymmetric information between teacher and student to ensure that the self-teacher provides a stronger learning signal than the student. Existing methods create this asymmetry either through privileged answers or visual evidence. We ask whether both can be removed, yielding a simpler form of OPSD driven purely by input conditioning. For this purpose, we propose Visual Contrastive Self-Distillation, namely VCSD, which converts image-content removal into an on-policy self-distillation signal. At each student-generated response prefix, the EMA teacher produces two next-token distributions under the same prompt and prefix -- one conditioned on the original image and the other on a content-erased control. Their token-wise log-probability difference highlights candidates whose likelihood is specifically increased by the instance-level visual content. We use this contrast to sharpen the teacher's original-image distribution within its plausible support, and distill the resulting full-distribution target into the student. Using ViRL39K dataset, VCSD consistently outperforms matched OPSD across Qwen3-VL and Qwen3.5 models. For example, on Qwen3-VL, it improves the seven-benchmark aggregate from $62.27\% \rightarrow 67.04\%$ at 2B, $71.30\% \rightarrow 73.16\%$ at 4B, and $72.51\% \rightarrow 76.26\%$ at 8B. Furthermore, VCSD requires no external teacher, privileged answers, visual evidence signals, reasoning traces, or additional inference-time cost.

Figures

Figures reproduced from arXiv: 2607.21556 by Di Fu, Furong Huang, Tianyi Zhou, Yijiang Li, Yijun Liang, Yunjie Tian, Yuqi Jia.

Figure 1
Figure 1. Figure 1: Comparison of different sources of target asymmetry. Existing methods sharpen the teacher distribution using privileged answers or visual evidence. Our method instead contrasts teacher predictions under the original image and a content-erased control, producing a visually in￾formed target using only the image and question, without auxiliary supervision. This motivates the central question of this work: Can… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of VCSD. At each student-generated prefix, the EMA teacher contrasts next￾token distributions under the original image and a content-erased control. The contrast sharpens the plausible original-image distribution to form the target, which is distilled into the student via forward KL. The response prefix is omitted for clarity. where V is the vocabulary. The rollout is on-policy because the student… view at source ↗
Figure 3
Figure 3. Figure 3: Ablation studies on Qwen3-VL-2B. (a) Effect of plausibility support. Restricting target shaping to the plausible support stabilizes long-horizon self-distillation, whereas removing it leads to progressive performance degradation. (b) Effect of contrastive strength. Performance is strongest and locally robust for α ∈ [1, 1.5], while weaker or stronger shaping reduces accuracy. (c) Effect of original-image a… view at source ↗
Figure 4
Figure 4. Figure 4: Training dynamics of VCSD and OPSD. Seven-benchmark aggregate accuracy (left) and per-benchmark accuracy on MMStar (middle) and MathVista (right) over the same training run. Q: What number is shown? Ground truth: 7519 Qwen3.5-2B Base predicts 6169 (wrong) Top row: 6 large cubes → 6,000. Middle: one medium cube, representing 100 → +100. Bottom: 5 flats, 1 rod, 9 units → 69. Total = 6,169. OPSD predicts 8519… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison on MathVista (Qwen3.5-2B). The decisive visual quantity is the number of thousand-cubes. Red marks the miscounts made by the base and OPSD models; green marks the correct count used by VCSD. shaping reallocates probability mass toward image-dependent tokens instead of merely amplifying existing language preferences. 5 CONCLUSION We introduced VCSD, which constructs the target asymmet… view at source ↗
Figure 6
Figure 6. Figure 6: Token-level image-dependent contrast. Contrast scores for the same response under the base model, OPSD, and VCSD. Warmer colors indicate stronger image-dependent preference, and the bottom row shows the additional contrast learned by VCSD over OPSD. Shiyuan Feng, Huan-ang Gao, Haohan Chi, Hanlin Wu, Zhilong Zhang, Zheng Jiang, Bingxiang He, Wei-Ying Ma, Ya-Qin Zhang, and Hao Zhou. Weak-to-strong generaliza… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 14 linked inside Pith

  1. [1]

    On-policy distillation of language models: Learning from self- generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self- generated mistakes. InInternational Conference on Learning Representations, volume 2024, pp. 21246–21263,

    2024

  2. [3]

    Warmer colors indicate stronger image-dependent preference, and the bottom row shows the additional contrast learned by VCSD over OPSD

    11 Figure 6:Token-level image-dependent contrast.Contrast scores for the same response under the base model, OPSD, and VCSD. Warmer colors indicate stronger image-dependent preference, and the bottom row shows the additional contrast learned by VCSD over OPSD. Shiyuan Feng, Huan-ang Gao, Haohan Chi, Hanlin Wu, Zhilong Zhang, Zheng Jiang, Bingxiang He, Wei...

    Pith/arXiv arXiv

  3. [4]

    Reinforcement learn- ing via self-distillation.arXiv preprint arXiv:2601.20802,

    Jonas H ¨ubotter, Frederike L ¨ubeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, et al. Reinforcement learn- ing via self-distillation.arXiv preprint arXiv:2601.20802,

    Pith/arXiv arXiv

  4. [8]

    ai/blog?id=qwen3.5

    URLhttps://qwen. ai/blog?id=qwen3.5. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  5. [9]

    Self-distillation enables con- tinual learning.arXiv preprint arXiv:2601.19897,

    Idan Shenfeld, Mehul Damani, Jonas H ¨ubotter, and Pulkit Agrawal. Self-distillation enables con- tinual learning.arXiv preprint arXiv:2601.19897,

    Pith/arXiv arXiv

  6. [10]

    V-zero: Answer-label-free on-policy distillation with contrastive evidence gating for fine-grained visual reasoning.arXiv preprint arXiv:2606.25319,

    Haoxiang Sun, Zhihang Yi, Langxuan Deng, Yuhao Zhou, Peiqi Jia, Jian Zhao, Li Yuan, Jiancheng Lv, and Tao Wang. V-zero: Answer-label-free on-policy distillation with contrastive evidence gating for fine-grained visual reasoning.arXiv preprint arXiv:2606.25319,

    Pith/arXiv arXiv

  7. [11]

    Vicur: Visual cues as recoverable privilege for multimodal on-policy distillation.arXiv preprint arXiv:2606.05718,

    Kanghui Tian, Siyuan Liu, Ziang Yan, Sheng Xia, Shuai Dong, and Yi Wang. Vicur: Visual cues as recoverable privilege for multimodal on-policy distillation.arXiv preprint arXiv:2606.05718,

    Pith/arXiv arXiv

  8. [12]

    Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning.Advances in Neural Information Processing Systems, 38:30865–30891, 2026a

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning.Advances in Neural Information Processing Systems, 38:30865–30891, 2026a. Kaishen Wang, Hengrui Gu, Meijun Gao, and Kaixiong Zhou. Damo: Decoding by accumulating activations momentum f...

    Pith/arXiv arXiv

  9. [13]

    Self-distilled rlvr.arXiv preprint arXiv:2604.03128,

    Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weip- ing Wang, Jiaqi Wang, and Nan Duan. Self-distilled rlvr.arXiv preprint arXiv:2604.03128,

    Pith/arXiv arXiv

  10. [14]

    Decomposed on-policy distillation for vision- language reasoning: Steering gradients for visual grounding.arXiv preprint arXiv:2606.00564,

    Hee Suk Yoon, Eunseop Yoon, Jaehyun Jang, SooHwan Eom, Ji Woo Hong, Mark Hasegawa- Johnson, Qi Dai, Chong Luo, and Chang D Yoo. Decomposed on-policy distillation for vision- language reasoning: Steering gradients for visual grounding.arXiv preprint arXiv:2606.00564,

    Pith/arXiv arXiv

  11. [15]

    Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation.arXiv preprint arXiv:2605.18740,

    Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, and Yaojie Lu. Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation.arXiv preprint arXiv:2605.18740,

    Pith/arXiv arXiv

  12. [16]

    Self-distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734,

    13 Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734,

    Pith/arXiv arXiv

  13. [17]

    Weak-to-strong preference optimization: Stealing reward from weak aligned model

    Wenhong Zhu, Zhiwei He, Xiaofeng Wang, Pengfei Liu, and Rui Wang. Weak-to-strong preference optimization: Stealing reward from weak aligned model. InInternational Conference on Learning Representations, volume 2025, pp. 37116–37144,

    2025

  14. [2023]

    Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe.arXiv preprint arXiv:2604.13016,

    Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huan- ang Gao, Wenkai Yang, Zhiyuan Liu, et al. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe.arXiv preprint arXiv:2604.13016,

    Pith/arXiv arXiv

  15. [2024]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631,

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631,

    Pith/arXiv arXiv

  16. [2025]

    https://thinkingmachines.ai/blog/on-policy-distillation

    doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InInternational Conference on Learning Repres...

    doi:10.64434/tml.20251026 2024

  17. [2026]

    Visual-advantage on-policy distillation for vision-language models.arXiv preprint arXiv:2605.21924,

    12 Ruiqi Liu, Xiaolei Lv, Gengsheng Li, Ximo Zhu, Zhiheng Wang, Zhengbo Zhang, Junkai Chen, Zhiheng Li, Bo Li, Jun Gao, et al. Visual-advantage on-policy distillation for vision-language models.arXiv preprint arXiv:2605.21924,

    Pith/arXiv arXiv