Pith. sign in

REVIEW 4 major objections 6 minor 61 references

High- and low-resolution views of the same image give multimodal models a free teacher for self-distillation, improving original-resolution accuracy while cutting training cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 14:42 UTC pith:MOTORIBG

load-bearing objection Clean resolution-as-privilege trick for multimodal OPSD: real speedup and solid averages, but the causal claim still needs a same-view control. the 4 major comments →

arxiv 2607.24447 v1 pith:MOTORIBG submitted 2026-07-27 cs.CV

RP-OPSD: Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models

Qihui Zhu , Yuchen Wang , Zijian Wen , Tao Zhang , Mengjie Zhang , Yang Liu , Shuangwu Chen , Siying Wu
show 2 more authors
Jian Yang Xiaofeng Jiang
This is my paper
classification cs.CV
keywords multimodal large language modelson-policy self-distillationresolution-privileged learningknowledge distillationvisual question answeringtoken-level supervisionEMA teacher
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Multimodal language models often need expensive privileged signals—verified answers, external explanations, or cropped evidence—to run on-policy self-distillation. This paper argues that a simpler signal already exists: the same image at full resolution versus one-quarter of the pixels. A student rolls out answers from the cheap low-resolution view; a same-weight teacher, seeing the original image, supplies dense next-token targets on those student prefixes. Matching the teacher’s distribution strengthens the model under limited visual evidence, and that gain transfers back to full-resolution inference. On Qwen3.5-9B the method lifts average original-resolution benchmark scores by about 5.45% relative to the base model and trains 1.78× faster than answer-hint OPSD, using only image–question pairs.

Core claim

Resolution differences alone can serve as effective privileged information for on-policy self-distillation in multimodal LLMs: an original-resolution teacher supervising a half-resolution student’s own trajectories improves both low-resolution capability and original-resolution performance, without extra annotations or external models.

What carries the argument

RP-OPSD: the student samples on-policy responses from a half-width/height image; an EMA teacher scores the same prefixes on the full-resolution image; the student is trained with bias-corrected teacher-top-K reverse KL on that shared trajectory support.

Load-bearing premise

The gap between full- and half-resolution views stays a useful teacher signal throughout training, and skill gained under low-resolution inputs actually transfers to full-resolution use rather than only teaching the model to handle downsampled images.

What would settle it

Train with the same recipe but freeze or remove the resolution gap (same-resolution student and teacher, or a frozen initial teacher) and check whether original-resolution average gains and half-resolution transfer disappear relative to the reported RP-OPSD numbers on the same nine benchmarks.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Multimodal post-training can use ordinary image–question pairs without answer labels, solution traces, or region crops.
  • Lower-resolution student rollouts cut visual-token cost and yield roughly 1.78× wall-clock speedup versus full-resolution OPSD under matched budgets.
  • Half-resolution capability gains (about +6 points average on five benchmarks) accompany original-resolution gains, supporting transfer rather than resolution specialization alone.
  • Bias-corrected teacher-top-K reverse KL plus a light EMA teacher is a practical default for this resolution-asymmetric setup.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any cheap input degradation that preserves semantics but drops detail (compression, grayscale, fewer frames) might play the same privileged-information role as resolution.
  • If the gap collapses mid-training, adaptive downsampling schedules or occasional full-resolution student rollouts could keep the teacher informative.
  • The efficiency win suggests resolution-privileged distillation is especially attractive when visual tokens dominate sequence length.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes RP-OPSD, an on-policy self-distillation method for MLLMs in which the student generates rollouts from half-resolution (half width, half height) images while an EMA teacher scores the same prefixes on the original-resolution image, using a bias-corrected teacher-top-100 reverse-KL objective. The key selling point is that the privileged information is just the high-resolution view of the same image, so training needs only image–question pairs — no ground-truth answers, external rationale generators, or region annotations. On Qwen3.5-9B and 4B, RP-OPSD reports the best average over nine original-resolution benchmarks (80.43 vs. 79.80 for answer-hint OPSD at 9B), a +6.09 average gain under half-resolution evaluation, and a 1.78× training-time speedup over OPSD under matched data, steps, batch, rollouts, and hardware. Ablations cover the distillation objective (Table 3), EMA vs. frozen teacher (Table 4), student downsampling factor including a canvas-restoring 1/2→1 control (Table 5), and efficiency (Table 6).

Significance. If the results hold, the contribution is practically meaningful: it removes the annotation/external-model bottleneck for multimodal on-policy self-distillation, at lower training cost than answer-hint OPSD, with matched or better benchmark averages at two model scales. Strengths worth naming: the training setup is fully specified (Supplement Tables 2–3: optimizer, EMA rate, seeds, system config) and code is promised; the 1/2→1 canvas control in Table 5 is a genuinely informative ablation separating detail loss from canvas effects; the efficiency comparison in Table 6 is run under matched conditions with an explicit FLOPs argument (Eqs. 10–12) rather than an unsupported speedup claim; and the half-resolution evaluation (Table 2) directly tests the transfer hypothesis rather than asserting it. The method is simple enough to be widely adopted if the attribution holds.

major comments (4)
  1. [§4 / Tables 3–5] §4, Tables 3–5: the central causal claim — that the resolution gap is what supplies the effective privileged signal — is never ablated against a no-privilege control. Every ablation preserves the resolution asymmetry: Table 4's frozen teacher still sees full resolution; Table 5 varies only the student side. OPSD cannot serve as the control because it carries answer-hint privilege. The missing run is plain on-policy EMA self-distillation with teacher and student on identical inputs (e.g., half-res EMA teacher on half-res student, or full-res/full-res without hint). Rollouts are sampled at temperature 1.0 and the teacher lags the student, so that condition still has non-zero distillation signal; if it reaches ~80 average, the framing 'resolution differences alone supply effective privileged information' (§1, Abstract) collapses to generic self-distillation dynamics plus a speedup. The run
  2. [§4, Table 1] §4, Table 1: the headline margin over OPSD is 0.63 points (80.43 vs. 79.80) at 9B and 0.61 at 4B, from a single seed (42), one epoch, 55 optimization steps. Per-benchmark deltas over OPSD are frequently within noise (POPE +0.35, HR-4K −0.38, MME-RW EN −1.21) and the average gain is driven largely by VisualProbe (+7.00). With no variance estimate, the claim that RP-OPSD 'outperforms OPSD' is not established; what is established is parity at lower cost and without answer annotations — which is itself a solid claim and should perhaps be the framing. Please report at least 3 seeds for the 9B main comparison and the OPSD baseline, or provide per-benchmark confidence intervals. Relatedly, the identical V*Bench value 91.10 for SFT, OPSD, and RP-OPSD (9B) should be explained — if this is a genuine tie from discrete rule-based scoring, say so; if it reflects an evaluation artifact, it affects the
  3. [§4, Benchmarks] §4, Experimental Setup: responses not handled by rule-based parsing are scored by a Qwen3.5-9B LLM judge, while the models being evaluated are fine-tuned derivatives of Qwen3.5-9B. This creates a potential self-preference channel: the judge shares the base distribution with the evaluated checkpoints, and RP-OPSD explicitly trains the model to match its own EMA distribution. The paper does not report which benchmarks fall under judge evaluation versus rule-based parsing, nor any judge-agreement validation (e.g., human spot-check or a second, unrelated judge). Given that several key margins (VisualProbe, MME-RW) are a few points, please disclose the per-benchmark evaluation method and provide an agreement check on a sample. This is load-bearing because the largest gains are on fine-grained perception benchmarks where parsing is hardest and judge reliance is most likely.
  4. [Tables 1, 3–5] Table 1 vs. Tables 3–5: the same default configuration (bias-corrected RKL, EMA teacher, 1/2 student) appears with two different averages — 80.43 in Table 1 (nine metrics including MME-RW EN/CN) and 82.73 in Tables 3–5 (eight metrics, MME-RW replaced by CV-Bench). This is presumably a metric-set difference, but it is never stated, and a reader comparing Table 4's frozen-teacher row (82.22) against Table 1's RP-OPSD row (80.43) could draw wrong conclusions. State the metric sets explicitly in each table caption and explain why the ablation suite swaps MME-RW for CV-Bench.
minor comments (6)
  1. [Abstract / §1] Abstract and §1: 'one-quarter of the original resolution' is imprecise — the student sees half width and half height, i.e., one-quarter of the pixels at half the linear resolution. Figure 1's '1/4 pixels' label is clearer; align the text.
  2. [§3, Eq. (8)] §3, Eq. (8): the claim that the correction makes the gradient vanish when student matches teacher on the retained support is correct for the restricted objective, but the objective is not a proper divergence over the full vocabulary — say explicitly that gradients on the off-support tail are zero by construction, and briefly note the implication (the student is free to reallocate mass outside the teacher's top-100, which interacts with temperature-1.0 sampling).
  3. [§4, Baselines] §4: GRPO and OPSD baselines require ground-truth answers while RP-OPSD does not; this asymmetry is a selling point and should be stated in Table 1's caption or a column, since it changes the practical comparison, not just the score comparison.
  4. [§3 / Table 4] §3, Eq. (6) / Supplement: EMA rate ρ=0.05 with only 55 steps means the teacher lags the student substantially; a short sensitivity note on ρ (or a pointer to prior justification) would help, since Table 4 shows the EMA-vs-frozen margin is only 0.51 points.
  5. [Supplement A] Training data (Supplement A): the 5,295-sample mix draws heavily from ZwZ-RL-VQA (3,034). Please note whether any benchmark images overlap the training sources, particularly for HR-Bench and V*Bench where images are high-resolution.
  6. [Algorithm 1] Figure 2 and Algorithm 1 are clear, but Algorithm 1 line 15 updates the teacher after every minibatch while §3 says 'after each successful student update' — reconcile the wording (is there a failure case?).

Circularity Check

0 steps flagged

No significant circularity: empirical method paper whose benchmark gains are not forced by definition or self-citation.

full rationale

RP-OPSD is an empirical post-training recipe. The central claims are measured averages on held-out multimodal benchmarks (Tables 1–2) under original- and half-resolution evaluation, plus wall-clock training time (Table 6). The training objective (bias-corrected teacher-top-K reverse KL on student rollouts, Eqs. 7–9; EMA teacher, Eq. 6) optimizes next-token distributions under asymmetric resolutions; it does not algebraically define or fit the reported V*/HR-Bench/MME/POPE/etc. scores. Resolution privilege, EMA rate, Top-100 support, and half-resolution student inputs are design choices and hyperparameters, not tautological rewrites of the metrics. Citations to OPD/OPSD/Vision-OPD/MOPD supply methodological framing and a baseline comparator; none import a uniqueness theorem that forces the numerical gains. Causal attribution gaps (missing same-view no-privilege control) are external-validity concerns, not circularity. Derivation chain is self-contained experimental comparison; score 0.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

Load-bearing content is mostly standard ML practice plus one domain hypothesis (resolution gap as privilege) and several hand-chosen training knobs. No new physical entities. The claim rests on prior OPD/OPSD machinery, the empirical resolution gap in Figure 1, and hyperparameter choices that were ablated only partially.

free parameters (4)
  • student downsampling factor (default 1/2 width & height) = 1/2 (quarter pixels); Lanczos
    Chosen as the privilege gap; Table 5 shows 1/3 and 1/4 hurt average score. Central efficiency and transfer claims depend on this operating point.
  • EMA rate ρ = 0.05
    Teacher update speed; compared only to frozen teacher, not swept broadly.
  • teacher-top-K support size = 100
    Truncates distillation support; objective ablations fix K=100.
  • learning rate, rollouts G, batch, epoch budget = 2e-6; G=8; batch 96; 1 epoch / 55 steps
    Standard optim knobs that determine whether the reported 55-step gains appear; not derived.
axioms (5)
  • domain assumption On-policy token-level distribution matching to a privileged teacher improves the student on states it visits (OPD/OPSD principle).
    Imported from Agarwal et al. 2024 and Zhao et al. 2026; §3 builds RP-OPSD on this without re-deriving when it fails.
  • domain assumption Same scene at higher resolution supplies strictly richer visual evidence that yields a useful teacher–student capability gap for the same question.
    Motivated by Figure 1’s 6.21-point drop; assumed stable enough to drive distillation for the whole training run.
  • domain assumption Bias-corrected top-K reverse KL on teacher-selected support is a valid dense surrogate for full-vocab matching.
    Taken from MOPD-style objectives (Ma et al. 2026) and adopted after Table 3 ablation.
  • standard math Standard autodiff / softmax / EMA update mathematics.
    Eqs. (6)–(9) use ordinary stopped-gradient EMA and KL algebra.
  • ad hoc to paper Half-resolution student improvement transfers to original-resolution inference because teacher and student share parameters.
    Core transfer hypothesis in §1; supported by joint Table 1/2 results but not proven as a general law.
invented entities (1)
  • RP-OPSD training setup (resolution-asymmetric student/teacher pair + bias-corrected teacher-top-K RKL) no independent evidence
    purpose: Name the proposed post-training procedure and loss assembly.
    Methodological construct, not a latent physical object; existence is the algorithm itself. independent_evidence is the empirical tables, not an external measurement channel.

pith-pipeline@v1.2.0-grok45-kimik3 · 19675 in / 3577 out tokens · 80263 ms · 2026-07-31T14:42:12.218620+00:00 · methodology

0 comments
read the original abstract

On-Policy Self-Distillation (OPSD) uses privileged information available only to the teacher to provide dense token-level supervision on trajectories generated by the student. However, existing methods often rely on verified solution traces, explanations generated by external models, or manually localized visual evidence, which limits their scalable application to multimodal large language models. To address this issue, we exploit the information gap between high- and low-resolution views of the same image and propose RP-OPSD (Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models). During training, the student policy generates on-policy trajectories from images at one-quarter of the original resolution, while the teacher policy provides supervision using the original-resolution images. By minimizing the divergence between their output distributions along the student trajectories, the student learns the predictive behavior of the teacher under high-resolution inputs, thereby strengthening its low-resolution capability and transferring the learned improvement to original-resolution inference. RP-OPSD requires neither additional human annotations nor external models to generate solution traces but only image--question pairs. Experiments on Qwen3.5-9B show that RP-OPSD achieves a 5.45\% relative improvement in average performance at the original resolution and a $1.78\times$ training speedup over OPSD. These results demonstrate that resolution differences can serve as a simple and scalable source of privileged information, providing an effective and efficient approach to on-policy self-distillation for multimodal large language models.

Figures

Figures reproduced from arXiv: 2607.24447 by Jian Yang, Mengjie Zhang, Qihui Zhu, Shuangwu Chen, Siying Wu, Tao Zhang, Xiaofeng Jiang, Yang Liu, Yuchen Wang, Zijian Wen.

Figure 1
Figure 1. Figure 1: Resolution-induced capability gap of Qwen3.5- [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of RP-OPSD. We first use a low-resolution student to generate on-policy trajectories from downsampled [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 1
Figure 1. Figure 1: Fine-grained OCR on a shipping container. The target crop makes the localization requirement explicit, while the [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Distant number recognition in a container terminal. RP-OPSD grounds the spatial reference to the correct crane and [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 20 linked inside Pith

  1. [1]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  2. [2]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  3. [3]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  4. [4]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  5. [5]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    doi:10.1016/s0020-7373(84)80003-6 1984

  6. [6]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  7. [7]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  8. [8]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

  9. [9]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  10. [10]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  11. [11]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  12. [12]

    2023 , eprint =

    Visual Instruction Tuning , author =. 2023 , eprint =

    2023

  13. [16]

    2026 , howpublished =

    2026

  14. [17]

    2015 , eprint =

    Distilling the Knowledge in a Neural Network , author =. 2015 , eprint =

    2015

  15. [18]

    International Conference on Learning Representations , editor =

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author =. International Conference on Learning Representations , editor =

  16. [19]

    Thinking Machines Lab: Connectionism , year =

    On-Policy Distillation , author =. Thinking Machines Lab: Connectionism , year =

  17. [23]

    2026 , eprint =

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author =. 2026 , eprint =

    2026

  18. [24]

    2026 , eprint =

    Self-Distillation Enables Continual Learning , author =. 2026 , eprint =

    2026

  19. [25]

    2026 , eprint =

    Reinforcement Learning via Self-Distillation , author =. 2026 , eprint =

    2026

  20. [29]

    Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results , url =

    Tarvainen, Antti and Valpola, Harri , booktitle =. Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results , url =

  21. [30]

    2026 , eprint =

    Self-Rewarding Vision-Language Model via Reasoning Decomposition , author =. 2026 , eprint =

    2026

  22. [31]

    2026 , eprint =

    From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models , author =. 2026 , eprint =

    2026

  23. [32]

    2026 , eprint =

    Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception , author =. 2026 , eprint =

    2026

  24. [33]

    Wu, Penghao and Xie, Saining , booktitle =

  25. [34]

    2024 , eprint =

    Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models , author =. 2024 , eprint =

    2024

  26. [35]

    Zhang, Yi-Fan and Zhang, Huanyu and Tian, Haochen and Fu, Chaoyou and Zhang, Shuangqing and Wu, Junfei and Li, Feng and Wang, Kun and Wen, Qingsong and Zhang, Zhang and Wang, Liang and Jin, Rong and Tan, Tieniu , booktitle =

  27. [37]

    Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal

    Tong, Shengbang and Liu, Zhuang and Zhai, Yuexiang and Ma, Yi and LeCun, Yann and Xie, Saining , booktitle =. Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal

  28. [38]

    2024 , eprint =

    Are We on the Right Way for Evaluating Large Vision-Language Models? , author =. 2024 , eprint =

    2024

  29. [39]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , month =

    Evaluating Object Hallucination in Large Vision-Language Models , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , month =. 2023 , address =. doi:10.18653/v1/2023.emnlp-main.20 , pages =

    doi:10.18653/v1/2023.emnlp-main.20 2023

  30. [40]

    doi:10.52202/079017-2771 , editor =

    Tong, Shengbang and Brown, Ellis and Wu, Penghao and Woo, Sanghyun and Middepogu, Manoj and Akula, Sai Charitha and Yang, Jihan and Yang, Shusheng and Iyer, Adithya and Pan, Xichen and Wang, Austin and Fergus, Rob and LeCun, Yann and Xie, Saining , booktitle =. doi:10.52202/079017-2771 , editor =

    doi:10.52202/079017-2771

  31. [41]

    2026 , eprint=

    GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning , author=. 2026 , eprint=

    2026

  32. [42]

    Agarwal, R.; Vieillard, N.; Zhou, Y.; Stanczyk, P.; Ramos Garea, S.; Geist, M.; and Bachem, O. 2024. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. In Kim, B.; Yue, Y.; Chaudhuri, S.; Fragkiadaki, K.; Khan, M.; and Sun, Y., eds., International Conference on Learning Representations, volume 2024, 21246--21263

    2024

  33. [43]

    Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng, L.; Ding, W.; Gao, C.; Ge, C.; et al. 2025. Qwen3-VL Technical Report. arXiv:2511.21631

    Pith/arXiv arXiv 2025

  34. [44]

    Chen, L.; Li, J.; Dong, X.; Zhang, P.; Zang, Y.; Chen, Z.; Duan, H.; Wang, J.; Qiao, Y.; Lin, D.; and Zhao, F. 2024. Are We on the Right Way for Evaluating Large Vision-Language Models? arXiv:2403.20330

    Pith/arXiv arXiv 2024

  35. [45]

    Cheng, Z.; Chen, S.; Yang, B.; Guan, Y.; Chen, J.; Lian, Z.; Peng, X.; Ma, F.; Cui, L.; and Tian, Q. 2026. OmniOPSD : Rationale-Privileged On-Policy Self-Distillation for Affective Computing. arXiv:2606.15920

    Pith/arXiv arXiv 2026

  36. [46]

    Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531

    Pith/arXiv arXiv 2015

  37. [47]

    Huang, W.; Jia, B.; Zhai, Z.; Cao, S.; Ye, Z.; Zhao, F.; Xu, Z.; Tang, X.; Hu, Y.; and Lin, S. 2025. Vision-R1 : Incentivizing Reasoning Capability in Multimodal Large Language Models. arXiv:2503.06749

    Pith/arXiv arXiv 2025

  38. [48]

    u botter, J.; L \

    H \"u botter, J.; L \"u beck, F.; Behric, L.; Baumann, A.; Bagatella, M.; Marta, D.; Hakimi, I.; Shenfeld, I.; Buening, T. K.; Guestrin, C.; and Krause, A. 2026. Reinforcement Learning via Self-Distillation. arXiv:2601.20802

    Pith/arXiv arXiv 2026

  39. [49]

    Lai, X.; Li, J.; Li, W.; Liu, T.; Li, T.; and Zhao, H. 2025. Mini-o3 : Scaling Up Reasoning Patterns and Interaction Turns for Visual Search. arXiv:2509.07969

    Pith/arXiv arXiv 2025

  40. [50]

    Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, X.; and Wen, J.-R. 2023. Evaluating Object Hallucination in Large Vision-Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 292--305. Singapore: Association for Computational Linguistics

    2023

  41. [51]

    Li, Z.; Yu, W.; Huang, C.; Liang, Z.; Liu, R.; Liu, F.; Che, J.; Yu, D.; Boyd-Graber, J.; Mi, H.; and Yu, D. 2026. Self-Rewarding Vision-Language Model via Reasoning Decomposition. arXiv:2508.19652

    Pith/arXiv arXiv 2026

  42. [52]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual Instruction Tuning. arXiv:2304.08485

    Pith/arXiv arXiv 2023

  43. [53]

    Lu, K.; and Thinking Machines Lab . 2025. On-Policy Distillation. Thinking Machines Lab: Connectionism. https://thinkingmachines.ai/blog/on-policy-distillation/

    2025

  44. [54]

    Ma, W.; Wei, J.; Zhao, L.; Zhang, H.; Xiao, B.; Li, L.; Yang, Q.; Gao, B.; Wang, Y.; Li, R.; Dong, J.; Sui, Z.; and Luo, F. 2026. MOPD : Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training. arXiv:2606.30406

    Pith/arXiv arXiv 2026

  45. [55]

    Qwen Team . 2026. Qwen3.5 : Towards Native Multimodal Agents. https://qwen.ai/blog?id=qwen3.5. Accessed: 2026-07-23

    2026

  46. [56]

    K.; Wu, Y.; and Guo, D

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D. 2024. DeepSeekMath : Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300

    Pith/arXiv arXiv 2024

  47. [57]

    Shen, H.; Liu, P.; Li, J.; Fang, C.; Ma, Y.; Liao, J.; Shen, Q.; Zhang, Z.; Zhao, K.; Zhang, Q.; Xu, R.; and Zhao, T. 2025. VLM-R1 : A Stable and Generalizable R1-Style Large Vision-Language Model. arXiv:2504.07615

    Pith/arXiv arXiv 2025

  48. [58]

    Shenfeld, I.; Damani, M.; H \"u botter, J.; and Agrawal, P. 2026. Self-Distillation Enables Continual Learning. arXiv:2601.19897

    Pith/arXiv arXiv 2026

  49. [59]

    Tarvainen, A.; and Valpola, H. 2017. Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results. In Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 30, 1195--1204. Curran Associates, Inc

    2017

  50. [60]

    Team, V.; Hong, W.; Yu, W.; Gu, X.; Wang, G.; Gan, G.; Tang, H.; Cheng, J.; Qi, J.; Ji, J.; Pan, L.; Duan, S.; Wang, W.; Wang, Y.; Cheng, Y.; He, Z.; Su, Z.; Yang, Z.; Pan, Z.; Zeng, A.; Wang, B.; Chen, B.; Shi, B.; Pang, C.; Zhang, C.; Yin, D.; Yang, F.; Chen, G.; Li, H.; Zhu, J.; Chen, J.; Xu, J.; Xu, J.; Chen, J.; Lin, J.; Chen, J.; Wang, J.; Chen, J.;...

    Pith/arXiv arXiv 2026

  51. [61]

    C.; Yang, J.; Yang, S.; Iyer, A.; Pan, X.; Wang, A.; Fergus, R.; LeCun, Y.; and Xie, S

    Tong, S.; Brown, E.; Wu, P.; Woo, S.; Middepogu, M.; Akula, S. C.; Yang, J.; Yang, S.; Iyer, A.; Pan, X.; Wang, A.; Fergus, R.; LeCun, Y.; and Xie, S. 2024 a . Cambrian-1 : A Fully Open, Vision-Centric Exploration of Multimodal LLMs . In Globerson, A.; Mackey, L.; Belgrave, D.; Fan, A.; Paquet, U.; Tomczak, J.; and Zhang, C., eds., Advances in Neural Info...

    2024

  52. [62]

    Tong, S.; Liu, Z.; Zhai, Y.; Ma, Y.; LeCun, Y.; and Xie, S. 2024 b . Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9568--9578

    2024

  53. [63]

    Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Fan, Y.; Dang, K.; Du, M.; Ren, X.; Men, R.; Liu, D.; Zhou, C.; Zhou, J.; and Lin, J. 2024 a . Qwen2-VL : Enhancing Vision-Language Model's Perception of the World at Any Resolution. arXiv:2409.12191

    Pith/arXiv arXiv 2024

  54. [64]

    Wang, W.; Ding, L.; Zeng, M.; Zhou, X.; Shen, L.; Luo, Y.; and Tao, D. 2024 b . Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models. arXiv:2408.15556

    Pith/arXiv arXiv 2024

  55. [65]

    Wang, W.; Gao, Z.; Gu, L.; Pu, H.; Cui, L.; Wei, X.; Liu, Z.; Jing, L.; Ye, S.; Shao, J.; et al. 2025. InternVL3.5 : Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency. arXiv:2508.18265

    Pith/arXiv arXiv 2025

  56. [66]

    Wei, L.; He, L.; Lan, J.; Dong, L.; Cai, Y.; Li, S.; Zhu, H.; Wang, W.; Kong, L.; Wang, Y.; Zhang, Z.; and Huang, W. 2026. Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception. arXiv:2602.11858

    arXiv 2026

  57. [67]

    Wu, J.; Chen, H.; Tu, H.; Tang, X.; Shi, F.; Liu, H.; Lu, H.; Xie, C.; and Zhou, Y. 2026. From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models. arXiv:2605.20177

    Pith/arXiv arXiv 2026

  58. [68]

    Wu, P.; and Xie, S. 2024. V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13084--13094

    2024

  59. [69]

    Yuan, Q.; Lou, J.; Yu, X.; Lin, H.; Sun, L.; Han, X.; and Lu, Y. 2026. Vision-OPD : Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation. arXiv:2605.18740

    Pith/arXiv arXiv 2026

  60. [70]

    Zhang, Y.-F.; Zhang, H.; Tian, H.; Fu, C.; Zhang, S.; Wu, J.; Li, F.; Wang, K.; Wen, Q.; Zhang, Z.; Wang, L.; Jin, R.; and Tan, T. 2025. MME-RealWorld : Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios That Are Difficult for Humans? In Yue, Y.; Garg, A.; Peng, N.; Sha, F.; and Yu, R., eds., International Conference on Learning Repr...

    2025

  61. [71]

    Zhao, S.; Xie, Z.; Liu, M.; Huang, J.; Pang, G.; Chen, F.; and Grover, A. 2026. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. arXiv:2601.18734

    Pith/arXiv arXiv 2026