{"id":"f56f6aaa-ab28-42c7-ac4c-85931bd2d2fa","arxiv_id":"2608.09931","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"CVPD finds visual blind spots by comparing a model's own crop, full-image, and ghosted responses, then uses the crop as a positive teacher and the ghost as a negative teacher for dense self-distillation.","lead":"This paper teaches a vision-language model to notice small details by finding spots where zooming in makes the model more confident, then distilling those sharper answers back into the model. The authors report gains on OCR, fine-grained perception, and reasoning benchmarks without external labels, segmentation tools, or larger teacher models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 15,000-image unlabeled training pool is undisclosed and no deduplication against evaluation benchmarks is reported; the headline OCRBench and MMStar gains may reflect image overlap rather than self-distillation.","rationale":"The crop-teacher correctness issue identified by the reader is genuine, but it is less decisive: if the self-generated crop teacher were broadly wrong, distillation would be expected to degrade external benchmarks, yet the reported results improve them. Contamination, in contrast, can fully explain the observed benchmark improvements without requiring the method to work as claimed. The paper's own Training Data section leaves the pool undescribed, and the absence of sources and deduplication is missing support for the central claim. A hash-release and overlap check is a single concrete step that settles the issue. I therefore agree with the reader's conditional verdict but focus on the data-leakage premise rather than the teacher-correctness premise.","tokens_in":15280,"tokens_out":7169,"duration_ms":68738,"concrete_test":"Release SHA-256 hashes or image embeddings for the 15,000-image pool, then run exact and near-duplicate matching against the official image sets of AI2D, ScienceQA, InfoVQA, OCRBench, CV-Bench, RealWorldQA, MMBench-EN, MME-P, SEED-Image, and MMStar. Report per-benchmark overlap counts, remove any overlapping images from evaluation or training, and recompute Table 1 on the disjoint subsets. If all overlaps are zero, the contamination concern is settled; if any overlap exists, the advertised gains must be recomputed and the headline claims adjusted accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's Training Data section states that the pool is 10,000 natural-scene images plus 5,000 images in a 40:60 mix of reasoning-domain content (charts, scientific diagrams, structured document figures) and natural scenes, but it does not name sources, report hashes, or describe any deduplication. The evaluation suite includes exactly the modalities most likely to overlap: AI2D, ScienceQA, InfoVQA, OCRBench, CV-Bench, and RealWorldQA all contain charts, diagrams, documents, or real-world scenes. Because the model is trained with LoRA on these images, even without benchmark labels, seeing the same image during training can inflate performance by memorizing image-specific OCR strings, layout patterns, and visual details; the reported +3.60 OCRBench and +3.38 MMStar Fine-Grained Perception gains are the pattern contamination would produce. This is load-bearing because the central empirical claim, including 'without a single regression' and the gains over six baselines, assumes the unlabeled pool and evaluation sets are disjoint. The paper provides no way to verify this assumption, making the headline results uncheckable as reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CVPD, a self-distillation method for multimodal large language models that derives dense token-level supervision from the model's own counterfactual responses on unlabeled images. A three-gate Counterfactual Criterion identifies regions where a cropped view sharpens the model's answer distribution while a ghosted (blurred) view leaves the full-image distribution largely unchanged. Phase 2 then trains a LoRA student using the crop-conditioned EMA teacher as a positive target and the ghost-conditioned EMA teacher as a negative target, together with a KL anchor to the frozen reference policy. Experiments on Qwen3-VL-4B and 8B claim consistent gains over six self-evolving baselines across twelve benchmarks, with the largest improvements on OCRBench and MMStar fine-grained and logical reasoning subscales. Ablations show that replacing the counterfactual criterion with random regions and removing the contrastive ranking objective both cause substantial drops, and hyperparameter sensitivity for the ranking weight, margin, and KL target is reported as flat.","tokens_in":15503,"tokens_out":5635,"duration_ms":53041,"significance":"If the result holds, CVPD is a meaningful step toward fully self-contained visual self-distillation: it removes the need for external region annotations, segmentation tools, rewards, or stronger teachers, and it provides per-token rather than scalar supervision. The counterfactual blind-spot criterion is a novel operationalization of 'latent perceptual capacity,' and the paper supports it with a focused ablation study, a blind-spot source analysis, and a sensitivity analysis. The paper also releases code, a model, and a project page, which strengthens reproducibility once the data provenance is clarified. The main open question is whether the benchmark gains reflect genuine distillation or overlap between the undisclosed unlabeled pool and the evaluation sets, and whether the crop-conditioned teacher is trustworthy in the absence of any correctness check.","major_comments":[{"comment":"The central empirical claim, including the headline +3.60 on OCRBench, +3.38 on MMStar Fine-Grained Perception, and the 'without a single regression' statement, presupposes that the 15,000-image unlabeled pool is disjoint from the evaluation benchmarks. The Training Data section does not name the image sources, provide hashes, or describe any deduplication; it only says that Stage 2 consists of charts, scientific diagrams, and structured document figures plus natural scenes. The evaluation suite includes AI2D, ScienceQA, InfoVQA, OCRBench, CV-Bench, and RealWorldQA, which contain exactly these modalities. Because the model is trained with LoRA on these images, overlapping images could inflate per-image recall, especially for OCR strings and layout patterns, without improving perception. This is load-bearing, not a reproducibility nicety, and it is fixable: disclose sources and hashes, run exact and near-duplicate deduplication against all evaluation sets, and report results on a provably disjoint subset.","section":"5 Training Data / Table 1"},{"comment":"Gate G3 equates a drop in predictive entropy under the crop with epistemic sharpening, but without labels or an external verifier there is no check that the sharpened distribution is more correct rather than merely more confident. A crop that confidently produces a wrong answer passes all three gates and is then reinforced by the latent-transfer and contrastive-ranking objectives in Eq. (6). Figure 3 reports distributional statistics only (divergences and entropy deltas), not crop-conditioned accuracy, so the claim that these regions 'reveal perceptual information the model can encode' is not directly evidenced. The manuscript should provide a labeled audit subset, a human evaluation, or a cross-view consistency check that verifies crop-conditioned answers are at least as accurate as full-image answers before treating the crop view as a trustworthy teacher.","section":"4.1 Eq. (4), 4.2 Eq. (6)"}],"minor_comments":[{"comment":"Gate G3 uses only the first token y0 for the entropy comparison, while the distillation objective in Eq. (6) averages over all rollout tokens; please justify this choice or extend the criterion to the full probe answer.","section":"4.1 Eq. (4)"},{"comment":"The track membership percentages sum to 110% (27% + 53% + 30%); the text says this is due to multi-region retention, but reporting per-track counts or a Venn-style breakdown would make the numbers easier to interpret.","section":"Algorithm 1 / Table 4"},{"comment":"The 'no curation (random regions)' ablation should specify how the random regions are generated; if they are uniformly random boxes within the same 1%-50% area range, the comparison to the gated criterion is cleaner than if they are sampled from a different distribution.","section":"Table 2 row (d)"},{"comment":"The sensitivity analysis covers lambda_rank, margin m, and KL target kappa, but not the discovery thresholds tau_crop and tau_ghost; since these thresholds control the 17.2% yield, a brief sweep would strengthen the claim that the criterion is not finely tuned.","section":"5.3 Sensitivity Analysis"},{"comment":"The sentence 'with no captions, bounding boxes, semantic labels, ground-truth answers, reward signals, or external models are used at any stage' contains a grammar error and should be reworded, for example as 'no captions, bounding boxes, semantic labels, ground-truth answers, reward signals, or external models are used at any stage.'","section":"5 Training Data"}],"recommendation":"major_revision","confidential_remarks":"The unlabeled-pool issue is the most serious concern. I would want to see source disclosure, deduplication hashes, and results on a guaranteed-disjoint subset before accepting. The crop-teacher trustworthiness concern is also real and should be addressed with some form of oracle audit or consistency check. The method itself is novel and the ablations are internally consistent, so the paper is worth a major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe core idea here is genuinely worth your time: CVPD finds regions where a crop sharpens the model's answer distribution and a ghosted version doesn't, then distills the crop-as-teacher against the ghost-as-negative. That's a clean, self-contained mechanism that moves beyond Vision-OPD's dependence on external region proposals and labels. The three-gate criterion and the ghost-conditioned negative teacher are real additions, and the ablations support that both pieces matter: random regions and no contrastive ranking both cause large drops.\n\nThe paper is clearly written and the benchmark pattern is internally consistent—largest gains on OCRBench and MMStar Fine-Grained Perception, no regressions across twelve tasks. If those numbers are real, it's a meaningful advance.\n\nBut the stress-test note is on target, and it's load-bearing. The Training Data section says the pool is 10,000 natural images plus 5,000 in a 40:60 mix of charts, diagrams, and documents, but it never names sources, gives hashes, or reports dedup against the evaluation set. The eval suite includes exactly those modalities: AI2D, ScienceQA, InfoVQA, OCRBench, CV-Bench, RealWorldQA. Training with LoRA on overlapping images can inflate OCR and layout-sensitive benchmarks without the model learning anything general. The +3.60 OCRBench and +3.38 MMStar FG gains are precisely the pattern contamination produces. As reported, the headline results are uncheckable.\n\nTwo smaller concerns. First, G3 gates on entropy, not correctness—a confidently wrong crop teacher would be reinforced just as strongly as a right one, and the paper doesn't probe this. Second, the ablations appear to be single-seed; with gains of one to three points, \"without a single regression\" is fragile without variance estimates.\n\nThe right response is not to dismiss the method. The right response is to demand the data release, dedup checks, and seed variance before taking the empirical claims at face value. A serious referee should engage with this paper; it just shouldn't be accepted in its current form.\n\nBring it to reading group—it's a good test case for how much we trust self-distillation results without contamination checks.","headline":"CVPD's counterfactual blind-spot criterion is a real idea, but the undisclosed 15k-image training pool makes the headline benchmark gains uncheckable until contamination is ruled out.","tokens_in":16075,"tokens_out":3010,"would_cite":true,"duration_ms":25683,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A model learns to see better from its own counterfactual blind spots.","keywords":["visual self-distillation","multimodal large language models","counterfactual blind spots","on-policy distillation","self-improvement","contrastive learning","fine-grained perception","OCRBench"],"falsifier":"On a labeled subset of images, compare answer accuracy under the full image, the crop, and the ghost for regions that pass all three gates. If crop-conditioned answers are not systematically more accurate than full-image answers on these regions, Gate 3 is selecting confidence rather than capability and the central mechanism fails; if they are more accurate, the paper's premise holds.","tokens_in":15060,"feed_emoji":"🔍","tokens_out":10311,"duration_ms":86615,"temperature":0.7,"pith_summary":"The paper claims that a multimodal large language model can improve its own fine-grained visual perception using no external labels, tools, rewards, or stronger teachers. It locates regions where cropping the image changes and sharpens the model's answer distribution, while blurring that same region leaves the full-image behavior almost unchanged; such regions are diagnosed as 'blind spots' where the model has perceptual ability it fails to use under full-image conditioning. CVPD turns each discovered region into dense token-level supervision, training the full-image student toward its own crop-conditioned answers and away from its ghost-conditioned default, with a KL anchor preserving general capabilities. On the 8-billion-parameter instruct model this yields gains of +3.60 on OCRBench, +3.38 on MMStar Fine-Grained Perception, and +3.08 on MMStar Logical Reasoning, with no regressions across twelve benchmarks.","feed_headline":"No labels, no rewards: model trains on its own blind spots","feed_subtitle":"It beats reward-based rivals on twelve benchmarks using only unlabeled images and its own answers.","key_machinery":"The central machinery is the three-gate Counterfactual Criterion (Equations 2–4) together with the contrastive self-distillation objective. A region $R$ is a blind spot when the Jensen–Shannon divergence between crop-conditioned and full-image distributions exceeds a threshold, the divergence between ghost-conditioned and full-image distributions stays below a threshold, and the crop-conditioned prediction has lower entropy than the full-image prediction. That selection step converts raw unlabeled images into a curated pool, and the training loss then makes the crop-conditioned distribution the positive teacher and the ghost-conditioned distribution the negative teacher at every token position, while a KL anchor to the frozen reference policy limits drift in general capabilities.","core_discovery":"CVPD's central assertion is that the three-gate Counterfactual Criterion identifies regions where the crop view and the ghost view are maximally opposed: the crop must diverge from the full image, the ghost must stay close to it, and the crop must lower predictive entropy. Regions passing all three gates are interpreted as visual blind spots where the model encodes information it does not consistently exploit under full-image conditioning, and the ghost view is a faithful proxy for its inattentive default. These regions form a curated pool of (image, question, region, answer) tuples, and training instantiates four policies from the same backbone: an online full-image student, a momentum crop-conditioned teacher, a momentum ghost-conditioned teacher, and a frozen reference policy. The combined loss transfers the crop teacher into the student, ranks the crop teacher above the ghost teacher with a margin, and anchors the student to the reference policy. On Qwen3-VL-8B-Instruct the method outperforms six self-evolving baselines on all twelve benchmarks without a single regression, including baselines that rely on an external annotator.","pith_inferences":["A natural extension the paper leaves implicit is an iterative loop: re-running blind-spot discovery on the trained student should surface harder regions, and one could test whether gains compound or plateau across multiple rounds.","Because the criterion uses only distributional divergence and entropy, it should transfer to other token-modalities: for example, treating a temporal window in audio or a patch in video as the 'crop' and time-masking or patch-blurring as the 'ghost.'","The method's premise that entropy sharpening equals perceptual correctness can be tested directly: label a sample of curated tuples and compare crop-conditioned accuracy against full-image accuracy; the framework could then be strengthened by a confidence-weighting scheme that down-weights confidently wrong crops."],"forward_implications":["Dense token-level visual supervision can be recovered from a model's own counterfactual responses, eliminating the need for external region proposals, segmentation systems, rewards, or stronger models.","The largest gains appear exactly on tasks requiring localized attention: OCRBench (+3.60), MMStar Fine-Grained Perception (+3.38), and MMStar Logical Reasoning (+3.08) at the 8B scale.","The contrastive ghost teacher is an active training signal, not just a filter: removing the ranking objective drops OCRBench by 2.30 points and MMStar Fine-Grained Perception by 2.16 points.","The Counterfactual Criterion is the main driver of improvement: replacing it with random regions causes the largest ablation drop, 2.60 points on OCRBench and 2.53 on MMStar Fine-Grained Perception.","The same pipeline transfers to the 4-billion-parameter model, improving every benchmark at both scales with no regression."],"supporting_citations":[{"why":"Defines on-policy distillation by matching a teacher on student-generated trajectories, the training regime CVPD adapts to the visual setting.","marker":"[1]"},{"why":"Supplies the multimodal base and instruct models on which CVPD is instantiated and evaluated.","marker":"[2]"},{"why":"Closest prior work: shows the regional-to-global perception gap and uses crop-conditioned on-policy distillation, but builds privileged context with external segmentation and annotators.","marker":"[37]"},{"why":"Represents group-relative reward-based self-improvement, the coarse-supervision approach CVPD is contrasted with.","marker":"[23]"},{"why":"A self-evolving multimodal baseline with continuous rewards that CVPD must outperform in the comparison.","marker":"[26]"},{"why":"A self-play baseline whose chart and real-world variants use an external annotator during dataset construction, serving as the external-supervision comparison point.","marker":"[30]"}],"fun_headline_variants":["Self-distills from its own blind spots, no labels needed","No rewards, no labels: model learns from its blind spots","Blind spot distillation: self-contained, beats external supervision","Model trains on counterfactual blind spots, no external tools"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a crop that makes the model more confident is revealing latent correct perception, since nothing in the pipeline checks that the sharper answer is actually the right one.","fun_headline_variants_meta":{"raw":{"variants":["Self-distills from its own blind spots, no labels needed","No rewards, no labels: model learns from its blind spots","Blind spot distillation: self-contained, beats external supervision","Model trains on counterfactual blind spots, no external tools"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000437,"raw_usage":{"total_tokens":2267,"prompt_tokens":1039,"completion_tokens":1228,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":655,"completion_tokens_details":{"reasoning_tokens":1158}},"tokens_in":655,"tokens_out":1228,"duration_ms":8812,"temperature":1.0,"reasoning_tokens":1158,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:12:34.590867+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a labeled subset of images, compare answer accuracy under the full image, the crop, and the ghost for regions that pass all three gates. If crop-conditioned answers are not systematically more accurate than full-image answers on these regions, Gate 3 is selecting confidence rather than capability and the central mechanism fails; if they are more accurate, the paper's premise holds.","supporting_citations":[{"cited_title":"On-policy distillation of language models: Learning from self-generated mistakes","cited_arxiv_id":null,"evidence_quote":"Defines on-policy distillation by matching a teacher on student-generated trajectories, the training regime CVPD adapts to the visual setting."}],"review_version":1}