{"id":"fcbad4e8-488b-4b7c-a6ca-e1162a476a04","arxiv_id":"2608.09789","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":11,"one_line_summary":"ADOPD distills a reference-aware teacher's comparison skills into a query-only student, reaching 77.31% zero-shot accuracy on MMAD, up 6.14 points over Qwen3-VL-4B.","lead":"This paper trains a vision-language model to spot industrial defects without needing a reference image at test time, by using reference images only during training. The method, ADOPD, reports a 77.31% average accuracy on the MMAD benchmark, beating its base model by 6.14 points.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reference-support score s_g (Eq. 9) is an unvalidated proxy; a within-prompt permutation of s_g would test whether the 2.36-point gain over uncalibrated OPD is actually due to reference-specific weighting.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: s_g (Eq. 9) is an unvalidated proxy for reference-specific support, and the sequence weight w_g (Eq. 10) could therefore rescale rollouts for reasons unrelated to the query-reference comparison. This is the crux of the paper's novelty because the only difference between ADOPD and reference-privileged OPD is this calibration, and the claimed 2.36-point gain is attributed to it. If the proxy is confounded, the central mechanistic conclusion is unsupported even though the raw 77.31% accuracy may still hold. I considered other issues such as the mismatch between the one-shot normal reference and the matched same-state reference used in training, and the lack of error bars or released code; these are real caveats but they do not strike at the specific technical contribution as directly as the validity of s_g. The proposed permutation test is a clean, low-cost experiment that isolates whether the per-rollout pairing of s_g is causally necessary for the gain. Given the paper already includes functional ablations that partially support the calibration, the appropriate verdict remains CONDITIONAL, unchanged from the reader's assessment.","tokens_in":12386,"tokens_out":16524,"duration_ms":146468,"concrete_test":"Run the ADOPD pipeline exactly as described, but within each prompt permute the computed s_g values across the G=8 rollouts before applying Eq. 10, so that w_g is paired with a support score from a different rollout. Keep everything else identical. Compare the resulting MMAD average accuracy to the reported 77.31% with correct pairing and to the 74.95% uncalibrated OPD. If the permuted version retains most of the 2.36-point gain, the weight is not using reference-specific per-rollout information and the proxy concern lands. If the gain collapses to the OPD level, the specific s_g-rollout pairing is causally responsible, supporting the mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ADOPD internalizes reference-comparison behavior rests on the contrastive reference weight w_g (Eq. 10), which is built from s_g (Eq. 9), the mean matched-minus-mismatched teacher log-likelihood gap over a student rollout. Table 2 shows this calibration adds 2.36 points over reference-privileged OPD (74.95 -> 77.31). The load-bearing problem is that s_g is never validated as a measure of genuine query-reference grounding. It is a teacher log-likelihood difference on student-sampled tokens, so it can be inflated by crop-specific artifacts, red-bounding-box cues, or language priors that correlate with token content but are not about the validity of the reference relation. The within-prompt standardization in Eq. 10 removes constant offsets, but it does not remove token-dependent confounds. The Table 3 ablation comparing mismatched crop to random crop shows that the identity of the control image matters, but it does not establish that s_g separates good from bad rollouts for the right reason. If s_g is driven by an unrelated signal, the sequence weight rescales rollouts for the wrong reason, and the demonstrated gain over OPD does not support the paper's mechanistic claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ADOPD, a reference-privileged on-policy distillation method for MLLM-based industrial anomaly detection. A query-only student samples rollouts, and a frozen reference-conditioned teacher scores the same tokens under a matched reference, producing a token-level advantage, while a mismatched reference provides a sequence-level contrastive weight. The weighted policy-gradient objective is trained on 6,000 paired Real-IAD records. On MMAD, ADOPD reports 77.31% zero-shot average accuracy, improving the Qwen3-VL-4B backbone by 6.14 points and exceeding the backbone's one-shot setting by 2.64 points. Ablations compare SFT, uncalibrated OPD, reference granularity, and teacher construction.","tokens_in":12673,"tokens_out":6839,"duration_ms":61975,"significance":"If the central claim is fully supported, ADOPD would be a notable demonstration that the benefit of inference-time reference comparison can be amortized into model parameters, with a zero-shot student surpassing a one-shot teacher-conditioned baseline. The OPD derivation in Section 2 and Appendix A is internally consistent, and the matched/mismatched contrastive design is well motivated. However, the paper's mechanistic conclusion rests on an unvalidated proxy, the sequence-level support score s_g, and the experimental evidence is reported without variance estimates. These issues make the current version prematurely conclusive.","major_comments":[{"comment":"The central mechanistic claim that ADOPD's 2.36-point gain over uncalibrated OPD comes from reference-specific weighting is not yet supported because s_g is never validated as a measure of genuine query–reference grounding. s_g is a teacher log-likelihood gap on student-sampled tokens; it can be inflated by crop artifacts, red-bounding-box cues, or language priors that are unrelated to the validity of the reference relation. The within-prompt standardization in Eq. (10) removes constant offsets but not token-dependent confounds. A minimal check would be a within-prompt permutation test: randomly permute the weights w_g across the G rollouts of the same prompt and show that the calibration gain in Table 2 degrades or disappears. Additionally, s_g could be correlated with rollout quality, for example by comparing each rollout against the reference-conditioned teacher's answer or against the audit metadata answer. Without such a test, the observed gain could come from an unrelated mechanism, and the paper's central interpretation remains an assertion.","section":"§3.2, Eqs. (9)–(10), Table 2"},{"comment":"The claim that w_g is non-negative and therefore never reverses the matched-reference OPD direction is not implied by Eq. (10). Since w_g = s_g [1 + clip((s_g − μ)/σ, −ε_w, ε_w)] and the bracketed factor is non-negative for ε_w ≤ 1, w_g becomes negative whenever s_g is negative. The paper does not report whether s_g is always positive in the 60 training steps. If negative s_g occurs, the sequence weight reverses the token-level advantage, contradicting the stated design. The authors should either use a strictly positive transformation of s_g, such as |s_g| or exp(s_g), or provide empirical evidence that s_g is always non-negative in their runs.","section":"§3.2, Eq. (10), sentence 'the weight w_g is detached and non-negative'"},{"comment":"The EMA-teacher ablation does not isolate teacher construction. The student policy is query-only, so an EMA copy of the student cannot provide the matched/mismatched reference-conditioned distributions used in ADOPD unless the same reference conditioning is also fed to the EMA model. If the EMA teacher is evaluated on student rollouts without any reference input, then this row changes both the teacher construction and the availability of privileged references, and the comparison to the 32B reference-aware teacher is confounded. The text should state explicitly what inputs the EMA teacher receives; otherwise the 74.96 vs. 77.31 comparison cannot be attributed to teacher construction.","section":"§4.3, Table 4, '4B EMA' row"},{"comment":"All quantitative claims are based on single runs with no variance estimates. Several headline differences are small relative to typical MLLM training noise: ADOPD exceeds Reason-IAD by 0.60 points in Table 1, SFT exceeds uncalibrated OPD by 0.75 points in Table 2, and the random-crop vs. no-image contrast in Table 3 is 0.71 points. With a single run, these differences may not be statistically reliable. The authors should report mean and standard deviation over at least three seeds, or provide a significance test, for the main comparison and the ablations that support the core claims.","section":"§4.1–§4.3, Tables 1–4"}],"minor_comments":[{"comment":"The teacher input protocol is inconsistent: Section 3.1 states that the teacher receives either a matched or a mismatched reference, while Appendix B and Table 5 show a teacher raw prompt containing both <teacher image> and <teacherneg image>. Please clarify whether the two reference views are implemented by replacing the image in a single prompt or by masking one of the two images, and whether the teacher ever sees both references simultaneously.","section":"§3.1 and Appendix B, Table 5"},{"comment":"The phrase 'outperforming its one-shot setting by 2.64 points' is ambiguous because ADOPD does not have a one-shot setting; the comparison is against the Qwen3-VL-4B backbone evaluated one-shot. Please rephrase to 'outperforming the backbone's one-shot setting'.","section":"Abstract and §4.2"},{"comment":"The statement 'as no answer is involved, s_g is a verifier-free proxy' is misleading: s_g is computed over the student-generated rollout, which includes answer tokens. The intended meaning is presumably that no ground-truth answer label is used. Please clarify the wording.","section":"§3.2, Eq. (9)"},{"comment":"The per-model annotations '+3.31', '+3.56', '+1.25', and '+5.18' are not explained in the caption or text. It should be stated what these deltas represent and whether they are one-shot minus zero-shot accuracy differences.","section":"Figure 1(a)"},{"comment":"Several references are cited to arXiv preprints dated 2026; this is acceptable for a preprint, but the list should be checked for consistency with the ICLR-style formatting, particularly the use of 'LLaV A-NeXT' and the VeRL citation.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper has a sound derivation and an interesting experimental setup, but the central mechanistic interpretation is currently under-supported. The missing validation of s_g is the most important issue: without a permutation test or a correlation with rollout quality, the 2.36-point calibration gain could be explained by a confound. The single-run reporting is also a serious concern for a benchmark paper. I would be willing to reconsider after the authors add the missing control experiments and variance estimates."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nADOPD is a genuine extension of on-policy distillation to reference-privileged industrial anomaly detection, and the matched/mismatched reference control is a real idea. The central claim—77.31% zero-shot accuracy on MMAD with Qwen3-VL-4B, +6.14 over the backbone and +2.64 over its one-shot counterpart—is evaluated on an external benchmark, and no parameter is fitted to MMAD. The training data come from Real-IAD with no overlap. That much holds up.\n\nThe new combination is the token-level OPD direction from the matched reference plus a sequence-level weight from the matched-minus-mismatched teacher log-likelihood gap, standardized within each prompt's rollout group. Appendix A's derivation is clean, and the ablation in Table 3 is the best evidence for the mechanism: mismatched crop (77.31) beats random crop (75.80) beats no second image (75.09). That pattern is consistent with the claim that the weight depends on the validity of the query-reference relation.\n\nThe soft spots are real but addressable. The key proxy s_g—the mean matched-minus-mismatched teacher likelihood gap on a student rollout—is asserted to measure reference-specific support, but it is never validated. It could be inflated by crop artifacts, the red bounding box, or language priors. The within-prompt standardization removes constant offsets, not token-dependent confounds. A permutation test that shuffles s_g across rollouts, or a comparison against random weighting, would test whether the 2.36-point gain over uncalibrated OPD actually comes from reference-specific calibration. Without that, the mechanistic story is plausible but not proven.\n\nSecond, every table reports single runs with no variance. The improvements over Reason-IAD (0.60) and over uncalibrated OPD (2.36) are modest enough that run-to-run noise could shift the conclusions. Third, the paper cites OPD-IAD and V-Zero as closely related but does not compare against them in the main tables. Direct numbers would help position the contribution. Code and data are promised but not yet available; the main numbers should be reproducible once released.\n\nThe central argument holds up. This is a training-recipe paper with a clean idea, a good ablation, and an unvalidated proxy that needs one more experiment. It is worth a serious referee, not a desk reject.","headline":"A clean training-recipe paper with a real mechanism and an unvalidated proxy; worth refereeing, not desk-rejecting.","tokens_in":13250,"tokens_out":4673,"would_cite":true,"duration_ms":35883,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ADOPD internalizes reference comparison into a query-only model, gaining 6.14 points on MMAD.","keywords":["industrial anomaly detection","multimodal large language model","on-policy distillation","knowledge distillation","reference comparison","privileged information","zero-shot","MMAD benchmark"],"falsifier":"Train ADOPD with the mismatched reference replaced by a matched reference from a different instance of the same product class, keeping the prompt and crop procedure identical; if accuracy stays at the matched/mismatched level, the support score is not measuring reference-validity discrimination.","tokens_in":12163,"feed_emoji":"🏭","tokens_out":4719,"duration_ms":36674,"temperature":0.7,"pith_summary":"ADOPD asks whether an industrial anomaly detector based on a multimodal language model can absorb the benefit of comparing a query image against a normal reference during training, so that no reference is needed at deployment. The paper's answer is yes: a teacher that sees both a matched and a deliberately mismatched reference supervises a student that sees only the query image, and the student learns to inspect fine-grained defects on its own. On the MMAD benchmark, the resulting query-only student reaches 77.31% average zero-shot accuracy, improving its Qwen3-VL-4B backbone by 6.14 points and beating the one-shot reference-conditioned version by 2.64 points. The gain concentrates in comparison-sensitive subtasks such as anomaly discrimination and defect localization, which is the signature of learned reference comparison rather than generic supervision.","feed_headline":"Training with references beats using them at test time","feed_subtitle":"ADOPD's query-only student hits 77.31% zero-shot accuracy on MMAD, beating its one-shot reference-conditioned baseline by 2.64 points.","key_machinery":"The load-bearing component is the contrastive reference support score $s_g = \\frac{1}{T_g}\\sum_{t=1}^{T_g}[\\log p_T^\\mathrm{mat}(y_t) - \\log p_T^\\mathrm{mis}(y_t)]$, the average teacher log-likelihood gap between the matched and mismatched reference views on a student-generated rollout. This score is standardized within the group of rollouts for the same prompt to produce the sequence-level weight $w_g = s_g[1 + \\mathrm{clip}((s_g - \\mu)/\\sigma, -\\epsilon_w, \\epsilon_w)]$, which scales the matched-reference token advantage $A_g^t = \\mathrm{sg}[\\log p_T^\\mathrm{mat}(y_t) - \\log p_S(y_t)]$ before a policy-gradient update. The matched view sets the learning direction; the mismatched view calibrates how strongly to trust that direction without ever reversing it, because the weight is detached and non-negative.","core_discovery":"The paper establishes that the benefits of inference-time reference comparison can be internalized into model parameters through a distillation scheme that separates what to learn from how strongly to learn it. A frozen reference-aware teacher evaluates student-generated rollouts under two asymmetric views: a matched regional reference that preserves the valid query–reference relation, and a mismatched reference that deliberately breaks it. The matched-view teacher–student log-ratio gives the token-level learning direction, while the matched–mismatched likelihood gap provides a sequence-level weight that calibrates how much each rollout should contribute. The student, which sees only the query image and question, thereby learns an anomaly inspection strategy that at zero-shot inference outperforms the same backbone with explicit one-shot reference access on the MMAD benchmark.","pith_inferences":["Because ADOPD improves recall sharply (+23.95 points over one-shot) at some precision cost, the learned strategy may be a more aggressive detector; future work could tune the reference weight to trade false alarms against missed defects.","The matched/mismatched contrast suggests a general recipe for privileged-information distillation: any teacher-side condition whose removal should change the answer can calibrate update strength, and text-only or object-detection privileged signals could be tried outside industrial anomaly detection.","The support score could be evaluated explicitly against human-labeled reference usefulness, or by ablations that corrupt only the reference region while keeping the prompt identical, to test whether the weight tracks genuine reference validity rather than low-level artifacts.","Since ADOPD still depends on paired images and defect annotations, automatically constructing weak references, such as anomaly-free patches, is a direct next step the paper itself flags."],"forward_implications":["Zero-shot MLLM anomaly detection can match or exceed explicit one-shot reference use on average accuracy, while removing retrieval and reference processing at inference time.","The training signal is not generic visual supervision: gains cluster in anomaly discrimination, defect localization, and defect analysis, and regional reference crops outperform full-image references.","On-policy distillation with a reference-privileged teacher beats off-policy SFT on the same 6K-record dataset, and the contrastive reference weight adds a further gain over uncalibrated OPD.","The method transfers to a 2B student and to an EMA self-teacher, though the gains are larger with a 4B student and an external 32B teacher."],"supporting_citations":[{"why":"Supplies the MMAD benchmark, the seven subtasks and average accuracy used for all main evaluations.","marker":"(Jiang et al., 2025)"},{"why":"Supplies Real-IAD, the source of the 6,000 paired training records with matched and mismatched references.","marker":"(Wang et al., 2024)"},{"why":"Supplies the Qwen3-VL model family, used for the 4B student and 32B teacher default configuration.","marker":"(Bai et al., 2025a)"},{"why":"Defines on-policy distillation and the stop-gradient token advantage on student-generated prefixes that ADOPD builds on.","marker":"(Lu & Lab, 2025)"},{"why":"Represents the SFT-style reference-conditioned distillation baseline that ADOPD compares against in the training-paradigm ablation.","marker":"(Gu et al., 2024)"},{"why":"Contextualizes the one-shot reference-conditioned inference performance that ADOPD seeks to surpass without references at test time.","marker":"(Jiang et al., 2026)"}],"fun_headline_variants":["Distill references in, drop retrieval at test","Zero-shot ADOPD outdoes its one-shot self on MMAD","Query-only student beats reference-based MLLM","ADOPD: Zero-shot wins over one-shot IAD"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The contrastive support score $s_g$ truly measures whether the teacher's preference is grounded in the query–reference comparison; if it mainly tracks crop artifacts, prompt wording, or language priors, the weighting rescales rollouts for the wrong reason.","fun_headline_variants_meta":{"raw":{"variants":["Distill references in, drop retrieval at test","Zero-shot ADOPD outdoes its one-shot self on MMAD","Query-only student beats reference-based MLLM","ADOPD: Zero-shot wins over one-shot IAD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00025,"raw_usage":{"total_tokens":1560,"prompt_tokens":956,"completion_tokens":604,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":537}},"tokens_in":572,"tokens_out":604,"duration_ms":6238,"temperature":1.0,"reasoning_tokens":537,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:45:38.855841+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train ADOPD with the mismatched reference replaced by a matched reference from a different instance of the same product class, keeping the prompt and crop procedure identical; if accuracy stays at the matched/mismatched level, the support score is not measuring reference-validity discrimination.","supporting_citations":[],"review_version":1}