{"id":"90a1d938-10eb-4aed-ae14-dcc09cb0b22e","arxiv_id":"2505.22039","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"OmniAD unifies industrial anomaly detection and understanding in a single multimodal model using text-encoded masks and reinforcement learning, reporting 79.1 on MMAD and strong detection scores.","lead":"OmniAD is a new AI system that detects defects in factory images and explains them in natural language, combining visual and textual reasoning in one model. It claims to beat much larger general-purpose models like GPT-4o on an anomaly understanding benchmark while also localizing defects without a manual threshold.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training on one MMAD example per category in Section 3.3 makes the Table 2 '0-shot' OmniAD result non-comparable with untuned baselines like GPT-4o, undermining the headline 79.1 claim.","rationale":"The reader's weakest_assumption is precisely the load-bearing concern: the paper's Section 3.3 reveals that one example per MMAD category was used in both SFT and GRPO training, and Table 2 labels the resulting evaluation as '0-shot'. Since the comparison models have no such training exposure, the 79.1 score is not a comparable zero-shot result. This is not a matter of disagreement with consensus; it is an internal inconsistency between the claim of '0-shot' and the training protocol. The paper discloses the data inclusion, which is good scientific practice, but disclosure does not make the comparison fair. The detection results on four external datasets are credible and could form the basis of a revised claim, but the abstract's headline—'surpassing models such as Qwen2.5-VL-7B and GPT-4o' on MMAD—is not substantiated. A straightforward retraining without MMAD examples would settle the magnitude of the inflation. My read does not change the reader's verdict; it reinforces it.","tokens_in":22727,"tokens_out":4042,"duration_ms":39276,"concrete_test":"Retrain OmniAD with the exact SFT+GRPO pipeline described in Section 3.3, but delete the 'one example per category from MMAD' from both the SFT and GRPO training sets. Evaluate on MMAD using the same inference settings and report the average accuracy. Compare against the reported 79.1 and against GPT-4o's 74.9 and Qwen2.5-VL-7B's 64.0. Repeat with three random seeds to assess variance. If the retrained model fails to exceed 74.9 (or its gain over Qwen2.5-VL-7B drops substantially), the contamination is material and the headline claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 (Data preparation) explicitly states: 'Following the approach in [9], we add one example per category from MMAD to avoid domain gaps between datasets.' This one example per category is included in both the SFT data (1.6K images) and the GRPO data (293 images). The model is then evaluated on the full MMAD benchmark, and Table 2 reports 'OmniAD(0-shot)' at 79.1. In contrast, all comparison models—GPT-4o, Qwen2.5-VL-7B, and the rest—were not fine-tuned on any MMAD examples. Thus the comparison is not zero-shot for OmniAD; the model has seen category-level examples from the evaluation benchmark during training. This can inflate the score by providing prior knowledge about the types of questions, anomaly classes, and object categories present in MMAD. The magnitude of inflation is unknown, but the burden is on the authors to show that removing these examples does not change the conclusion. Without such a control, the central claim of surpassing GPT-4o by 4.2 points is not fairly established. The detection experiments on MVTec-AD, VisA, MVTec-LOCO, and GoodsAD are independent of this issue and provide value, but they do not rescue the headline understanding result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OmniAD, a multimodal large language model framework that unifies industrial anomaly detection and anomaly understanding. Detection is performed via a Text-as-Mask encoding that converts anomaly segmentation into text generation, avoiding manual thresholds; understanding is performed through a Visual Guided Textual Reasoning step that produces a structured <seg>, <think>, <answer> output. The model is built on Qwen2.5-VL-7B and trained with an integrated SFT+GRPO pipeline using three reward functions. The authors report 79.1 average accuracy on the MMAD benchmark, surpassing GPT-4o (74.9) and the base Qwen2.5-VL-7B (64.0), and report strong threshold-free detection results on MVTec-AD, VisA, MVTec-LOCO, and GoodsAD. The training data includes one example per MMAD category in both SFT and GRPO stages, and the evaluation is performed on the full MMAD benchmark.","tokens_in":22953,"tokens_out":2819,"duration_ms":33420,"significance":"The proposed unification of detection and understanding in a single text-generation MLLM is a useful direction, and the Text-as-Mask encoding is a sensible adaptation of segmentation-as-text-generation to sparse, irregular anomaly masks. The detection results on the four external benchmarks, if reproducible, would be a meaningful step toward threshold-free few-shot anomaly detection. However, the headline understanding result on MMAD is compromised by the inclusion of MMAD training examples in the fine-tuning data, so the claimed superiority over GPT-4o is not currently established. The paper also gives no code, seeds, or error bars, which further limits confidence in the reported margins.","major_comments":[{"comment":"The training set includes one example per MMAD category in both the SFT data (1.6K images) and the GRPO data (293 images), yet Table 2 reports OmniAD(0-shot) as 79.1 on the full MMAD benchmark. All comparison models, including GPT-4o and Qwen2.5-VL-7B, were not fine-tuned on MMAD. This makes the headline comparison non-zero-shot for OmniAD and potentially inflates the result by providing category-level priors about the question types, anomaly classes, and object categories in MMAD. The authors must provide a control experiment that removes all MMAD examples from training, or evaluates only on held-out MMAD categories/instances, and report the resulting accuracy. Without such a control, the claim that OmniAD surpasses GPT-4o by 4.2 points is not supported.","section":"Section 3.3, Table 2"},{"comment":"All experimental results are reported as single runs with no code, random seeds, or error bars. The differences between OmniAD and baselines in Tables 3 and 4 are often small (e.g., 0.3-1.0 points in F1-score), and the improvement over AnomalyR1 in Table 2 is also within a plausible run-to-run variation for MLLM fine-tuning. Please report at least three seeds for the main MMAD result and the four detection benchmarks, or provide a significance test, so that the claimed improvements can be assessed statistically.","section":"Section 4.1, Tables 3-5"},{"comment":"The Detection Accuracy Reward defines a special case for normal images (FG = ∅), but the paper does not describe how the model is trained or encouraged to emit an empty <seg></seg> sequence for normal images, nor how empty predictions are parsed at inference. Since the Text-as-Mask encoding removes normal patches and relies on generating coordinates for anomaly patches only, the treatment of normal test images is critical for the threshold-free detection claim. Please clarify the training data construction for normal images and provide examples of empty-segmentation outputs and their reward assignment.","section":"Section 3.2, Eq. (1)"}],"minor_comments":[{"comment":"The method name is misspelled as 'AnomlyR1'; it should be 'AnomalyR1'.","section":"Table 2"},{"comment":"The phrase 'Detection results are report as' should be 'Detection results are reported as'.","section":"Tables 3-5 captions"},{"comment":"The data preparation paragraph says 'one example per category from several industrial anomaly detection datasets' but does not list which datasets; please enumerate them.","section":"Section 3.3"},{"comment":"The limitations paragraph mentions edge errors and inference efficiency but does not acknowledge the MMAD training/evaluation overlap; this should be disclosed and addressed.","section":"Section 5, Limitations"},{"comment":"The scale factor alpha in Eq. (1) is set to 2, but no sensitivity analysis is provided. Please either provide a small ablation or state that the result is insensitive to this value.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The central MMAD claim is not clean because of the training/evaluation overlap, but the issue is fixable with a control experiment. The detection benchmarks are external and appear reasonably conducted, though missing code and seeds reduce confidence. I would encourage the editor to request the control experiment and reproducibility details before further consideration. The paper's scope and contribution are otherwise appropriate for the venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the system integration is real and the detection experiments are worth a look, but the headline 79.1 on MMAD is not a clean zero-shot result because the model was fine-tuned on one example per MMAD category. That needs to be fixed before the understanding claims can be taken at face value.\n\nThe genuinely new piece is Text-as-Mask Encoding applied to anomaly detection, i.e., turning segmentation into a sparse RLE patch coordinate string and training the MLLM to emit it inside <seg> tags, then having the same model do Visual-Guided Textual Reasoning for the multiple-choice answer. The three-reward SFT+GRPO recipe is a reasonable engineering contribution, and the ablation study is honest: SFT+GRPO beats either alone, both rewards help, and the visual reasoning helps more than textual reasoning alone. The detection results on MVTec, VisA, LOCO, and GoodsAD are external, use standard protocols, and show strong threshold-free performance. That part looks solid.\n\nThe soft spot is the MMAD evaluation. Section 3.3 says they add one example per category from MMAD to both SFT and GRPO data. Then Table 2 reports OmniAD(0-shot) at 79.1, compared against GPT-4o and the base Qwen2.5-VL which were not fine-tuned on MMAD at all. That is not zero-shot; the model has seen category-level examples from the evaluation benchmark. The label is misleading and the 4.2-point advantage over GPT-4o is not fairly established. The authors should either retrain without MMAD examples and report that number, or explicitly rename the setting to something like \"MMAD-few-shot\" and compare against baselines given the same one-example-per-category data. Without that control, the central understanding claim is unsubstantiated as stated.\n\nMinor issues: no code or seeds released yet despite the abstract promise, no error bars or repeated runs, and the inference cost is acknowledged but not quantified. The writing is clear and the related work is accurate; the authors do cite AnomalyR1 and Text4Seg properly.\n\nBottom line: this is a useful system paper for people working on MLLM-based industrial inspection, with a credible detection component, but the headline understanding result needs a clean evaluation. I'd send it to review, but the reviewers should insist on the MMAD control.","headline":"Solid detection results and a neat text-as-mask integration, but the MMAD \"0-shot\" score is contaminated by training on the benchmark itself.","tokens_in":23524,"tokens_out":1880,"would_cite":false,"duration_ms":18202,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OmniAD unifies defect detection and explanation in a single vision-language model, reporting 79.1% average accuracy on MMAD as a 0-shot model.","keywords":["industrial anomaly detection","anomaly understanding","multimodal large language models","Text-as-Mask encoding","GRPO reinforcement learning","few-shot generalization","MMAD benchmark","threshold-free detection"],"falsifier":"Retrain OmniAD with all MMAD images removed from both SFT and GRPO, keeping the other training images and the same rewards, and re-evaluate on MMAD: if the average falls from 79.1 toward the 64.0 of the Qwen2.5-VL-7B base, the claimed few-shot understanding gain is an artifact of benchmark examples in training rather than of the multimodal-reasoning mechanism.","tokens_in":22455,"feed_emoji":"🔍","tokens_out":8036,"duration_ms":74889,"temperature":0.7,"pith_summary":"OmniAD sets out to prove that one vision-language model can do both halves of industrial inspection: pinpoint a defect and then explain what it is, why it matters, and how it affects the object. The paper claims that encoding anomaly segmentation as text, rather than as pixel masks or bounding boxes, lets the model detect defects without hand-picked thresholds, and that coupling supervised fine-tuning with GRPO reinforcement learning makes this work from very few examples. On the MMAD anomaly-understanding benchmark, the model reports 79.1 average accuracy, above GPT-4o's 74.9 and well above its Qwen2.5-VL-7B base's 64.0. On four anomaly-detection datasets (MVTec-AD, VisA, MVTec-LOCO, GoodsAD), it reports image-level F1 scores in the high 80s to 90s with no threshold selection. If true, this makes defect understanding a natural sequel to defect detection rather than a separate expert pipeline.","feed_headline":"One model spots and explains factory defects, beating GPT-4o","feed_subtitle":"OmniAD encodes defect masks as text and reports 79.1% accuracy on MMAD without hand-picked thresholds.","key_machinery":"The load-bearing mechanism is Text-as-Mask Encoding: the image is cut into fixed patches (24×24 in the experiments), normal patches are dropped, and the remaining anomalous patches are compressed row-wise with run-length encoding into a short string of coordinates such as '(2,4), (3,2)-(3,4)'. The model produces this string inside segmentation tags, turning segmentation into next-token prediction and thereby removing any external threshold. That textual mask then feeds Visual Guided Textual Reasoning, where the model must emit the segmentation, then a thinking step, then the answer, so localization and explanation share one generation pass. The second mechanism is the SFT+GRPO training recipe with three rewards (format, F1-based detection accuracy, and answer accuracy), and the ablations show that only the combined recipe yields the large reported gains.","core_discovery":"The paper's central claim is that anomaly understanding improves when detection is inside the language model instead of bolted on. OmniAD encodes the anomaly mask as a short run-length-encoded string of patch coordinates, so the model emits a segmentation string before it reasons, and then reasons with that segmentation in view; this visual guided textual reasoning is what lets the same 7B model both localize and explain. Training mixes SFT for format and domain knowledge with GRPO under three verifiable rewards, namely format, detection F1, and answer accuracy, and the paper reports that the combination is needed: SFT alone gives 69.8 average on MMAD, GRPO alone gives 55.7, and SFT+GRPO gives 79.1. The headline results are 79.1 average accuracy on MMAD in the 0-shot setting, 79.9 in the 1-shot setting, and an average image-level detection F1 of 92.2 in the 0-shot setting across the four detection datasets, all obtained without any threshold selection.","pith_inferences":["An untested consequence of the one-example-per-category MMAD data is that the 79.1 score may be a lower bound on what the method would achieve on genuinely unseen question formats; removing MMAD from training entirely would separate the mechanism's gain from benchmark-format leakage.","Because the mask is emitted as a compact text string, the same model could be extended to conversational inspection, asking follow-up questions about a defect and getting revised segmentation updates, without changing the architecture; the paper does not explore this.","The F1-based detection reward and the 24×24 patch grid are likely to interact with defect size; a testable extension is to measure how the patch count and scale factor affect localization of very small versus spread-out anomalies."],"forward_implications":["OmniAD performs anomaly segmentation and image-level detection in both 0-shot and 1-shot settings without threshold selection, reporting average image-level F1 of 92.2 (0-shot) and 92.0 (1-shot) across MVTec-AD, VisA, MVTec-LOCO, and GoodsAD.","Training only with SFT yields 69.8 average on MMAD and only with GRPO yields 55.7, while SFT+GRPO yields 79.1, so the paper's recipe, not either stage alone, carries the gain.","Ablations removing visual and textual reasoning drop average understanding from 79.1 to 68.0, with defect localization falling from 73.5 to 64.9, showing that emitting the mask before answering is what improves localization.","On structural and logical anomaly datasets MVTec-LOCO and GoodsAD, where CLIP-based baselines typically classify almost everything as normal, OmniAD reports image-level F1 around 90 to 93, indicating the text-as-mask approach generalizes beyond texture defects.","On MMAD, 0-shot OmniAD outperforms GPT-4o by 4.2 points and its own Qwen2.5-VL-7B base by 15.1 points, and it reaches or exceeds ordinary-human accuracy on Defect Analysis and Object Classification."],"supporting_citations":[{"why":"Defines the MMAD benchmark and its seven anomaly-understanding subtasks, and supplies the data-generation pipeline and human baselines used in evaluation.","marker":"[25]"},{"why":"Text4Seg supplies the segmentation-as-text-generation paradigm that Text-as-Mask Encoding adapts to sparse irregular anomaly masks.","marker":"[30]"},{"why":"Qwen2.5-VL-7B is the base MLLM that OmniAD fine-tunes, and its 64.0 MMAD score is the key baseline.","marker":"[3]"},{"why":"AnomalyR1 provides the prior GRPO-based anomaly MLLM and the one-example-per-category MMAD data practice that OmniAD follows.","marker":"[9]"},{"why":"Visual-RFT motivates the use of GRPO with verifiable rewards under limited data, the basis of OmniAD's training strategy.","marker":"[38]"},{"why":"Seg-ZERO verifies GRPO for reasoning segmentation, supporting the design of the detection accuracy reward.","marker":"[37]"},{"why":"MVTec-AD is one of the four detection datasets used to report threshold-free pixel- and image-level results.","marker":"[5]"},{"why":"MVTec-LOCO provides the structural and logical anomaly benchmark where OmniAD reports strong detection.","marker":"[4]"},{"why":"VisA is one of the four detection datasets used for the threshold-free evaluation.","marker":"[59]"},{"why":"GoodsAD provides the supermarket-goods anomaly benchmark included in the detection comparisons.","marker":"[53]"}],"fun_headline_variants":["OmniAD: Multimodal reasoner detects and explains industrial defects, beats GPT-4o","One model localizes and explains flaws, no thresholds, 79.1 on MMAD","OmniAD fuses vision and text for anomaly detection, outperforms GPT-4o","Detect and understand defects: OmniAD's mask-as-text trick","SFT+GRPO is key: OmniAD hits 79.1 on MMAD, beating GPT-4o"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on the assumption that including one example from every MMAD category in the training set does not materially teach the model the benchmark's question style, so that the 79.1 score is a fair zero-shot comparison against models that were not trained on MMAD.","fun_headline_variants_meta":{"raw":{"variants":["OmniAD: Multimodal reasoner detects and explains industrial defects, beats GPT-4o","One model localizes and explains flaws, no thresholds, 79.1 on MMAD","OmniAD fuses vision and text for anomaly detection, outperforms GPT-4o","Detect and understand defects: OmniAD's mask-as-text trick","SFT+GRPO is key: OmniAD hits 79.1 on MMAD, beating GPT-4o"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000575,"raw_usage":{"total_tokens":2716,"prompt_tokens":949,"completion_tokens":1767,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":1646}},"tokens_in":565,"tokens_out":1767,"duration_ms":11992,"temperature":1.0,"reasoning_tokens":1646,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:16:49.055063+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain OmniAD with all MMAD images removed from both SFT and GRPO, keeping the other training images and the same rewards, and re-evaluate on MMAD: if the average falls from 79.1 toward the 64.0 of the Qwen2.5-VL-7B base, the claimed few-shot understanding gain is an artifact of benchmark examples in training rather than of the multimodal-reasoning mechanism.","supporting_citations":[{"cited_title":"MMAD: A comprehensive benchmark for multimodal large language models in industrial anomaly detection","cited_arxiv_id":null,"evidence_quote":"Defines the MMAD benchmark and its seven anomaly-understanding subtasks, and supplies the data-generation pipeline and human baselines used in evaluation."},{"cited_title":"Beyond dents and scratches: Logical constraints in unsupervised anomaly detection and localization.International Journal of Computer Vision, 130(4):947–969, 2022","cited_arxiv_id":null,"evidence_quote":"MVTec-LOCO provides the structural and logical anomaly benchmark where OmniAD reports strong detection."},{"cited_title":"Spot-the-difference self-supervised pre-training for anomaly detection and segmentation","cited_arxiv_id":null,"evidence_quote":"VisA is one of the four detection datasets used for the threshold-free evaluation."},{"cited_title":"Pku-goodsad: A supermarket goods dataset for unsupervised anomaly detection and segmentation.IEEE Robotics and Automation Letters, 9(3):2008– 2015, 2024","cited_arxiv_id":null,"evidence_quote":"GoodsAD provides the supermarket-goods anomaly benchmark included in the detection comparisons."}],"review_version":1}