{"id":"bbb04aae-8b3d-4e71-870d-57a308ad58da","arxiv_id":"2411.16110","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"FUN-AD trains an anomaly detector without any labels by using nearest-neighbor distances in a self-refining memory bank to pseudo-label normal and anomalous patches.","lead":"FUN-AD is a method for training industrial defect detectors on completely unlabeled images that may contain defects. It uses distances between image features to guess which samples are normal, then trains a small network that both finds defects and localizes them.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim is undermined by oracle model selection: the supplementary implementation details state 'we train for 1500 epochs and choose the model with the best average of image-wise and pixel-wise AUROCs,' i.e., test labels are used to pick the checkpoint per class, which is unavailable in the…","rationale":"I considered the reader's declared weakest assumption, the Sec. 3.1 variance-ordering condition, and the related incompleteness of Eq. (5). Both are genuine weaknesses: the analytic argument only establishes two special cases, and the real-data validation is limited to the bottle class, with the paper's own limitations section conceding failure when normal diversity is comparable to anomaly diversity. However, these are limitations of the method's scope rather than invalidations of the reported experiments. The epoch-selection issue is more load-bearing because it attacks the empirical claim directly: under the paper's stated protocol, test labels are used during model selection, so the numbers in Table 1 and Table 2 cannot be taken as evidence of fully unsupervised SOTA performance. The released code is a plus, and the ablations are informative, but they do not repair the protocol flaw. Since my primary concern matches the reader's rationale although not their declared weakest_assumption, I mark agreement as partial and keep the verdict at reject, i.e., UNCHANGED.","tokens_in":22496,"tokens_out":6385,"duration_ms":62208,"concrete_test":"Run the released code on the MVTec AD no-overlap, 10%-contamination setting with all hyperparameters unchanged, but replace the per-class test-AUROC checkpoint selection with the final epoch after the fixed 1500-epoch schedule. Recompute the mean image-wise and pixel-wise AUROC and compare them with Table 1 and with SoftPatch (98.3/97.3) and RealNet (99.2/96.9). If the average falls below these baselines, or if a paired per-class comparison shows no significant advantage, the claimed SOTA under fully unsupervised conditions is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is the evaluation protocol used for the headline results. In the supplementary implementation details, the authors write: 'For each object/texture class, we train for 1500 epochs and choose the model with the best average of image-wise and pixel-wise AUROCs.' Because MVTec AD and VisA have no validation split, this means the test labels are used to choose the checkpoint, separately for every class. FUN-AD is a self-training loop whose anomaly-score quality changes across epochs, so oracle epoch selection can materially inflate the reported numbers. In the claimed fully unsupervised deployment, no such selection is possible. The baselines in Table 1 are not awarded an equivalent per-class test-set selection, so the 'state-of-the-art' comparison is not a fair test of the method. This flaw directly undermines the strongest claim in the abstract; even if the algorithm is sound, the paper has not demonstrated SOTA performance under the setting it claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FUN-AD, a fully unsupervised anomaly detection method for training sets that are unlabeled and potentially contaminated with anomalies. The method rests on two statistical observations: (i) normal-normal feature pairs tend to have smaller pairwise distances than other pair types, and (ii) mutually closest feature pairs tend to be class-homogeneous. Building on these, the authors design an iteratively reconstructed memory bank (IRMB) that stores features judged normal by the Local-Net, a nearest-neighbor distance based pseudo-labeling scheme, and a mutual smoothness loss that aligns the anomaly scores of mutually closest pairs. Experiments on MVTec AD and VisA with several contamination ratios report state-of-the-art image-wise and pixel-wise AUROC numbers, with ablations of the loss components and hyperparameters. The paper also includes a semantic anomaly detection example on CIFAR-10/STL-10 and releases code.","tokens_in":22661,"tokens_out":6924,"duration_ms":61453,"significance":"If the reported results are obtained under a genuinely unsupervised protocol, the contribution is significant: it would show that industrial anomaly detection can be trained on raw, unlabeled, possibly contaminated image collections without a clean normal-only training set. The paper offers a clear motivation, a simple and well-described framework, a solid set of ablations, and publicly available code, which are strengths. The key weakness is the evaluation protocol: the supplementary implementation details state that the best epoch is selected using test-set AUROC, which invalidates the fully unsupervised claim and makes the head-to-head comparisons with baselines unfair. This issue is load-bearing for the abstract and Section 5.2 SOTA claims, so the experimental section must be redone before the central claim can be accepted.","major_comments":[{"comment":"The paper selects the best training epoch per class using test labels: 'For each object/texture class, we train for 1500 epochs and choose the model with the best average of image-wise and pixel-wise AUROCs.' Since MVTec AD and VisA have no validation split, this is an oracle model-selection protocol that uses test labels. The reported numbers in Tables 1 and 2 are therefore not achievable in the claimed fully unsupervised deployment, and baselines are not given the same oracle selection. This directly undermines the 'state-of-the-art' claim in the abstract and Section 5.2. Please re-run all main experiments with a fixed training schedule (e.g., a single predefined epoch or a validation-based criterion) and report both the oracle and non-oracle numbers, and state whether the ranking against baselines is preserved.","section":"Supplementary Implementation details"},{"comment":"The claim that Eq. (5) is 'always greater than 1' is justified only by two special cases: equal means, and different means with similar variances. The sentence 'In practice, the normal and anomaly data usually have different means and variances to safely go over 1' is not a proof. Although a small-τ asymptotic expansion of the chi-square and non-central chi-square CDFs would establish the claim, the text as written overstates the result. Please replace the intuitive argument with the actual asymptotic expansion, or explicitly restrict the claim to the τ ≪ 1 regime.","section":"Sec. 3.1, Eq. (5)"},{"comment":"The central assumption of the paper, namely that normal features have smaller variance than anomaly features, is empirically validated on real data only for the bottle class of MVTec AD (supplementary Fig. 1). The matching-ratio plots of Fig. 2 are reported as MVTec averages, but per-class results are not shown, so the reader cannot tell whether the assumption holds for classes with high normal diversity or a dominant anomaly type. The limitations section itself concedes that the method 'may be compromised if the feature diversity of the normal data is comparable to that of the anomalies.' Please provide per-class validation of the distance ordering or the matching ratio, or list the classes where the assumption is violated and report how the method behaves on them.","section":"Sec. 3.2 and supplementary Sec. 1"},{"comment":"The training loop is self-referential: the memory bank is constructed from the Local-Net's own anomaly scores using Eq. (6), distances to that bank are used to assign pseudo-labels (Eqs. (7)-(8)), and the same network is trained with those labels. The paper states that the iterative process 'will' leave only normal features in the bank, but no evidence is provided to show that pseudo-label accuracy or memory-bank purity improves over training. In a fully unsupervised setting, there is no external check against confirmation bias. Please include a plot of pseudo-label accuracy (or memory-bank purity) as a function of training iterations, or otherwise provide direct evidence that the loop converges to a better solution rather than amplifying initial errors.","section":"Sec. 4.1 / Algorithm 1"}],"minor_comments":[{"comment":"The statement in the abstract and Section 5.2 that FUN-AD achieves SOTA 'across various contaminated settings' is not supported for the 0% contamination setting in Table 2, where SoftPatch outperforms FUN-AD on MVTec AD (98.32 vs. 93.33 image AUROC). Please either qualify the SOTA claim to nonzero contamination ratios or explain why the 0% case is considered out of scope.","section":"Sec. 5.2 / Table 2"},{"comment":"Algorithm 1 line 8 says 'Construct M using ϕ, Eq. (6)' but does not specify how often the memory bank is rebuilt, whether it uses the full training set or a random subset, and what the bank size is. The supplementary discusses a sampling ratio but not the memory-bank capacity. Please provide these details for reproducibility.","section":"Algorithm 1 / Sec. 4.1"},{"comment":"For the MVTec AD contamination setting, the text says anomalies are randomly incorporated at a 1:10 ratio, but the exact number of training images per class and the random seed(s) are not reported. Please state the dataset statistics and the number of random trials used to obtain the averages in Tables 1 and 2.","section":"Sec. 5.2 / dataset setup"},{"comment":"The min-max normalization in Eq. (8) uses the global min and max over the mini-batch. At the beginning of training, when the Local-Net is random, these values may be degenerate or unstable. Please comment on how the normalization behaves at the first iteration and whether any numerical safeguards are used.","section":"Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue is the oracle model selection in the supplementary implementation details. If the authors re-run the experiments without test-set-based checkpoint selection and the non-oracle numbers still place FUN-AD at or near the top of Tables 1 and 2, the paper would make a solid contribution. The self-referential training loop is a standard self-training pattern and, in my view, is not automatically disqualifying; it needs empirical safeguards rather than a conceptual rejection. I would not recommend acceptance until the evaluation-protocol problem is resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the method is a genuinely new self-training design for fully unsupervised anomaly detection, but the paper's central empirical claim is compromised by an evaluation protocol that uses test labels to pick the best checkpoint for every class. As presented, the SOTA claim does not hold.\n\nWhat is actually new: IRMB iteratively rebuilds the memory bank from the model's own scores, nearest-neighbor distances to that bank drive patch-level pseudo-labels, and the mutual smoothness loss aligns anomaly scores of mutually closest pairs. These pieces are not in SoftPatch or InReaCh. The ablations show each component contributes, the code is public, and the CIFAR-10 toy example is a reasonable sanity check. That is real work and worth building on.\n\nThe math in Sec. 3.1 is better than the reader's note suggests. Eq. (5) is true for all tau when sigma_N < sigma_A: the noncentral chi-square CDF is bounded above by the central chi-square CDF, and the central argument for normal-normal pairs is larger. The authors' two-case explanation is just a weak way to present it, not a false claim. The genuinely fragile part is the variance-ordering assumption, which is validated on real data only for the bottle class and is conceded in the limitations. That is a real limitation but not a fatal one.\n\nThe stress-test note is correct and it lands. The supplementary implementation details say: \"we train for 1500 epochs and choose the model with the best average of image-wise and pixel-wise AUROCs.\" MVTec and VisA have no validation split, so this is oracle selection on test labels, separately per class. A fully unsupervised deployment cannot do that, and the baselines are not given the same advantage. This directly inflates the numbers in Table 1 and the abstract's \"state-of-the-art\" claim. It is a load-bearing flaw, not a cosmetic one.\n\nThe self-training loop is not a logical circularity in the objectionable sense; iterative pseudo-labeling with a memory bank is a coherent strategy. It does mean early errors can reinforce, which is consistent with the observed weakness at 0-1% contamination rates.\n\nWho this is for: people working on fully unsupervised anomaly detection and noisy-label learning. It is also a useful cautionary example of how oracle checkpoint selection can invalidate a comparison. I would not desk-reject this; I would send it to a serious referee and require the authors to re-run without test-set selection, using a fixed training schedule or a validation-based selection, and re-report. If the results survive that correction, the paper has value. As presented, I would not accept it.","headline":"Clever self-training framework whose headline SOTA claim is not supported because the reported numbers are selected with test-set AUROC oracle checkpoint selection per class.","tokens_in":23212,"tokens_out":3062,"would_cite":false,"duration_ms":31136,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An industrial anomaly detector can be trained entirely on unlabeled, potentially contaminated images, without a clean-normal set, by exploiting pairwise feature distance statistics.","keywords":["fully unsupervised anomaly detection","contaminated training data","pseudo-labeling","pairwise distance statistics","iteratively reconstructed memory bank","mutual smoothness loss","industrial defect detection","anomaly localization"],"falsifier":"Construct or select a category in which the anomaly distribution is narrow and close to the normal cluster while the normal distribution is broad (for example a high-variation object class with one dominant defect type, or an artificially spiked normal distribution). Run FUN-AD with 10% contamination and compare against a clean-data one-class baseline; if image-wise AUROC falls to near the baseline's contamination level or below, and the pairwise-distance histograms show normal-normal distances no longer dominating at small radii, the central ordering claim is falsified. The paper's own limitation section concedes this failure mode.","tokens_in":22224,"feed_emoji":"🔍","tokens_out":6552,"duration_ms":57440,"temperature":0.7,"pith_summary":"FUN-AD targets a practical problem: training an industrial defect detector on completely unlabeled images, where the training set may quietly contain defects and there is no curated set of clean normal images. The paper's central claim is that two statistical regularities in a pretrained feature space are enough to get started: feature pairs from normal samples tend to be closer than pairs involving anomalies, and mutually closest feature pairs tend to belong to the same class. These regularities hold when the normal features are less spread out than the anomaly features, a condition the paper argues is typical in industrial settings. The paper converts them into a working pipeline—iteratively reconstructed memory bank, nearest-neighbor pseudo-labeling, and a mutual smoothness loss—and reports top detection and localization results on MVTec AD and VisA across contamination levels. If the claim is right, clean-data one-class training becomes unnecessary for many factory inspection tasks.","feed_headline":"No labels, no clean data: unsupervised defect detector beats baselines","feed_subtitle":"FUN-AD trains on unlabeled, contaminated industrial images and still leads MVTec AD and VisA.","key_machinery":"The load-bearing identity is the ordering of small-distance probabilities: under the variance assumption σ_N < σ_A, the cumulative distribution of ∥x1−x2∥ for normal-normal pairs dominates that of anomaly-anomaly and mixed pairs at small radii, which the paper derives from chi-square and non-central chi-square CDFs. The corresponding empirical fact is that mutually closest pairs are homogeneous with high probability. These two facts are operationalized by three components: an iteratively reconstructed memory bank (a bank rebuilt each iteration from features the current model scores as normal, passed through a learnable adaptor), a nearest-neighbor distance that is min-max normalized into anomaly scores and thresholded by τ_n and τ_c to make hard pseudo-labels and an ambiguous band, and a mutual smoothness loss—an L1 penalty on anomaly scores of mutually closest pairs—that makes the pseudo-labeling robust to threshold errors. Gaussian perturbation of ambiguous features closes the loop by preventing confident normal features from being pushed into the anomaly class.","core_discovery":"The discovery is that pairwise distance statistics in a pretrained feature space can carry the entire learning signal for fully unsupervised anomaly detection. For isotropic Gaussian normal and anomaly distributions with σ_N < σ_A, the paper proves that within a small distance threshold a normal-normal pair is more likely than an anomaly-anomaly or mixed pair, using chi-square and non-central chi-square cumulative distributions. It further validates empirically that mutually nearest neighbors are predominantly homogeneous. On this basis the method pseudo-labels patch features: features whose nearest-neighbor distance to an iteratively reconstructed memory bank is low become normal, and high ones become anomalies, with an ambiguous band perturbed by Gaussian noise. A mutual smoothness loss then enforces score consistency on mutually closest pairs, and the whole pipeline is retrained iteratively. The paper reports that this scheme matches or exceeds prior one-class and fully unsupervised methods on contaminated MVTec AD and VisA, in both no-overlap and overlap evaluation.","pith_inferences":["A direct consequence the paper leaves implicit: the method's pretrained feature extractor becomes a hidden hyperparameter; the pair-distance ordering should be checked on the target domain's features, and a domain-mismatched extractor could break it even when the variance assumption holds in image space.","The variance-ordering assumption suggests a crisp test for new domains: measure the per-class feature variances before training. If the anomaly distribution is tight and the normal distribution is broad, the method should be expected to fail, matching the paper's own limitation note.","The mutual smoothness loss resembles a local label-consistency regularizer, so combining it with stronger consistency penalties (e.g., time or view augmentation) might reduce the need for the confident-anomaly threshold τ_c.","Because the pipeline only needs pairwise distances, it could be adapted to non-visual modalities—tabular or time-series anomaly detection—provided the feature extractor is replaced by an embedding in which the variance ordering holds."],"forward_implications":["Fully unsupervised training can replace one-class classification in industrial defect detection, so factories can build detectors from raw unlabeled production images without curating clean-normal sets.","Contamination does not force a trade-off in the overlap setting: the method stays near its no-overlap performance when training anomalies also appear at test time, whereas one-class models degrade sharply.","Higher contamination levels (up to 20%) are handled without retuning, and at very low contamination (0–1%) adding synthetic anomalies restores most of the gap.","The mutual smoothness loss and the Gaussian perturbation of ambiguous features each contribute measurably; removing them lowers image- and pixel-wise AUROC.","The same pairing statistics generalize beyond industrial patches to semantic anomaly detection on natural images."],"supporting_citations":[{"why":"Supplies the nearest-neighbor memory-bank design that FUN-AD adapts into an iteratively reconstructed bank.","marker":"[28]"},{"why":"Provides the Gaussian-noise pseudo-anomaly idea and the simple discriminator architecture used for Local-Net.","marker":"[22]"},{"why":"Fully unsupervised baseline that filters noisy training data; serves as the main comparison for contaminated settings.","marker":"[38]"},{"why":"Fully unsupervised baseline chosen by span/spread of normal patches; direct comparison in the experiments.","marker":"[25]"},{"why":"Primary industrial benchmark whose normal/anomaly images validate the pairwise-distance statistics and are used in the main experiments.","marker":"[3]"},{"why":"Second industrial benchmark (VisA) used to test generalization to multiple objects without camera alignment.","marker":"[51]"},{"why":"Basis for the mutual-closest-pair label-consistency assumption that motivates the mutual smoothness loss.","marker":"[50]"},{"why":"Pretrained self-supervised feature extractor used to compute the image- and patch-level features on which all pairwise distances are measured.","marker":"[5]"}],"fun_headline_variants":["Fully unsupervised defect detection from contaminated images","Pairwise distances replace labels in anomaly detection","No clean data needed: unsupervised method spots anomalies","IRMB pseudo-labels: learning anomalies with zero labels","Unsupervised detector handles messy training data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that in the feature space used, normal samples are more tightly clustered than anomalies, and this ordering holds for every category; if a normal class is as spread out as a dominant anomaly type, the pseudo-labels and mutual-smoothness loss lose their grounding.","fun_headline_variants_meta":{"raw":{"variants":["Fully unsupervised defect detection from contaminated images","Pairwise distances replace labels in anomaly detection","No clean data needed: unsupervised method spots anomalies","IRMB pseudo-labels: learning anomalies with zero labels","Unsupervised detector handles messy training data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001514,"raw_usage":{"total_tokens":6075,"prompt_tokens":959,"completion_tokens":5116,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":5046}},"tokens_in":575,"tokens_out":5116,"duration_ms":38691,"temperature":1.0,"reasoning_tokens":5046,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:32:21.902882+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct or select a category in which the anomaly distribution is narrow and close to the normal cluster while the normal distribution is broad (for example a high-variation object class with one dominant defect type, or an artificially spiked normal distribution). Run FUN-AD with 10% contamination and compare against a clean-data one-class baseline; if image-wise AUROC falls to near the baseline's contamination level or below, and the pairwise-distance histograms show normal-normal distances no longer dominating at small radii, the central ordering claim is falsified. The paper's own limitation section concedes this failure mode.","supporting_citations":[{"cited_title":"Towards to- tal recall in industrial anomaly detection","cited_arxiv_id":null,"evidence_quote":"Supplies the nearest-neighbor memory-bank design that FUN-AD adapts into an iteratively reconstructed bank."},{"cited_title":"Simplenet: A simple network for image anomaly detection and localization","cited_arxiv_id":null,"evidence_quote":"Provides the Gaussian-noise pseudo-anomaly idea and the simple discriminator architecture used for Local-Net."},{"cited_title":"SoftPatch: Un- supervised anomaly detection with noisy data","cited_arxiv_id":null,"evidence_quote":"Fully unsupervised baseline that filters noisy training data; serves as the main comparison for contaminated settings."},{"cited_title":"Inter- realization channels: Unsupervised anomaly detection be- yond one-class classification","cited_arxiv_id":null,"evidence_quote":"Fully unsupervised baseline chosen by span/spread of normal patches; direct comparison in the experiments."},{"cited_title":"MVTec AD — A Comprehensive Real- World Dataset for Unsupervised Anomaly Detection","cited_arxiv_id":null,"evidence_quote":"Primary industrial benchmark whose normal/anomaly images validate the pairwise-distance statistics and are used in the main experiments."},{"cited_title":"Spot-the-difference self-supervised pre- training for anomaly detection and segmentation","cited_arxiv_id":null,"evidence_quote":"Second industrial benchmark (VisA) used to test generalization to multiple objects without camera alignment."},{"cited_title":"Learning with local and global consistency.Advances in Neural Information Process- ing Systems, 16, 2003","cited_arxiv_id":null,"evidence_quote":"Basis for the mutual-closest-pair label-consistency assumption that motivates the mutual smoothness loss."},{"cited_title":"Emerg- ing properties in self-supervised vision transformers","cited_arxiv_id":null,"evidence_quote":"Pretrained self-supervised feature extractor used to compute the image- and patch-level features on which all pairwise distances are measured."}],"review_version":1}