REVIEW 4 major objections 5 minor 2 cited by
Zero-Shot Image Anomaly Detection Using Generative Foundation Models
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read DiffPathV2 claims that a six-dimensional summary of the error between a diffusion model's predicted Stein score and the ground-truth noise, weighted by (1 − SSIM), separates unseen inlier from outlier images, reaching an average AUROC of…
desk verdict DiffPathV2 reports a plausible 1.8-point AUROC gain over DiffPath, but the paper never defines how a test image is paired with a ground-truth noise epsilon(t), so the central score is not well-defined as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the six-dimensional score-error trajectory signature of Eq. (9): the per-timestep mean squared error $\mathrm{mse}(x_0, t) = \|\epsilon_\theta(x_t, t) - \epsilon(t)\|_2^2$ between the denoiser's predicted Stein score (the estimated noise direction) and the ground-truth noise, summed over timesteps in the $\ell^1$, $\ell^2$, and $\ell^3$ norms, together with the same three norms of its time derivative. This signature is then modulated per pixel by $(1 - \mathrm{SSIM}(x_0, \sum_t \epsilon_\theta(x_t, t)))$, where SSIM is the Structural Similarity Index Measure, so that structurally salient regions where noise estimation is poor carry more weight before global pooling. A Gaussian mixture model fitted to the validation set converts the six-dimensional score into a log-likelihood for each test sample. The mechanism's claim is that error statistics, not raw scores, expose the difference between the model's learned manifold and an unseen distribution.
What would settle it
Recompute the DiffPathV2 score after replacing the noise returned by DDIM inversion with any fixed noise tensor of the same shape; if the AUROC separating inliers from outliers stays near 94.9, the claimed dependence on ground-truth noise is not what drives the separation.
Extended reading notes
Core claim
On its own terms, the paper's discovery is that what discriminates distributions is not the predicted score itself but the way the prediction errs along the diffusion path. For each test image, DiffPathV2 computes the per-timestep mean squared error between the denoiser's output and the ground-truth noise, aggregates the first, second, and third norms of that error and their time derivatives into a six-dimensional score, then multiplies the per-pixel errors by an inverted SSIM map before pooling. The ablation shows that the raw-score baseline already works on far semantic shifts, but the error-plus-SSIM combination sharply separates near-OOD distributions such as CIFAR-10 versus CIFAR-100, lifting that benchmark from 59.0 to 99.4 AUROC. The paper also claims that CelebA is a better base distribution than ImageNet for this trajectory-error scoring, implying a structurally coherent training set sharpens sensitivity to semantic deviations.
Load-bearing premise
The method assumes that every test image can be paired with the actual random noise that produced its noisy version, even though the paper never explains how a real image is matched to that noise.
Editorial extensions
If this is right
- A single diffusion model trained only on CelebA can separate unseen natural-image datasets, with an average AUROC of 94.9 across the reported benchmarks, so zero-shot anomaly detection needs no target-specific retraining.
- Near-semantic shifts, the hardest case, improve most: CIFAR-10 versus CIFAR-100 goes from 59.0 AUROC with the raw-score baseline to 99.4 with the error-plus-SSIM score.
- Using the inverted SSIM map as a spatial amplifier means the method can point to which regions drive the anomaly decision, not just whether a sample is anomalous.
- Heterogeneous base distributions are not automatically better: CelebA outperforms ImageNet for DiffPath and DiffPathV2, so base-dataset choice should be treated as a tunable property of the detector.
- The success of score-error statistics over raw scores suggests that likelihood-free trajectory signals carry distributional information that exact-likelihood methods miss.
Reading between the lines
- The paper never shows how a real test image is paired with a ground-truth noise $\epsilon(t)$; if DDIM inversion only returns a deterministic reconstruction, then Eq. (8) measures consistency with an inverted path rather than error against the true noise, and the theoretical framing would need to change even if the empirical scores stand.
- The same trajectory-error idea could be tested on other generative paradigms, such as flow matching or consistency models, where a 'true noise' analogue is better defined; a positive result would generalize the claim beyond diffusion.
- The SSIM weighting suggests a cheap way to build pixel-level anomaly maps: thresholding the per-pixel $(1 - \mathrm{SSIM})$-weighted error could localize semantically anomalous regions without any segmentation training.
- The CelebA result hints that base distributions with low structural variance act like a sharper reference against which deviations are measured; this is testable by sweeping base datasets with controlled semantic breadth while holding the scoring formula fixed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes DiffPathV2, a zero-shot semantic anomaly-detection method built on an unconditional denoising diffusion model trained on CelebA. The method computes, for each test image, a six-dimensional score from the per-timestep mean squared error between the predicted denoising score, epsilon_theta(x_t,t), and a quantity called the ground-truth noise epsilon(t); these errors are spatially weighted by (1 - SSIM) before pooling. A GMM is fitted to validation scores and used to assign anomaly scores to test samples. Experiments on CIFAR-10, SVHN, and CelebA as inlier datasets against several OOD sets report an average AUROC of 94.9, outperforming the DiffPath baseline average of 93.1, and ablations show that CelebA is a more effective base distribution than ImageNet for this method.
Significance. If the method can be defined coherently, the paper contributes a useful data point: a single unconditional diffusion model can serve as a zero-shot semantic anomaly detector, and the choice of base training distribution matters in a non-obvious way. The ablation separating raw-score signals, error signals, and SSIM weighting is informative, and the finding that the proposed SSIM-weighted error signal improves near-OOD separation is genuinely interesting. The paper also honestly reports cases with large headroom and does not overclaim universality. However, the central inference procedure, as written, does not specify how a real test image can be paired with a ground-truth noise realization, so the main quantitative claim is not currently established. The availability of the exact inference protocol, and possibly code, is essential before the state-of-the-art claim can be evaluated.
major comments (4)
- [Section 4.1.1, Eq. (8); Algorithm 1 line 12] The quantity epsilon(t) called ground-truth noise cannot be obtained for a real test image by DDIM inversion. During training, epsilon(t) is the independent Gaussian draw in Eq. (5) used to construct x_t from x_0. At test time, x_0 is observed without that draw. DDIM inversion is a deterministic map from x_0 to a model-dependent latent trajectory; it does not recover the random epsilon from the forward process. If epsilon(t) is read off the inversion path as (x_t - sqrt(alpha_bar_t) x_0)/sigma_t, it is a deterministic function of the same model predictions that produce epsilon_theta, not ground truth; if epsilon(t) is taken to be the model's own predicted noise, Eq. (8) is identically zero. The paper must specify the exact test-time construction of epsilon(t), for example by explicitly sampling new forward-process noise draws, or the central anomaly score in Eq. (8) is undefined as written.
- [Section 3.2, Eq. (6)] Equation (6) omits the square in the denominator: the conditional score of p(x_t|x_0) = N(sqrt(alpha_bar_t) x_0, sigma_t^2 I) is -(x_t - sqrt(alpha_bar_t) x_0)/sigma_t^2. As printed, the equation is dimensionally inconsistent. This matters because Eq. (7), which defines the ground-truth score epsilon, is derived from Eq. (6); the derivation should be corrected and the sign/scale conventions made explicit.
- [Section 4.1.2, Eq. (10); Algorithm 1 lines 6-7] Equation (10) multiplies the global six-dimensional vector s_6D(x_0) by a pixelwise SSIM map, which is dimensionally incompatible. Algorithm 1 correctly applies the (1 - SSIM) weighting before spatial pooling. Equation (10) should be rewritten to show that the SSIM modulation is applied to the per-pixel, per-timestep error maps before the p-norm and temporal aggregation, otherwise the notation does not match the implementation.
- [Section 5.1, Table 1; Section 5.3, Table 3] All reported AUROC values are single-run point estimates with no standard deviations, confidence intervals, or seeds. Many entries are exactly 100.0, while other cells take extreme low values such as 00.1 in Table 3, and the claimed average improvement over DiffPath is 94.9 versus 93.1. Without repeated runs or a statistical comparison, the reader cannot assess whether the average gain is reliable or driven by particular benchmark combinations. The paper should report variance across runs or at least describe the number of seeds and the stability of the results.
minor comments (5)
- [Section 3.2, Eq. (5)] The notation alpha_bar = product over s of alpha_s is ambiguous: the index s over which the product runs should be stated explicitly, and the maximum timestep should be defined.
- [Section 4.2] The paper states that optimal GMM parameters are found by grid search but does not report the grid, the number of components, the covariance type, or the regularization. These details are needed for reproducibility.
- [Section 5.2, Figure 2] The histograms in Figure 2 have no axis labels or units; adding x-axis labels such as anomaly score and y-axis labels such as count would make the qualitative claim easier to evaluate.
- [Section 4.3.1] The text says the non-foundational baselines are categorized into three distinct baselines but then lists four categories: energy-based, flow-based, diffusion-based, and foundation model-based. This should be corrected.
- [Section 5.3] The phrase 'in it's diffusion trajectory' contains a typo; it should be 'its diffusion trajectory.'
Circularity Check
The central anomaly score in Eq. (8) is defined against a 'true noise' that Algorithm 1 obtains from DDIM inversion, a deterministic function of the same model; the score therefore reduces by construction to a self-consistency residual (or zero), not a ground-truth error.
-
self definitional
[Section 4.1.1, Eq. (8); Algorithm 1, lines 11-12]
"for x0 in Xval do {ϵθ(xt, t), ϵ(t)} ←DDIMInversion(x0, gθ)"
Eq. (8) defines the anomaly signal as the error between the model's predicted noise and the 'true noise' ϵ(t). In Eqs. (4)-(7), ϵ(t) is an independent Gaussian draw used to construct xt from x0. For a real test image, no such independent draw exists. Algorithm 1 instead supplies ϵ(t) as a second output of DDIMInversion. DDIM inversion is a deterministic map from x0 to a latent trajectory computed with the same network gθ; it does not sample the forward-process noise of Eq. (5). The only noise-like quantity available on that trajectory is the model's own prediction, so the 'error' in Eq. (8) is either identically zero or a self-consistency residual, not an error against ground truth.
full rationale
The paper's derivation chain is short: it hypothesizes that OOD samples induce larger denoising score errors, defines the error in Eq. (8) using an alleged ground-truth noise, and summarizes it in a 6D score. The fatal circular step is the provenance of ϵ(t). The training definition (Eq. 5) makes ϵ(t) an independent random draw, but Algorithm 1 line 12 says DDIMInversion returns it; DDIM inversion is deterministic and model-dependent, so the MSE is not an error against an independent ground truth. This makes the central anomaly score self-referential (and, in the common reading, identically zero), so the headline 94.9 AUROC is not supported by the written method. Other potential concerns: references [1] and [33] are self-citations, but they appear only in related-work context and are not load-bearing; the main comparison is against the external DiffPath baseline [9]. The GMM fitting on the validation set is a standard thresholding step, not a circular prediction. Given that the core quantity in Eq. (8) reduces by definition to the model comparing itself with itself, the circularity score is high, though the flaw is also a correctness/definitional gap that could be repaired by explicitly sampling ϵ(t) from the forward noising process.
Assumptions & free parameters
free parameters (3)
- GMM hyperparameters =
not reported
- Trajectory norm orders =
p = 1, 2, 3
- SSIM modulation =
1 - SSIM (no exponent or temperature)
assumptions (5)
- standard math The denoising score-matching objective equips epsilon_theta with an estimate of the Stein score.
- domain assumption The denoising trajectory of a diffusion model trained on a single base dataset is informative enough to separate any unseen inlier/outlier dataset pair.
- domain assumption Test samples from pOOD induce larger Stein score estimation errors than samples from pID.
- domain assumption The SSIM map between the original image and the accumulated predicted noise highlights semantically important error regions.
- domain assumption The ID validation set is drawn from the same distribution as the ID test set, so a GMM fit on validation scores calibrates test-time likelihoods.
Cite this review
Pith. "Pith review of Zero-Shot Image Anomaly Detection Using Generative Foundation Models." pith.science (2026). https://pith.science/paper/LV3BVZVG
@misc{pith2026250722692,
author = {Pith},
title = {Pith review of: Zero-Shot Image Anomaly Detection Using Generative Foundation Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/LV3BVZVG}},
note = {Machine review of arXiv:2507.22692}
}
read the original abstract
Detecting out-of-distribution (OOD) inputs is pivotal for deploying safe vision systems in open-world environments. We revisit diffusion models, not as generators, but as universal perceptual templates for OOD detection. This research explores the use of score-based generative models as foundational tools for semantic anomaly detection across unseen datasets. Specifically, we leverage the denoising trajectories of Denoising Diffusion Models (DDMs) as a rich source of texture and semantic information. By analyzing Stein score errors, amplified through the Structural Similarity Index Metric (SSIM), we introduce a novel method for identifying anomalous samples without requiring re-training on each target dataset. Our approach improves over state-of-the-art and relies on training a single model on one dataset -- CelebA -- which we find to be an effective base distribution, even outperforming more commonly used datasets like ImageNet in several settings. Experimental results show near-perfect performance on some benchmarks, with notable headroom on others, highlighting both the strength and future potential of generative foundation models in anomaly detection.
Figures
Forward citations
Cited by 2 Pith papers
-
Uncertainty-Aware Distribution-to-Distribution Flow Matching for Scientific Imaging
Bayesian Stochastic Flow Matching augments flow models with stochastic diffusion for better generalization and uses Monte Carlo Dropout with antithetic sampling to disentangle uncertainties and detect out-of-distribut...
-
Uncertainty-Aware Distribution-to-Distribution Flow Matching for Scientific Imaging
SFM improves generalization under distribution shift for scientific imaging tasks while AVUQ supplies sample-efficient epistemic and aleatoric uncertainty estimates plus anomaly scores.
Reference graph
Works this paper leans on
-
[1]
Typicality excels likelihood for unsupervised out-of-distribution detec- tion in medical imaging
Lemar Abdi, MM Amaan Valiuddin, Christiaan GA Viviers, Peter HN de With, and Fons van der Sommen. Typicality excels likelihood for unsupervised out-of-distribution detec- tion in medical imaging. In Uncertainty for Safe Utiliza- tion of Machine Learning in Medical Imaging: 6th Interna- tional Workshop, Held in Conjunction with MICCAI 2024, Proceedings, pa...
work page 2024
-
[2]
Improving normalizing flows with the approx- imate mass for out-of-distribution detection
Samy Chali, Inna Kucher, Marc Duranton, and Jacques- Olivier Klein. Improving normalizing flows with the approx- imate mass for out-of-distribution detection. In Proceed- ings of the IEEE/CVF Conference on CVPR, pages 750–758,
-
[3]
Hyunsun Choi, Eric Jang, and Alexander A. Alemi. W AIC, but Why? Generative Ensembles for Robust Anomaly De- tection, 2019. arXiv:1810.01392 [cs, stat]. 2, 5
arXiv 2019
- [4]
-
[5]
Nice: Non-linear independent components estimation
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516, 2014. 5
arXiv 2014
-
[6]
Implicit generation and mod- eling with energy based models
Yilun Du and Igor Mordatch. Implicit generation and mod- eling with energy based models. Advances in neural infor- mation processing systems, 32, 2019. 5
work page 2019
-
[7]
Improved contrastive divergence training of energy- based models
Yilun Du, Shuang Li, Joshua Tenenbaum, and Igor Mor- datch. Improved contrastive divergence training of energy- based models. In International Conference on Machine Learning, pages 2837–2848. PMLR, 2021. 5
work page 2021
-
[8]
Denoising diffusion models for out-of-distribution detection
Mark S Graham, Walter HL Pinaya, Petru-Daniel Tudosiu, Parashkev Nachev, Sebastien Ourselin, and Jorge Cardoso. Denoising diffusion models for out-of-distribution detection. In Proceedings of the IEEE/CVF CVPR , pages 2948–2957,
Show all 37 references
-
[9]
Out-of-distribution detection with a single unconditional diffusion model
Alvin Heng, Harold Soh, et al. Out-of-distribution detection with a single unconditional diffusion model. Advances in NeurIPS, 37:43952–43974, 2024. 1, 2, 4, 6, 8
2024
-
[10]
Denoising diffu- sion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in NeurIPS , 33:6840– 6851, 2020. 1, 4
2020
-
[11]
Auto-encoding varia- tional bayes, 2022
Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes, 2022. 1
2022
-
[12]
Why Normalizing Flows Fail to Detect Out-of-Distribution Data
Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why Normalizing Flows Fail to Detect Out-of-Distribution Data. In NIPS, pages 20578–20589. Curran Associates, Inc.,
-
[13]
Information-theoretic diffusion
Xianghao Kong, Rob Brekelmans, and Greg Ver Steeg. Information-theoretic diffusion. In The Eleventh Interna- tional Conference on Learning Representations, 2023. 5, 8
2023
-
[14]
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 5
2009
-
[15]
Pseudo numerical methods for diffusion models on manifolds
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. InIn- ternational Conference on Learning Representations , 2022. 1
2022
-
[16]
Deep learning face attributes in the wild
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015. 5
2015
-
[17]
Unsupervised out-of-distribution detection with diffusion inpainting
Zhenzhen Liu, Jin Peng Zhou, Yufan Wang, and Kilian Q Weinberger. Unsupervised out-of-distribution detection with diffusion inpainting. In International Conference on Ma- chine Learning, pages 22528–22538. PMLR, 2023. 1, 5
2023
-
[18]
Multiscale score matching for out-of-distribution detection
Ahsan Mahmood, Junier Oliva, and Martin Andreas Styner. Multiscale score matching for out-of-distribution detection. In International Conference on Learning Representations ,
-
[19]
Density of states estimation for out of distribution detection
Warren Morningstar, Cusuh Ham, Andrew Gallagher, Balaji Lakshminarayanan, Alex Alemi, and Joshua Dillon. Density of states estimation for out of distribution detection. InInter- national Conference on Artificial Intelligence and Statistics, pages 3232–3240. PMLR, 2021. 5
2021
-
[20]
Anomaly detection with conditioned denoising diffusion models
Arian Mousakhan, Thomas Brox, and Jawad Tayyub. Anomaly detection with conditioned denoising diffusion models. In DAGM German Conference on Pattern Recog- nition, pages 181–195. Springer, 2024. 1, 2
2024
-
[21]
Detecting out-of-distribution inputs to deep generative models using typicality
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Bal- aji Lakshminarayanan. Detecting out-of-distribution inputs to deep generative models using typicality. arXiv preprint arXiv:1906.02994, 2019. 5
1906 arXiv
-
[22]
Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan G¨or¨ur, and Balaji Lakshminarayanan
Eric T. Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan G¨or¨ur, and Balaji Lakshminarayanan. Do deep generative models know what they don’t know? In ICLR 2019,, 2019. 2
2019
-
[23]
Reading digits in natural images with unsupervised feature learning
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bis- sacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learn- ing, page 4. Granada, 2011. 5
2011
-
[24]
Improved denoising diffusion probabilistic models
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR,
-
[25]
Osada, T
G. Osada, T. Takahashi, and T. Nishide. Understanding like- lihood of normalizing flow and image complexity through the lens of out-of-distribution detection. Proceedings of the AAAI Conference on Artificial Intelligence , 38(19):21492– 21500, 2024. 2
2024
-
[26]
Likelihood ratios for out-of-distribution detec- tion
Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshmi- narayanan. Likelihood ratios for out-of-distribution detec- tion. Advances in neural information processing systems , 32, 2019. 5
2019
-
[27]
A simple fix to mahalanobis distance for improving near-ood detection
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan. A simple fix to mahalanobis distance for improving near-ood detection. arXiv preprint arXiv:2106.09022, 2021. 2
2021 arXiv
-
[28]
Variational inference with normalizing flows
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference on ma- chine learning, pages 1530–1538. PMLR, 2015. 5
2015
-
[29]
Reconstruction- free anomaly detection with diffusion models via direct la- tent likelihood evaluation
Shunsuke Sakai and Tatsuhito Hasegawa. Reconstruction- free anomaly detection with diffusion models via direct la- tent likelihood evaluation. arXiv preprint arXiv:2504.05662,
-
[30]
N ´u˜nez, and Jordi Luque
Joan Serr `a, David ´Alvarez, Vicenc ¸ G´omez, Olga Slizovskaia, Jos´e F. N ´u˜nez, and Jordi Luque. Input complexity and out-of-distribution detection with likelihood-based genera- tive models. In International Conference on Learning Rep- resentations, 2020. 1, 5
2020
-
[31]
Score-based generative modeling through stochastic differential equa- tions
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In ICLR, 2021. 2, 3, 5
2021
-
[32]
A connection between score matching and denoising autoencoders
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661– 1674, 2011. 3
2011
-
[33]
Can your generative model detect out-of-distribution covari- ate shift? In ECCV, pages 184–201
Christiaan Viviers, Amaan Valiuddin, Francisco Caetano, Lemar Abdi, Lena Filatova, Fons van der Sommen, et al. Can your generative model detect out-of-distribution covari- ate shift? In ECCV, pages 184–201. Springer, 2025. 2
2025
-
[34]
Diffusion models for medical anomaly detection
Julia Wolleb, Florentin Bieder, Robin Sandk ¨uhler, and Philippe C Cattin. Diffusion models for medical anomaly detection. In MICCAI, pages 35–45. Springer, 2022. 1, 2
2022
-
[35]
{V AEBM}: A symbiosis between variational autoencoders and energy-based models
Zhisheng Xiao, Karsten Kreis, Jan Kautz, and Arash Vahdat. {V AEBM}: A symbiosis between variational autoencoders and energy-based models. In International Conference on Learning Representations, 2021. 5
2021
-
[36]
Generalized out-of-distribution detection: A survey
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. Inter- national Journal of Computer Vision , 132(12):5635–5662,
-
[37]
Rethinking reconstruction autoencoder-based out-of-distribution detection
Yibo Zhou. Rethinking reconstruction autoencoder-based out-of-distribution detection. In Proceedings of the IEEE/CVF CVPR, pages 7379–7387, 2022. 2
2022
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.