REVIEW 1 major objections 5 minor 75 references
A Probabilistic Circuit-Induced Pseudo-Metric for Out-of-Distribution Detection
T0 review · 1 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that the internal node likelihoods of a trained probabilistic circuit can be turned into a statistically calibrated, batch-level out-of-distribution detector that requires no held-out in-distribution data.
desk verdict HLD is a genuinely new PC-based OOD test, but the exact covariance computation as written requires lifting sub-scope circuits with constant-1 factors that the paper's PC class cannot represent, undermining the no-held-out threshold claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Hierarchical Likelihood Vector (HLV): for a selected set of circuit nodes, each coordinate is the probability that the node's subcircuit assigns to the part of the observation lying in that node's variable scope. The argument is carried by three structural facts about smooth, structured-decomposable probabilistic circuits: the scope trichotomy lemma (any two node scopes are disjoint, equal, or strictly nested), the tractable product operation for compatible PCs, and exact marginalization. Together they let the paper compute $\mathbb{E}_P[\mathrm{HLV}]$ and its covariance by forming product circuits and marginalizing, with worst-case $O(S N_r^2)$ and $O(S^2 N_r^3)$ costs; a two-moment Gaussian approximation to a generalized chi-square distribution then turns the traces into a closed-form threshold.
What would settle it
Train a structured-decomposable PC on a high-dimensional distribution it fits poorly, then feed it held-out in-distribution batches of increasing size at a fixed significance level. The paper's own 28x28 MNIST experiment is exactly this test: FPR_data rises from 0.178 at 7x7 resolution to 0.823 at T=1000, showing the detector responding to model misspecification rather than true OOD, whereas a detector calibrated to the real data distribution would keep the false-positive rate near the nominal level.
Extended reading notes
Core claim
The paper's central claim is that the hierarchical probabilistic summaries inside a trained probabilistic circuit can be converted into a principled distribution-comparison test. The Hierarchical Likelihood Distance (HLD), defined as the Euclidean distance between the expected Hierarchical Likelihood Vectors of two distributions, is an integral probability metric over the function class of linear combinations of node likelihood functions. For smooth, structured-decomposable PCs, the population mean vector and covariance matrix of the HLV, and hence the traces needed for a decision threshold, are computable exactly and tractably from the circuit itself. Consequently, a trained PC alone serves as the in-distribution representation for a batch-level goodness-of-fit test whose null hypothesis is that the test batch comes from the distribution represented by the PC.
Load-bearing premise
The trained probabilistic circuit is an accurate enough representation of the true in-distribution that testing whether a batch comes from the circuit's distribution is a meaningful proxy for testing whether it comes from the real data distribution; if the circuit is misspecified, the test fires false alarms on genuine in-distribution data.
Editorial extensions
If this is right
- At deployment the trained PC alone represents the in-distribution, so no held-out ID calibration or reference samples are needed for the HLD test.
- Because HLD is an integral probability metric, the detector compares distributions through a probabilistically interpretable function class rather than an ad hoc feature embedding.
- The exact population moments yield an approximate analytic threshold from the traces of the HLV covariance, and the approximation is accurate when no small number of eigenvalues dominates the covariance spectrum.
- On the evaluated tabular datasets and 7x7 MNIST, batch-level detection reaches near-perfect power at modest batch sizes, and the node-wise decomposition of the statistic localizes distribution shifts to variable subsets.
- Using leaf and sum nodes in the HLV gives the most robust detection across datasets; adding product nodes did not improve empirical detection power.
- The test is calibrated with respect to the distribution represented by the trained PC, so its false-positive rate on real in-distribution data directly reflects how faithfully the PC models the underlying data distribution.
Reading between the lines
- The paper's own 28x28 MNIST result, where FPR_data rises to 0.823 at T=1000, shows that the method becomes a detector of model misspecification when the circuit fits poorly; I infer that any practical deployment should validate the PC's fit before trusting the nominal significance level.
- The same exact moment-computation machinery could extend to other tractable probabilistic models that support product and marginalization operations, and to continuous variables if node densities are bounded.
- The node-scope ranking could be developed into an interpretable drift-localization tool for monitoring high-dimensional data, although the paper presents it only as qualitative localization cues.
- Whenever the covariance spectrum is concentrated, the two-moment Gaussian approximation is the main statistical weak point; computing the generalized chi-square quantile numerically from the exactly obtained eigenvalues is a drop-in replacement.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the Hierarchical Likelihood Vector (HLV), whose coordinates are likelihoods evaluated at selected nodes of a probabilistic circuit (PC), and the Hierarchical Likelihood Distance (HLD), defined as the Euclidean distance between expected HLVs under two distributions. It proves that HLD is an integral probability metric over the linear span of the node likelihood functions, and it develops a batch-level goodness-of-fit test for unsupervised OOD detection. The key claimed contribution is that, for smooth and structured-decomposable PCs over discrete variables, the population mean and covariance of the HLV can be computed exactly from the trained PC, yielding an approximate analytic decision threshold that requires no held-out in-distribution data. Experiments on five tabular datasets, binarized MNIST at 7x7 and 28x28, node-type ablations, architecture ablations, and a localization analysis are reported.
Significance. If the exact moment-computation claim is made rigorous, the paper makes a valuable contribution: it turns the internal hierarchy of a PC into a statistically interpretable representation, gives an IPM interpretation of the resulting distance, and removes the need for an ID calibration sample at deployment. The theoretical components that are fully specified—Theorem 1, the CLT/quadratic-form asymptotics, the same-scope product computations, and the IPM bound in the supplement—are standard and check out. The experimental study is extensive, with 500 Monte Carlo trials, per-pair MNIST tables, and explicit treatment of model misspecification. The paper is also candid about its two main limitations: the null hypothesis is calibrated with respect to the trained model rather than the true data distribution (FPR_data reaches 0.823 at T=1000 on 28x28 MNIST), and the closed-form threshold rests on a two-moment approximation to a generalized chi-square. The main technical gap is in the exact covariance construction, which is repairable but currently not well-defined as written.
major comments (1)
- [Exact computation of ΣP; Algorithm 1, line 11] The expression pSij ⊗ pi ⊗ pj is not a well-defined product operation under the manuscript's own PC definitions. The tractable product operation cited from Vergari et al. (2021) applies to compatible circuits over a common variable scope, whereas pSij, pi, and pj generally have different scopes. Lemma 2 (the scope trichotomy) only classifies the relations among Si and Sj; it does not state that circuits over different scopes are compatible operands for a product, and the paper gives no lemma defining a cross-scope product. Moreover, because every node of the PC is defined as a normalized distribution over its scope, the constant-1 function on Sij \ Si is not representable as a normalized input distribution for a categorical variable with more than one outcome, so the standard 'pad with an identity' lifting of a sub-scope circuit is unavailable under the paper's node semantics. This gap is load-bearing: ΣP and tr(ΣP^2) feed directly into the decision threshold in Algorithm 2, so without a valid construction of the second moments the 'no held-out ID data' contribution collapses. The issue is repairable within the same toolkit: for disjoint scopes, marginalize the root PC to S_i ∪ S_j rather than to the LCA scope S_ij; the induced v-tree on S_i ∪ S_j has root children Si and Sj, so pi ⊗ pj is a compatible same-scope circuit and the standard product operation applies. For nested scopes, first compute the same-scope product pSj ⊗ pj, marginalize the result to Si, and then multiply by pi. The authors should either rewrite Algorithm 1 along these lines or provide a precise lemma defining the cross-scope product they intend.
minor comments (5)
- [Goodness-of-Fit Test and Algorithm 2] The threshold expression is presented as a normal approximation to a generalized chi-square distribution, and the paper notes that the approximation can be poor when a few eigenvalues dominate. The experimental FPR_model values support the approximation in the tested settings, but the abstract and introduction should state more explicitly that the claimed calibration is with respect to the distribution represented by the trained PC, not the true data distribution; the 28x28 MNIST FPR_data result (Table 20) makes this distinction essential.
- [Tables 12-15] The captions of Tables 12-15 appear to be swapped. Table 12, labeled FPRdata, shows values that remain near 0.03 across all T at α=0.01, which is the behavior of the PC-vs-PC model null; Table 13, labeled FPRmodel, grows from 0.027 to 0.394 as T increases, which is the behavior of FPRdata. The same pattern appears for Tables 14 and 15 at α=0.1. Please verify the labels and correct them, since these tables directly support the calibration claims.
- [Lemma numbering] The main text's exact-covariance paragraph refers to 'Lemma 2' for the scope trichotomy, but the only lemma stated in the main text is Lemma 1; the supplement labels the scope trichotomy as Lemma 2. Renumber or cross-reference consistently.
- [Notation for product operation] Algorithm 1 uses the notation pSi ⊗ pni and pSij ⊗ pni ⊗ pnj without defining the ⊗ operation in the Background section. Since the operation is central to the exact-computation claim, a formal definition or a pointer to the precise definition in the cited composition atlas should be added.
- [Reproducibility] The paper does not mention code or data-release plans. Given the number of reported tables and ablations, a public implementation would substantially aid reproducibility and would let readers verify the exact moment computations on real circuits.
Circularity Check
No significant circularity: the derivation is self-contained and the central threshold computation does not reduce to fitted quantities.
full rationale
The paper's central chain is: define HLV from node likelihoods, define HLD as the L2 distance between expected HLVs, interpret it as an IPM over the linear span of node likelihoods, then compute the null moments mu_P and Sigma_P exactly from the structured-decomposable PC and form a two-moment threshold. The IPM theorem (Theorem 1) is a definitional equivalence: the function class F_C is explicitly defined so that the IPM supremum equals ||mu_P - mu_Q||_2. This is a true reformulation, not a derivation that assumes its conclusion, and it is not used to force the empirical results. The population mean and covariance computation is an algorithmic consequence of tractable PC operations, not a fitted parameter renamed as a prediction; the threshold depends only on the trained circuit and the batch size T. There are no self-citations, and no uniqueness theorem or ansatz is imported from the authors' prior work. The acknowledged limitation that FPR_data depends on model fidelity (e.g., 0.823 at 28x28 MNIST) is a misspecification caveat, not a circular step. Any concern that the product operation for differing scopes is not well-defined is a correctness or technical-support issue, not a circularity of the kind this pass flags.
Assumptions & free parameters
assumptions (3)
- domain assumption The PC is smooth and structured-decomposable with respect to a v-tree.
- domain assumption Input distributions are discrete (Bernoulli or categorical).
- standard math Standard multivariate CLT and Gaussian quadratic form theory.
Cite this review
Pith. "Pith review of A Probabilistic Circuit-Induced Pseudo-Metric for Out-of-Distribution Detection." pith.science (2026). https://pith.science/paper/VNEDJCGJ
@misc{pith2026260809117,
author = {Pith},
title = {Pith review of: A Probabilistic Circuit-Induced Pseudo-Metric for Out-of-Distribution Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/VNEDJCGJ}},
note = {Machine review of arXiv:2608.09117}
}
read the original abstract
Probabilistic Circuits (PCs) are tractable generative models whose internal nodes encode a hierarchy of probabilistic sum- maries over different variable scopes. Existing PC-based out- of-distribution (OOD) detection methods ignore this hierar- chy, reducing the entire circuit to the scalar likelihood (or its uncertainty) computed at the root. We introduce Hierar- chical Likelihood Vector (HLV), a representation whose en- tries are the likelihoods associated with selected PC nodes and define the Hierarchical Likelihood Distance (HLD), a PC-induced pseudo-metric that compares the probability dis- tributions through the expectations of their HLVs. We show that HLD is an integral probability metric over a function class naturally induced by the PC and develop a principled goodness-of-fit hypothesis test for unsupervised OOD detec- tion. Unlike existing approaches, the trained PC alone serves as the representation of the in-distribution: no held-out in- distribution data are required at deployment. We further show that the quantities required by the hypothesis test can be com- puted exactly, directly from the trained circuit, yielding an ap- proximate analytic decision threshold. Experiments on tabular and MNIST datasets demonstrate that exploiting the hierarchi- cal probabilistic summaries encoded through the PC improve OOD detection over root-likelihood, uncertainty-, typicality- and kernel-based baselines, while naturally localizing distri- bution shifts to the PC nodes responsible for the shift.
Figures
Reference graph
Works this paper leans on
-
[1]
Choi, YooJung and Vergari, Antonio and Van den Broeck, Guy , title=. 2020 , journal=
work page 2020
-
[2]
Advances in Neural Information Processing Systems , year=
Vergari, Antonio and Choi, Amji and Teso, Stefano and Van den Broeck, Guy , title=. Advances in Neural Information Processing Systems , year=
-
[3]
International Conference on Learning Representations , year=
Do deep generative models know what they don't know? , author=. International Conference on Learning Representations , year=
-
[4]
Uncertainty in Artificial Intelligence , year=
Probabilistic circuits that know what they don’t know , author=. Uncertainty in Artificial Intelligence , year=
-
[5]
Liu and Emily Fertig and Jasper Snoek and Ryan Poplin and Mark A
Jie Ren and Peter J. Liu and Emily Fertig and Jasper Snoek and Ryan Poplin and Mark A. DePristo and Joshua V. Dillon and Balaji Lakshminarayanan , title =. Advances in Neural Information Processing Systems , year =
-
[6]
Advances in Neural Information Processing Systems , year =
Kirichenko, Polina and Izmailov, Pavel and Wilson, Andrew Gordon , title =. Advances in Neural Information Processing Systems , year =
-
[7]
Serr. Input complexity and out-of-distribution detection with likelihood-based generative models , booktitle =
-
[8]
International Conference on Artificial Intelligence and Statistics , year =
Morningstar, Warren and Ham, Cusuh and Gallagher, Andrew and Lakshminarayanan, Balaji and Alemi, Alex and Dillon, Joshua , title =. International Conference on Artificial Intelligence and Statistics , year =
Show all 75 references
-
[9]
Advances in neural information processing systems , year =
Xiao, Zhisheng and Yan, Qing and Amit, Yali , title =. Advances in neural information processing systems , year =
-
[10]
International Conference on Learning Representations , year =
Hendrycks, Dan and Mazeika, Mantas and Dietterich, Thomas , title =. International Conference on Learning Representations , year =
-
[11]
Advances in neural information processing systems , year =
Liu, Weitang and Wang, Xiaoyun and Owens, John and Li, Yixuan , title =. Advances in neural information processing systems , year =
-
[12]
International Conference on Learning Representations , year =
Du, Xuefeng and Wang, Zhaoning and Cai, Mu and Li, Yixuan , title =. International Conference on Learning Representations , year =
-
[13]
Neural computation , year =
MacKay, David JC , title =. Neural computation , year =
-
[14]
International conference on machine learning , year =
Blundell, Charles and Cornebise, Julien and Kavukcuoglu, Koray and Wierstra, Daan , title =. International conference on machine learning , year =
-
[15]
International conference on machine learning , year=
Dropout as a bayesian approximation: Representing model uncertainty in deep learning , author=. International conference on machine learning , year=
-
[16]
Advances in neural information processing systems , year =
Lakshminarayanan, Balaji and Pritzel, Alexander and Blundell, Charles , title =. Advances in neural information processing systems , year =
-
[17]
Advances in neural information processing systems , year =
Sensoy, Murat and Kaplan, Lance and Kandemir, Melih , title =. Advances in neural information processing systems , year =
-
[18]
Advances in neural information processing systems , year =
Kendall, Alex and Gal, Yarin , title =. Advances in neural information processing systems , year =
-
[19]
Advances in neural information processing systems , year =
Tack, Jihoon and Mo, Sangwoo and Jeong, Jongheon and Shin, Jinwoo , title =. Advances in neural information processing systems , year =
-
[20]
Advances in neural information processing systems , year =
Sun, Yiyou and Guo, Chuan and Li, Yixuan , title =. Advances in neural information processing systems , year =
-
[21]
arXiv preprint arXiv:1810.01392 , year=
WAIC, but why? generative ensembles for robust anomaly detection , author=. arXiv preprint arXiv:1810.01392 , year=
-
[22]
Special lecture on IE , year=
Variational autoencoder based anomaly detection using reconstruction probability , author=. Special lecture on IE , year=
-
[23]
arXiv preprint arXiv:1906.02994 , year=
Detecting out-of-distribution inputs to deep generative models using typicality , author=. arXiv preprint arXiv:1906.02994 , year=
1906 arXiv
-
[24]
International statistical review , year=
On choosing and bounding probability metrics , author=. International statistical review , year=
-
[25]
1999 , publisher=
Elements of information theory , author=. 1999 , publisher=
1999
-
[26]
The ICML 2026 Workshop on Hypothesis Testing , year=
Membership Circuits: Tractable Membership Testing via Probabilistic Circuits , author=. The ICML 2026 Workshop on Hypothesis Testing , year=
2026
-
[27]
Advances in applied probability , year=
Integral probability metrics and their generating classes of functions , author=. Advances in applied probability , year=
-
[28]
Electronic Journal of Statistics , year=
On the empirical estimation of integral probability metrics , author=. Electronic Journal of Statistics , year=
-
[29]
The journal of machine learning research , year=
A kernel two-sample test , author=. The journal of machine learning research , year=
-
[30]
2009 , publisher=
Optimal transport: old and new , author=. 2009 , publisher=
2009
-
[31]
International conference on machine learning , year=
A kernel test of goodness of fit , author=. International conference on machine learning , year=
-
[32]
International conference on machine learning , year=
A kernelized Stein discrepancy for goodness-of-fit tests , author=. International conference on machine learning , year=
-
[33]
Computational Statistics & Data Analysis , year=
A new chi-square approximation to the distribution of non-negative definite quadratic forms in non-central normal variables , author=. Computational Statistics & Data Analysis , year=
-
[34]
Biometrika , year=
Computing the distribution of quadratic forms in normal variables , author=. Biometrika , year=
-
[35]
Effect of inequality of variance in the one-way classification , author=
Some theorems on quadratic forms applied in the study of analysis of variance problems, I. Effect of inequality of variance in the one-way classification , author=. The annals of mathematical statistics , year=
-
[36]
Introduction to Nonparametric Estimation , author =
-
[37]
An Introduction to Multivariate Statistical Analysis , author =
-
[38]
Advances in Neural Information Processing Systems , year=
Tractable regularization of probabilistic circuits , author=. Advances in Neural Information Processing Systems , year=
-
[39]
2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) , year=
Sum-product networks: A new deep architecture , author=. 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) , year=
2011
-
[40]
Joint European conference on machine learning and knowledge discovery in databases , year=
Cutset networks: A simple, tractable, and scalable approach for improving the accuracy of chow-liu trees , author=. Joint European conference on machine learning and knowledge discovery in databases , year=
-
[41]
, author=
Probabilistic Sentential Decision Diagrams. , author=. Proceedings of the 14th International Conference on Principles of Knowledge Representation and Reasoning (KR) , year=
-
[42]
International Conference on Machine Learning , year=
Einsum networks: Fast and scalable learning of tractable probabilistic circuits , author=. International Conference on Machine Learning , year=
-
[43]
Uncertainty in Artificial Intelligence , year=
Random sum-product networks: A simple and effective approach to probabilistic deep learning , author=. Uncertainty in Artificial Intelligence , year=
-
[44]
International Conference on Learning Representations , year =
Liu, Anji and Zhang, Honghua and Broeck, Guy Van den , title =. International Conference on Learning Representations , year =
-
[45]
arXiv preprint arXiv:2406.00766 , year=
Scaling tractable probabilistic circuits: A systems perspective , author=. arXiv preprint arXiv:2406.00766 , year=
-
[46]
Artificial Intelligence and Statistics , year=
On theoretical properties of sum-product networks , author=. Artificial Intelligence and Statistics , year=
-
[47]
Advances in Neural Information Processing Systems , year =
Shen, Yujia and Choi, Arthur and Darwiche, Adnan , title =. Advances in Neural Information Processing Systems , year =
-
[48]
Advances in Neural Information Processing Systems , year =
Khosravi, Pasha and Choi, YooJung and Liang, Yitao and Vergari, Antonio and Van den Broeck, Guy , title =. Advances in Neural Information Processing Systems , year =
-
[49]
IEEE transactions on pattern analysis and machine intelligence , year=
On the latent variable interpretation in sum-product networks , author=. IEEE transactions on pattern analysis and machine intelligence , year=
-
[50]
International Conference on Machine Learning , year=
On the relationship between sum-product networks and Bayesian networks , author=. International Conference on Machine Learning , year=
-
[51]
Proceedings of the Tenth International Symposium on Imprecise Probability: Theories and Applications , year=
Credal sum-product networks , author=. Proceedings of the Tenth International Symposium on Imprecise Probability: Theories and Applications , year=
-
[52]
Advances in neural information processing systems , year =
Correia, Alvaro and Peharz, Robert and de Campos, Cassio P , title =. Advances in neural information processing systems , year =
-
[53]
Proceedings of the AAAI Conference on Artificial Intelligence , year=
Group fairness by probabilistic modeling with latent fair decisions , author=. Proceedings of the AAAI Conference on Artificial Intelligence , year=
-
[54]
International Conference on Learning Representations , year=
Subtractive mixture models via squaring: Representation and learning , author=. International Conference on Learning Representations , year=
-
[55]
Proceedings of the AAAI Conference on Artificial Intelligence , year=
Sum of squares circuits , author=. Proceedings of the AAAI Conference on Artificial Intelligence , year=
-
[56]
Proceedings of the AAAI Conference on Artificial Intelligence , year=
On the relationship between monotone and squared probabilistic circuits , author=. Proceedings of the AAAI Conference on Artificial Intelligence , year=
-
[57]
Entropy , year=
Perfect density models cannot guarantee anomaly detection , author=. Entropy , year=
-
[58]
1996 , howpublished =
Becker, Barry and Kohavi, Ronny , title =. 1996 , howpublished =
1996
-
[59]
1995 , howpublished =
Tromp, John , title =. 1995 , howpublished =
1995
-
[60]
Proceedings of the Institute of Electrical and Electronics Engineers , year=
Gradient-based learning applied to document recognition , author=. Proceedings of the Institute of Electrical and Electronics Engineers , year=
-
[61]
, title =
Efron, Bradley and Tibshirani, Robert J. , title =
-
[62]
The Annals of Statistics , year =
Efron, Bradley , title =. The Annals of Statistics , year =
-
[63]
1998 , howpublished =
Blackard, Jock , title =. 1998 , howpublished =
1998
-
[64]
Census-Income (KDD) , year =
-
[65]
2013 , howpublished =
Bator, Martyna , title =. 2013 , howpublished =
2013
-
[66]
1968 , publisher =
Convergence of Probability Measures , author =. 1968 , publisher =
1968
-
[67]
Statistical inference , author=
-
[68]
1994 , publisher=
Continuous univariate distributions, volume 1 , author=. 1994 , publisher=
1994
-
[69]
1995 , publisher=
Probability and measure , author=. 1995 , publisher=
1995
-
[70]
1998 , publisher =
Asymptotic Statistics , author=. 1998 , publisher =
1998
-
[71]
Proceedings of the AAAI Conference on Artificial Intelligence , year=
Tractable sharpness-aware learning of probabilistic circuits , author=. Proceedings of the AAAI Conference on Artificial Intelligence , year=
-
[72]
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics , year =
Credibility-Aware Multimodal Fusion Using Probabilistic Circuits , author =. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics , year =
-
[73]
Forty-third International Conference on Machine Learning , year=
Geometry-Aware Probabilistic Circuits via Voronoi Tessellations , author=. Forty-third International Conference on Machine Learning , year=
-
[74]
Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS) , year =
YooJung Choi and Tal Friedman and Guy Van den Broeck , title =. Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS) , year =
-
[75]
2026 , url=
PAR: Plausibility-aware Amortized Recourse Generation , author=. 2026 , url=
2026
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.