REVIEW 3 major objections 4 minor 47 references
Unfolding Tensors to Identify the Graph in Discrete Latent Bipartite Graphical Models
T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper proves that in discrete latent bipartite graphical models, the bipartite graph between observed and latent variables is identifiable—up to permutation of the latent variables—from the joint distribution of the observed…
desk verdict The tensor-unfolding certificate is a real idea, but Theorem 1 is false as stated: isolated observed variables break Proposition 1's rank test. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the $J$-way population probability tensor $T$, together with its unfoldings into matrices by grouping modes. The carrying mechanism is the factorization of each unfolded matrix through the latent variables: when $co(j_1)=co(j_2)=\{k\}$, the matrix $[T]_{\{j_1,j_2\},:}$ factors as $P(Y_{\{j_1,j_2\}}\mid A_k)\,P(A_k,A)\,P(Y_{\setminus\{j_1,j_2\}}\mid A)^\top$, forcing its rank at most $H$; the converse direction uses a rank lower bound on the conditional table $P(Y_{\{j_1,j_2\}}\mid A_{co(j_1)\cup co(j_2)})$ to show the rank exceeds $H$ generically. Proposition 2 uses the analogous factorization with the $2K$ already-identified pure variables as the column group, giving the threshold $H^{K-1}$. These rank certificates are what make the proof constructive.
What would settle it
A concrete falsifier of the unconditional statement: choose a model satisfying Assumptions 1 and 2 with each latent having exactly two pure observed variables, but pick the conditional probability tables so that $P(Y_{\{j_1,j_2\}}\mid A_{co(j_1)\cup co(j_2)})$ has rank no greater than $H$ for a pair $j_1,j_2$ that do not share a single pure latent. Then the unfolding $[T]_{\{j_1,j_2\},:}$ would have rank at most $H$, violating the 'only if' direction of Proposition 1, so two distinct graphs would produce the same population tensor and Theorem 1's conclusion would fail.
Extended reading notes
Core claim
Theorem 1 states that under Assumptions 1 and 2, if the $J\times K$ graph matrix $G$ contains two disjoint copies of the identity matrix $I_K$ after a row permutation—equivalently, $G=(I_K; I_K; G^\star)^\top$ for an arbitrary binary matrix $G^\star$—then the bipartite graph is identifiable from the $J$-way population tensor $T$ with entries $T_{i_1,\dots,i_J}=P(Y_1=i_1,\dots,Y_J=i_J)$, up to a permutation of the $K$ latent columns. Identification is constructive: unfolding $T$ into matrices and testing their ranks yields both $K$ and the full incidence matrix. Specifically, for any pair of observed variables $j_1\neq j_2$, the rank of the unfolding $[T]_{\{j_1,j_2\},:}$ is at most $H$ (the number of latent categories) exactly when $j_1$ and $j_2$ are both pure children of the same latent variable (Proposition 1); for a multi-parent observed variable $j$, the rank of the unfolding $[T]_{([K]\setminus\{k\})\cup\{j\},\{K+1,\dots,2K\}}$ exceeds $H^{K-1}$ exactly when $k\in co(j)$ (Proposition 2). These two rank certificates give a population-level structure-learning algorithm that the paper presents as the proof of Theorem 1.
Load-bearing premise
The rank certificates that drive the construction are proved only for generic parameter values, while Theorem 1 is stated without the generic qualifier; if the true conditional probability tables lie on the excluded algebraic variety, the strict rank inequalities can fail and the graph need not be recoverable by the stated tests.
Editorial extensions
If this is right
- If Theorem 1 is correct, the exact population distribution determines both the number of latent variables and the bipartite graph up to latent label permutation, with no prior knowledge of $K$ required.
- The identification holds across directed models like Noisy-Or networks and undirected models like restricted Boltzmann machines, with independent or dependent latents, and with nonlinear link functions and interaction effects.
- The condition is interpretable and actionable: each latent trait needs at least two pure observed indicators, while all other observed variables may have arbitrarily dense connections.
- Corollary 1 shows that only marginal tensors of order $2K$ are needed to identify the graph, so the full $J$-way contingency table is not required.
- Proposition 3 adds that two pure children per latent identify the graph but not necessarily all continuous parameters, whereas three pure children per latent identify every parameter.
Reading between the lines
- The rank-threshold structure hints at a practical finite-sample estimator: replace the population tensor by the empirical contingency table and threshold the estimated singular values of the unfolded matrices; the paper does not analyze the sample regime, so finite-sample guarantees remain open.
- The two-pure-measurement condition is analogous to anchor-item assumptions in psychometric measurement models, so the result may transfer to cognitive diagnosis and educational testing practice where such pure questions are already required by design.
- Because the proof only uses rank certificates from unfoldings, the approach might extend to multilayer latent models or to graphs with more than two observed layers, though the paper does not pursue this.
- For restricted Boltzmann machines, the result suggests that sparse loading structures are identifiable in a way that dense ones are not; this could motivate architectural constraints for interpretable deep generative models.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a constructive identifiability result for the bipartite loading graph in discrete latent bipartite graphical models, covering directed models such as Noisy-Or Bayesian networks and undirected models such as restricted Boltzmann machines. The central claim is that, under a sparsity condition in which every latent variable has at least two pure observed children/neighbors, the population tensor can be unfolded into matrices whose ranks certify which observed variables are connected to which latent variables. The proof gives a population-level algorithm recovering both the number of latent variables and the adjacency matrix up to column permutation. The main theorem is supported by Proposition 1, a rank test for pure (single-neighbor) structure, and Proposition 2, a rank test for multi-neighbor structure; a side proposition addresses identifiability of the remaining continuous parameters.
Significance. If the main theorem were correct as stated, it would be a useful contribution: it extends graph identifiability beyond linear Gaussian factor models, gives a constructive population-level algorithm rather than a nonconstructive tensor-decomposition argument, and covers flexible nonlinear discrete models with cycles. The tensor-unfolding perspective and the use of pure observed variables are clear and well motivated. The paper also correctly identifies the limitation of Kruskal-based proofs, which usually require knowing the number of latent variables in advance. However, the main result is currently overstated: Proposition 1 is false as stated for observed variables with no latent connections, and the strict rank inequalities in the proofs are established only generically while Theorem 1 is stated without a generic qualifier. These are load-bearing issues, but they are local in the sense that adding a no-isolated-variable condition and a generic-parameter qualifier may repair the main claim.
major comments (3)
- [§4.1, Proposition 1 and Theorem 1] Proposition 1's 'only if' direction is false for observed variables with no latent connections, a case allowed by Theorem 1 because G* is an arbitrary binary matrix and may contain all-zero rows. Take co(j1)=∅ and co(j2)={k}. Local independence gives [T]_{j1,j2},: = P(Y_{j1}) ⊗ P(Y_{j2},Y_rest) up to row/column permutation, so its rank equals rank(P(Y_{j2},Y_rest)). Since Y_{j2} depends on the rest only through the H-state latent A_k, rank(P(Y_{j2},Y_rest)) ≤ H. Thus rank([T]_{j1,j2},:) ≤ H although co(j1) and co(j2) are not the same singleton. The constructive algorithm in Theorem 1 can therefore group an isolated observed variable together with the pure children of any latent variable and output a wrong graph. The proof in §A.1 skips exactly this case when it asserts that failure of the singleton condition implies |co(j1)∪co(j2)| ≥ 2. The theorem needs an explicit assumption excluding empty co(j) (for example, every row of G has at least one 1), and the empty case needs to be analyzed rather than implicitly excluded.
- [§A.1 and §A.2, generic qualifier] Propositions 1 and 2 are stated as deterministic equivalences under Assumptions 1 and 2, but their proofs only establish the strict rank inequalities for generic parameters. The 'only if' direction of Proposition 1 invokes Lemma 3, which is itself stated 'for generic parameters', and additionally uses generic full rank of P2 and P3 in equation (18). Proposition 2 similarly uses generic full rank of P2 and P3 in equation (19). Consequently, for parameter values lying on the excluded algebraic variety, the strict inequalities rank > H and rank > H^{K-1} can fail, and the rank certificates no longer separate the graph structures. Theorem 1 therefore needs either a generic-parameter qualifier in its statement or additional conditions that rule out the exceptional variety. This is a separate issue from the empty-neighborhood counterexample, and both must be fixed before the main claim is true.
- [§A.2, proof of Proposition 2] The proof of the 'if' direction of Proposition 2 invokes Lemma 3 to lower-bound rank(P1,2), but P1,2 is a conditional probability table for |co(j)| observed variables, not a two-variable unfolding. The lower bound rank(P1,2) > H^{|co(S1)|} appears to follow from Lemma 2 with S1=co(j)\setminus{k} and S2={j}, not from Lemma 3. This is likely a citation error, but it should be corrected because the current reference makes the proof difficult to verify.
minor comments (4)
- [§2.3] The word 'bipartige' is a typo for 'bipartite' in the discussion of cognitive diagnostic models.
- [§3] The sentence 'For a set S, denotes its cardinality by |S|' is missing a subject; it should read 'For a set S, we denote its cardinality by |S|'.
- [§4.3, Proposition 3(a)] The statement relies on Theorem 3.3 of Gu (2024), cited as 'Bernoulli, page to appear'. If that reference is still in press, the paper should make the dependence explicit in the main text so readers can assess the side result.
- [§B.2, Lemma 2 proof] In the proof, the sentence '1^T E_{β_{h1},r} = 1^T E_{β_{h1},r} = 0' should more precisely say that both E_{β_{h1},r} and E_{β_{h2},r} are conditional probability vectors, so each sums to 1 and their difference sums to 0; the displayed equality as written appears to contain a typo.
Circularity Check
No circular derivation: the graph-identifiability proof proceeds from the model's own tensor factorizations; the only self-citation supports a side result, not the main theorem.
-
self citation load bearing
[Supplementary Material, Section B.5, proof of Proposition 3(a)]
"To prove part (a) of the proposition, consider H = 2 with binary latent variables and G = ( IK; IK)⊤. In this case, the latent bipartite graphical model reduces to the so-called BLESS model in Gu (2024) with a star-forest graph between the observed layer and the latent layer. Therefore, we can use Theorem 3.3 in Gu (2024) to obtain that if Ak is independent of A[K]\k for some k ∈ [K], then the model parameters (Θ, ν) are not identifiable."
This step imports Theorem 3.3 from the author's own prior work (Gu 2024) to establish a side result on non-identifiability of continuous parameters when each latent variable has exactly two pure observed children. It is a self-citation, but it is not load-bearing for the paper's central claim: Theorem 1's graph identifiability is proved directly from tensor unfoldings and rank certificates, not from this theorem. The cited result supports only the complementary Proposition 3(a), so the main derivation is not circular; the self-citation is minor and isolated.
full rationale
The main derivation chain in this paper is self-contained rather than circular. Theorem 1 is proved through Propositions 1 and 2, which analyze ranks of explicitly unfolded population tensors. The rank upper bounds in the 'if' directions come directly from factorizations such as [T]_{j1,j2},: = P(Y_{j1,j2}|A_k) P(A_k,A) P(Y_{-{j1,j2}}|A)^T, where the number of columns of the middle matrix is H, immediately giving rank ≤ H. The 'only if' directions use Lemmas 2–4 and generic-parameter rank arguments, again starting from the model definition and the conditional independence of observed variables given latents. These arguments do not assume the conclusion; they compute ranks of matrices defined from the population distribution and compare them to thresholds derived from the factorization. Likewise, Proposition 2's certificate rank([T]_{B_j,C}) > H^{K-1} iff k ∈ co(j) is established by writing the unfolded matrix as a product of conditional probability tables and applying rank lower bounds; it is not obtained by assuming the graph. The single self-citation to Gu (2024) occurs only in the proof of Proposition 3(a), a side result about parameter non-identifiability under exactly two pure children per latent, and Proposition 3(b) is proved via Kruskal's theorem rather than by relying on that citation. Thus the central graph-identifiability result does not reduce to its own inputs or to a self-citation chain. The skeptical concern about isolated observed variables with co(j)=∅ and the generic-parameter caveat in Lemmas 3 and the supplement is a correctness gap in the stated rank certificate, but it is not a circularity: the rank certificate is still derived from the model, just possibly incorrectly. Since there is one minor, non-load-bearing self-citation, the circularity score is 2 rather than 0.
Assumptions & free parameters
assumptions (5)
- domain assumption Assumption 1: V >= H, and every observed variable with a single latent parent/neighbor has a full-column-rank conditional probability table.
- domain assumption Assumption 2: for every edge (j,k), some configuration of the other parents makes the conditional distribution of Y_j vary with the value of A_k.
- ad hoc to paper Generic parameter values: all strict rank inequalities are required to hold outside a Lebesgue measure zero algebraic variety.
- domain assumption Each latent variable has at least two pure observed variables, i.e., G contains two stacked identity blocks after row permutation.
- ad hoc to paper Every observed variable is connected to at least one latent variable.
Cite this review
Pith. "Pith review of Unfolding Tensors to Identify the Graph in Discrete Latent Bipartite Graphical Models." pith.science (2026). https://pith.science/paper/UFQQUFGU
@misc{pith2026250110897,
author = {Pith},
title = {Pith review of: Unfolding Tensors to Identify the Graph in Discrete Latent Bipartite Graphical Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/UFQQUFGU}},
note = {Machine review of arXiv:2501.10897}
}
read the original abstract
We use a tensor unfolding technique to prove a new identifiability result for discrete bipartite graphical models, which have a bipartite graph between an observed and a latent layer. This model family includes popular models such as Noisy-Or Bayesian networks for medical diagnosis and Restricted Boltzmann Machines in machine learning. These models are also building blocks for deep generative models. Our result on identifying the graph structure enjoys the following nice properties. First, our identifiability proof is constructive, in which we innovatively unfold the population tensor under the model into matrices and inspect the rank properties of the resulting matrices to uncover the graph. This proof itself gives a population-level structure learning algorithm that outputs both the number of latent variables and the bipartite graph. Second, we allow various forms of nonlinear dependence among the variables, unlike many continuous latent variable graphical models that rely on linearity to show identifiability. Third, our identifiability condition is interpretable, only requiring each latent variable to connect to at least two "pure" observed variables in the bipartite graph. The new result not only brings novel advances in algebraic statistics, but also has useful implications for these models' trustworthy applications in scientific disciplines and interpretable machine learning.
Figures
Reference graph
Works this paper leans on
-
[1]
Allman, E. S., Matias, C., and Rhodes, J. A. (2009). Identifiability of parameters in latent structure models with many observed variables. The Annals of Statistics , 37(6A):3099--3132
work page 2009
-
[2]
Allman, E. S. and Rhodes, J. A. (2006). The identifiability of tree topology for phylogenetic models, including covarion and mixture models. Journal of Computational Biology , 13(5):1101--1113
work page 2006
-
[3]
Bing, X., Bunea, F., Ning, Y., and Wegkamp, M. (2020). Adaptive estimation in structured factor models with applications to overlapping clustering. Annals of Statistics , 48(4):2055--2081
work page 2020
-
[4]
Bishop, C. M. (2006). Pattern recognition and machine learning , volume 4. Springer
work page 2006
-
[5]
Culpepper, S. A. (2019). An exploratory diagnostic model for ordinal responses with binary attributes: identifiability and estimation. Psychometrika , 84(4):921--940
work page 2019
-
[6]
de la Torre, J. (2011). The generalized DINA model framework. Psychometrika , 76:179--199
work page 2011
-
[7]
Eriksson, N. (2005). Tree construction using singular value decomposition. New York, NY: Cambridge University Press, 2005. , pages 347--358
work page 2005
-
[8]
E., Hersh, P., Rinaldo, A., and Zhou, Y
Fienberg, S. E., Hersh, P., Rinaldo, A., and Zhou, Y. (2009). Maximum likelihood estimation in latent class models for contingency table data , page 27–62. Cambridge University Press
work page 2009
Show all 47 references
-
[9]
Glymour, C., Scheines, R., Spirtes, P., and Kelly, K. (1987). Discovering causal structure: Artificial intelligence, philosophy of science, and statistical modeling
1987
-
[10]
Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning . MIT press
2016
-
[11]
Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika , 61(2):215--231
1974
-
[12]
Gu, Y. (2024). Blessing of dependence: Identifiability and geometry of discrete models with multiple binary latent variables. Bernoulli , page to appear
2024
-
[13]
and Dunson, D
Gu, Y. and Dunson, D. B. (2023). Bayesian Pyramids: identifiable multilayer discrete latent structure models for discrete data . Journal of the Royal Statistical Society Series B: Statistical Methodology , 85(2):399--426
2023
-
[14]
and Xu, G
Gu, Y. and Xu, G. (2023). A joint MLE approach to large-scale structured latent attribute analysis. Journal of the American Statistical Association , 118(541):746--760
2023
-
[15]
A., Templin, J
Henson, R. A., Templin, J. L., and Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika , 74:191--210
2009
-
[16]
Hinton, G. E. and Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science , 313(5786):504--507
2006
-
[17]
D., Calhoun, V
Hjelm, R. D., Calhoun, V. D., Salakhutdinov, R., Allen, E. A., Adali, T., and Plis, S. M. (2014). Restricted B oltzmann machines for neuroimaging: an application in identifying intrinsic networks. NeuroImage , 96:245--260
2014
-
[18]
Ishteva, M., Park, H., and Song, L. (2013). Unfolding latent tree structures using 4th order tensors. In International Conference on Machine Learning , pages 316--324. PMLR
2013
-
[19]
Jaakkola, T. S. and Jordan, M. I. (1999). Variational probabilistic inference and the QMR-DT network. Journal of Artificial Intelligence Research , 10:291--322
1999
-
[20]
T., and Kluger, Y
Jaffe, A., Amsel, N., Aizenbud, Y., Nadler, B., Chang, J. T., and Kluger, Y. (2021). Spectral neighbor joining for reconstruction of latent tree models. SIAM Journal on Mathematics of Data Science , 3(1):113--141
2021
-
[21]
Junker, B. W. and Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement , 25:258--272
2001
-
[22]
Khemakhem, I., Kingma, D., Monti, R., and Hyvarinen, A. (2020). Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics , pages 2207--2217. PMLR
2020
-
[23]
Kivva, B., Rajendran, G., Ravikumar, P., and Aragam, B. (2022). Identifiability of deep generative models without auxiliary information. Advances in Neural Information Processing Systems , 35:15687--15701
2022
-
[24]
Kolda, T. G. and Bader, B. W. (2009). Tensor decompositions and applications. SIAM review , 51(3):455--500
2009
-
[25]
and Friedman, N
Koller, D. and Friedman, N. (2009). Probabilistic graphical models: principles and techniques . MIT press
2009
-
[26]
Koopmans, T. C. and Reiersol, O. (1950). The identification of structural characteristics. The Annals of Mathematical Statistics , 21(2):165--181
1950
-
[27]
Kruskal, J. B. (1977). Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and its Applications , 18(2):95--138
1977
-
[28]
Lauritzen, S. L. (1996). Graphical models , volume 17. Clarendon Press
1996
-
[29]
Lazarsfeld, P. F. and Henry, N. W. (1968). Latent Structure Analysis . Houghton Mifflin, Boston
1968
-
[30]
and Drton, M
Leung, D. and Drton, M. (2018). Algebraic tests of general G aussian latent tree models. Advances in Neural Information Processing Systems , 31:6301--6310
2018
-
[31]
G., Carleo, G., Carrasquilla, J., and Cirac, J
Melko, R. G., Carleo, G., Carrasquilla, J., and Cirac, J. I. (2019). Restricted B oltzmann machines in quantum physics. Nature Physics , 15(9):887--892
2019
-
[32]
E., Sridhar, D., Wang, Y., and Blei, D
Moran, G. E., Sridhar, D., Wang, Y., and Blei, D. M. (2021). Identifiable deep generative models via sparse decoding. arXiv preprint arXiv:2110.10804
2021 arXiv
-
[33]
Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference . Morgan Kaufmann
1988
-
[34]
Salakhutdinov, R., Mnih, A., and Hinton, G. (2007). Restricted B oltzmann machines for collaborative filtering. In Proceedings of the 24th international conference on Machine learning , pages 791--798
2007
-
[35]
Scheines, R., Spirtes, P., Glymour, C., Meek, C., and Richardson, T. (1998). The TETRAD project: Constraint based aids to causal model specification. Multivariate Behavioral Research , 33(1):65--117
1998
-
[36]
A., and Smith, J
Shiers, N., Zwiernik, P., Aston, J. A., and Smith, J. Q. (2016). The correlation space of G aussian latent tree models and model selection without fitting. Biometrika , 103(3):531--545
2016
-
[37]
A., Middleton, B., Heckerman, D
Shwe, M. A., Middleton, B., Heckerman, D. E., Henrion, M., Horvitz, E. J., Lehmann, H. P., and Cooper, G. F. (1991). Probabilistic diagnosis using a reformulation of the INTERNIST-1/QMR knowledge base. Methods of Information in Medicine , 30(04):241--255
1991
-
[38]
Silva, R., Scheines, R., Glymour, C., Spirtes, P., and Chickering, D. M. (2006). Learning the structure of linear latent variable models. Journal of Machine Learning Research , 7(2)
2006
-
[39]
and Sidiropoulos, N
Stegeman, A. and Sidiropoulos, N. D. (2007). On K ruskal’s uniqueness condition for the Candecomp/Parafac decomposition. Linear Algebra and its Applications , 420:540--552
2007
-
[40]
von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology , 61(2):287--307
2008
-
[41]
and Lee, Y.-S
von Davier, M. and Lee, Y.-S. (2019). Handbook of diagnostic classification models. Cham: Springer International Publishing
2019
-
[42]
u gelgen, J., Besserve, M., Wendong, L., Gresele, L., Keki \'c , A., Bareinboim, E., Blei, D., and Sch \
von K \"u gelgen, J., Besserve, M., Wendong, L., Gresele, L., Keki \'c , A., Bareinboim, E., Blei, D., and Sch \"o lkopf, B. (2024). Nonparametric identifiability of causal representations from unknown interventions. Advances in Neural Information Processing Systems , 36
2024
-
[43]
Wainwright, M. J. and Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning , 1(1--2):1--305
2008
-
[44]
Wu, C., Lee, H., and Ge, R. (2023). Connecting pre-trained language model and downstream task via properties of representation. Advances in Neural Information Processing Systems , 36:47216--47238
2023
-
[45]
Xu, G. (2017). Identifiability of restricted latent class models with binary responses. The Annals of Statistics , 45(2):675--707
2017
-
[46]
Zwiernik, P. (2018). Latent tree models. In Handbook of Graphical Models , pages 267--290. CRC Press
2018
-
[47]
and Smith, J
Zwiernik, P. and Smith, J. Q. (2012). Tree cumulants and the geometry of binary tree models. Bernoulli , 18(1):290--321
2012
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.