REVIEW 4 major objections 4 minor 34 references
Causal Representation Learning from Network Data
T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A graph-aware VAE inherits causal identifiability and improves CRISPR intervention prediction.
desk verdict GRACE-VAE is a useful empirical architecture for incorporating network context into causal representation learning, but its identifiability proof rests on a false algebraic claim, and the headline empirical claim overstates a baseline comparison. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing design choice is the placement of the graph neural network: GRACE-VAE inserts the GNN in the variational encoder only, so the generative likelihood p(X|U) and the latent SCM over U are identical to the model in Zhang et al. (2023). Identifiability then follows from the existing three-step proof: Lemma 1 identifies the latent vector up to an affine map from the observational distribution; Assumption 2 (linear interventional faithfulness) uses the changed marginal distributions under each intervention to pin down the coordinate permutation; Assumption 3 (total separation) orients every edge of the DAG. The remaining freedom is the CD-equivalence class. The intervention encoder
What would settle it
Run GRACE-VAE on synthetic data generated from a known latent SCM using an MLP decoder with leaky ReLU activations and the same training loss; if the recovered DAG and intervention targets are not equivalent up to permutation, scaling, and shift, the implemented model does not inherit the identifiability guarantee.
Extended reading notes
Core claim
The paper's central claim is that causal representation learning from soft interventions remains identifiable when the observed data carry network structure, provided the graph information is injected only into the variational encoder. GRACE-VAE instantiates this idea: a GraphSAGE (or other GNN) encoder processes the observed entities and their auxiliary group nodes; the decoder is an unchanged latent structural causal model with a soft intervention encoder; training couples an ELBO reconstruction term with an MMD alignment term that matches generated counterfactuals to observed interventional distributions. Because the GNN never enters the likelihood, the identifiability theorem from Zhang
Load-bearing premise
The identifiability guarantee holds only if the decoder is a polynomial map with full rank and non-empty latent support, and the intervention data satisfy the paper's faithfulness and separation conditions; the implemented MLP decoder and the real biological datasets are not verified against these conditions.
Editorial extensions
If this is right
- If the claim holds, any observed network with observational plus soft-interventional data can be used to recover latent causal variables and intervention targets without knowing those targets, up to relabeling and scaling.
- Structured context yields measurable gains: on NORMAN, single-intervention R² rises from 0.902 with the flat baseline to 0.945 with GRACE-VAE, and double-intervention R² rises from 0.738 to 0.785.
- Ablations show that both components matter: removing the causal decoder or removing the graph encoder degrades performance, so the benefit comes from combining network-aware inference with an explicit causal generative model.
- The identifiability guarantee is insensitive to the choice of GNN encoder, so future encoder architectures can be swapped in without re-proving identifiability.
- Richer pathway and gene topology helps single interventions but shows diminishing or slightly negative returns for unseen double interventions, suggesting context level should be chosen per task.
Reading between the lines
- An implication left implicit is that, since the guarantee concerns only the generative side, the GNN could be replaced by any sufficiently expressive inference network over the network and the identifiability result would still hold.
- The paper does not verify that real CRISPR data satisfy the faithfulness and separation assumptions or that the implemented MLP decoder is a polynomial map; a synthetic benchmark with known ground-truth SCM and the exact architecture would reveal whether the implemented model's recoveries are actually CD-equivalent.
- A testable extension suggested by the framing: adding graph context that is causally irrelevant (for example, permuted pathway assignments) should not improve intervention prediction; if it does, the gain is likely regularization rather than recovered causal structure.
- The same architecture should transfer to other domains with latent interventions and observed relational structure, such as social influence, ecology, or drug combination screens, where network context could act as weak supervision for latent mechanisms.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GRACE-VAE, a variational autoencoder that injects structured biological network context (pathway-gene, pathway-pathway, and protein-protein interactions) through a GNN encoder, while keeping a causal SCM-based decoder. It claims that, because the GNN is placed only in the variational encoder, the model inherits the identifiability guarantees of causal discrepancy VAEs and can recover the latent causal graph and intervention targets up to CD-equivalence. The authors evaluate GRACE-VAE on three CRISPR perturbation datasets, including unseen double interventions, and compare against CMVAE, CMVAE-multihot, and VGAE, with ablations over GNN architecture and context levels. The central claims are: (1) theoretical identifiability of the latent causal structure, and (2) empirical improvement in interventional outcome prediction from using structured context.
Significance. If the theoretical claim were correct and the empirical improvements robust, this would be a useful contribution to causal representation learning in structured biological settings. The empirical study is fairly comprehensive: three datasets, 10 seeds, multiple baselines, ablations, and hyperparameter reporting. The idea of using a GNN encoder to provide structural context while leaving the causal decoder unchanged is principled and could be practically valuable. However, the theoretical guarantee as presented is not established: the appendix contains a false lemma, and the implemented model does not satisfy the polynomial-decoder assumption of the theorem. The empirical claim that GRACE-VAE outperforms all baselines on R2 is also contradicted by the paper's own Table 1. The core idea has potential, but the paper needs substantial correction before the claims are supportable.
major comments (4)
- [Appendix, Lemma 1 / Theorem 1] Lemma 1 is false as stated, and its proof collapses. The proof's final step invokes a 'fundamental theorem of algebra for polynomial automorphisms' to conclude that two mutually inverse polynomial maps must have degree 1. That theorem is false: F(x,y)=(x+y^2,y) is a polynomial automorphism of R^2 with polynomial inverse (x-y^2,y), is full-rank everywhere, and is not affine. Concretely, take f=identity and \f=F^{-1}, and \U=F(U). Then \f(\U)=U, so the observational distribution is identical, but \U is not ΛU+b. Since Theorem 1's proof begins with Lemma 1 to establish an affine reparameterization, the CD-equivalence theorem is not established as written. The authors must either prove a corrected lemma under additional hypotheses or explicitly defer to Zhang et al. (2023) and verify that all of that paper's assumptions hold for the present setting.
- [Identifiability guarantee / Implementation details] Assumption 1 assumes a full-row-rank polynomial decoder, but the implemented decoder is a DAG layer plus two fully connected layers with leaky ReLU activations, which is not a polynomial map. Assumptions 2 and 3 (linear interventional faithfulness and total separation) are also never verified on the CRISPR datasets. Therefore the statement that 'GRACE-VAE inherits the identifiability guarantee and identifies the latent causal graph and intervention targets up to CD-equivalence' does not apply to the architecture and data actually evaluated. This is a load-bearing gap between theory and experiment.
- [Results, R1 / Table 1] The text states: 'Across the three datasets, GRACE-VAE outperforms all baselines on R2 (Table 1).' This is contradicted by Table 1: on NORMAN single-intervention R2, VGAE achieves 0.9493±0.0123 versus GRACE-VAE's 0.9452±0.0108. The claim needs to be qualified: GRACE-VAE outperforms CMVAE and CMVAE-multihot on this metric, and it has the best or comparable RMSE/MMD, but it is not strictly the best on single R2. This overstatement affects the paper's primary empirical contribution.
- [Conclusion and Experimental Setup] The conclusion claims that 'GRACE-VAE yields faithful latent DAGs' and that the experiments demonstrate recovery of the true latent causal graph. However, no structure-learning metric (e.g., SHD, edge AUROC, target recovery accuracy) is reported against a ground-truth latent DAG. The three datasets are real perturbation screens without known latent SCMs, and the reported metrics evaluate prediction of interventional outcomes only. To support the 'recovery' claim, the authors should include synthetic experiments with known latent graphs and report structural accuracy.
minor comments (4)
- [Identifiability guarantee] There is a typo in the heading ('guanrantee') and in the text: 'leaves the observational distribution PX changed' should read 'leaves the observational distribution PX unchanged' (or 'does not change').
- [References] The citation Fine and Rosenberger (1997) is a textbook on the fundamental theorem of algebra; it does not support the invoked statement about polynomial automorphisms. If the authors believe such a theorem exists, they should cite a proper source and state the exact conditions.
- [Implementation details] The hyperparameter selection is described as based on 'R2, MMD, and RMSE' collectively, which is ambiguous. Please specify the selection rule (e.g., a primary metric or a rank-based aggregate).
- [Figure 2] The learned DAG in Figure 2 would benefit from a description of how edges are thresholded (if at all) and how edge weights are normalized, so that the visualization is reproducible.
Circularity Check
Identifiability guarantee is imported from the authors' own prior work via a self-citation chain, and the appendix's re-derivation relies on a false theorem about polynomial automorphisms; the central theoretical claim is not independently established.
-
uniqueness imported from authors
[Identifiability guarantee (main text) and Appendix, Theorem 1 proof]
"In our setting, the identifiability guarantee still holds under assumptions 1, 2, and 3 in Zhang et al. (2023). G RACE-VAE enriches inference but leaves both the likelihood and the latent causal structure unchanged, so the original identifiability guarantee remains valid."
The paper's central theoretical claim is not derived in this paper; it is stated to 'inherit' the identifiability theorem of Zhang et al. (2023), whose first author is also the present paper's first author. The appendix's proof of Theorem 1 then defers the permutation-pinning and edge-orientation steps to 'Lemmas 5, 6, 7, and 8 in Zhang et al. 2023' and 'Theorem 2 (Zhang et al. 2023)', so the uniqueness conclusion is imported wholesale from the same research group. This is load-bearing: if the cited theorem or its assumptions fail, the paper's guarantee has no independent support. The implementation's decoder (DAG layer + MLP with leaky ReLU) is never checked against Assumption 1's polynomial full-row-rank requirement, so the import cannot be validated independently here.
-
other
[Appendix: Identifiability guarantee, Lemma 1 proof]
"By the fundamental theorem of algebra for polynomial automorphisms (Fine and Rosenberger 1997), poly1 and poly2 must each be of degree 1."
This step is the hinge of Lemma 1: it turns 'bU is a polynomial in U and vice versa' into the affine conclusion bU = Lambda U + b. The cited 'fundamental theorem of algebra for polynomial automorphisms' is false; F(x,y)=(x+y^2, y) is a polynomial automorphism with polynomial inverse (x-y^2, y) and is not degree 1. Therefore the lemma's conclusion is not a consequence of Assumption 1. The proof effectively assumes the degree-1 outcome it needs to establish, so the identifiability guarantee reduces to an unstated and false premise rather than to a proved theorem.
full rationale
The paper's central theoretical contribution is the claim that GraCE-VAE inherits CD-identifiability for latent causal graphs and intervention targets. The appendix proof is not self-contained: it explicitly defers to Lemmas 5-8 and Theorem 2 of Zhang et al. (2023), whose first author overlaps with the present paper. The main text states the guarantee holds 'under assumptions 1, 2, and 3 in Zhang et al. (2023)' and that 'the original identifiability guarantee remains valid,' making the theoretical core a load-bearing self-citation. Moreover, the appendix's Lemma 1 proof invokes a false 'fundamental theorem of algebra for polynomial automorphisms' to conclude that polynomial inverses must be degree 1; the counterexample F(x,y)=(x+y^2,y) shows the step is invalid. Consequently, the affine identifiability lemma, and hence Theorem 1, is not established by the paper's own argument. The paper also does not verify that the implemented decoder (DAG layer plus MLP with leaky ReLU) satisfies Assumption 1's polynomial full-row-rank requirement, so even the imported theorem may not apply to the architecture actually evaluated. Separately, Table 1 shows VGAE's NORMAN single R2 (0.9493) exceeds GraCE-VAE's (0.9452), contradicting the text's 'outperforms all baselines on R2'; this is an empirical overclaim rather than a circularity, but it further weakens support. The empirical double-intervention experiments are independent and not circular, but the central theoretical claim reduces to a self-citation chain and a defective proof step, warranting a score of 6.
Assumptions & free parameters
free parameters (6)
- Latent dimension p =
105 (NORMAN), 414 (REPLOGLE-SMALL), 300 (REPLOGLE-LARGE)
- MMD alignment coefficient alpha_max =
8, 10, 12, or 15 depending on dataset/model
- KL coefficient beta_max =
2, 3, or 4 depending on dataset/model
- DAG sparsity penalty lambda =
0.0001 or 0.001
- Softmax temperature Temp =
4, 5, or 6 depending on dataset/model
- GNN architecture and layer count =
1-layer GraphSAGE chosen; 1 or 3 layers in ablations
assumptions (6)
- domain assumption Assumption 1: full-row-rank polynomial decoder f with non-empty interior support of PU
- domain assumption Assumption 2: linear interventional faithfulness
- domain assumption Assumption 3: total separation for every edge
- domain assumption Faithfulness: latent graph G and latent distribution PU are faithful to each other
- domain assumption Each latent variable is targeted by at least one intervention
- standard math Polynomial automorphisms that compose to the identity must have degree 1
Cite this review
Pith. "Pith review of Causal Representation Learning from Network Data." pith.science (2026). https://pith.science/paper/QG257I7O
@misc{pith2026250901916,
author = {Pith},
title = {Pith review of: Causal Representation Learning from Network Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/QG257I7O}},
note = {Machine review of arXiv:2509.01916}
}
read the original abstract
Causal disentanglement from soft interventions is identifiable under the assumptions of linear interventional faithfulness and availability of both observational and interventional data. Prior work has focused on unstructured observations without leveraging known relational context among measured entities. In many scientific applications, however, the measured variables come with an observed interaction network that provides structured context, such as protein-protein interactions and pathway-gene membership. We propose GraCE-VAE, a graph-aware causal discrepancy variational autoencoder that treats pathway-level information as an auxiliary view of the latent causal programs. The graph neural network encoder conditions on this auxiliary pathway view and the biological graph to improve amortized inference, while the causal decoder remains a latent SCM with soft interventions. Assuming samples are i.i.d. within each intervention regime, we show that GraCE-VAE inherits the identifiability guarantees of causal discrepancy VAEs and identifies the latent causal graph and intervention targets up to the standard equivalence class. Experiments on three CRISPR perturbation datasets demonstrate that leveraging structured biological context improves prediction of interventional outcomes, including unseen perturbation combinations.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
A.; Tang, H.; Wu, G.; Marjan, S.; and Hussain, A
Bhatti, U. A.; Tang, H.; Wu, G.; Marjan, S.; and Hussain, A. 2023. Deep learning with graph convolutional networks: An overview and latest applications in computational intelligence. International Journal of Intelligent Systems, 2023(1): 8342104
work page 2023
-
[4]
Brehmer, J.; De Haan, P.; Lippe, P.; and Cohen, T. S. 2022. Weakly supervised causal representation learning. Advances in Neural Information Processing Systems, 35: 38319--38331
work page 2022
-
[5]
Buchholz, S.; Rajendran, G.; Rosenfeld, E.; Aragam, B.; Sch \"o lkopf, B.; and Ravikumar, P. 2024. Learning linear causal representations from interventions under general nonlinear mixing. Advances in Neural Information Processing Systems, 36
work page 2024
-
[6]
Chickering, D. M. 2002. Optimal structure identification with greedy search. Journal of machine learning research, 3(Nov): 507--554
work page 2002
-
[7]
Cussens, J. 2020. GOBNILP: Learning Bayesian network structure with integer programming. In International Conference on Probabilistic Graphical Models, 605--608. PMLR
work page 2020
-
[8]
Feng, J.; Zhang, L.; and Yang, L. 2023. Concept-free causal disentanglement with variational graph auto-encoder. arXiv preprint arXiv:2311.10638
work page Pith review arXiv 2023
Show all 34 references
-
[9]
Fine, B.; and Rosenberger, G. 1997. The fundamental theorem of algebra. Springer Science & Business Media
1997
-
[10]
M.; Rasch, M
Gretton, A.; Borgwardt, K. M.; Rasch, M. J.; Sch \"o lkopf, B.; and Smola, A. 2012. A kernel two-sample test. The journal of machine learning research, 13(1): 723--773
2012
-
[11]
Hamilton, W.; Ying, Z.; and Leskovec, J. 2017. Inductive representation learning on large graphs. Advances in neural information processing systems, 30
2017
-
[12]
Higgins, I.; Amos, D.; Pfau, D.; Racaniere, S.; Matthey, L.; Rezende, D.; and Lerchner, A. 2018. Towards a definition of disentangled representations. arXiv preprint arXiv:1812.02230
2018 arXiv
-
[13]
P.; Glorot, X.; Botvinick, M
Higgins, I.; Matthey, L.; Pal, A.; Burgess, C. P.; Glorot, X.; Botvinick, M. M.; Mohamed, S.; and Lerchner, A. 2017. beta-vae: Learning basic visual concepts with a constrained variational framework. ICLR (Poster), 3
2017
-
[14]
Jassal, B.; Matthews, L.; Viteri, G.; Gong, C.; Lorente, P.; Fabregat, A.; Sidiropoulos, K.; Cook, J.; Gillespie, M.; Haw, R.; et al. 2020. The reactome pathway knowledgebase. Nucleic acids research, 48(D1): D498--D503
2020
-
[15]
Khemakhem, I.; Monti, R.; Leech, R.; and Hyvarinen, A. 2021. Causal autoregressive flows. In International conference on artificial intelligence and statistics, 3520--3528. PMLR
2021
-
[16]
N.; and Welling, M
Kipf, T. N.; and Welling, M. 2016. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308
2016 arXiv
-
[17]
Kong, L.; Huang, B.; Xie, F.; Xing, E.; Chi, Y.; and Zhang, K. 2023. Identification of nonlinear latent hierarchical models. Advances in Neural Information Processing Systems, 36: 2010--2032
2023
-
[18]
Kumar, A.; Sattigeri, P.; and Balakrishnan, A. 2017. Variational inference of disentangled latent concepts from unlabeled observations. arXiv preprint arXiv:1711.00848
2017 arXiv
-
[19]
M.; Huang, Y.; Sumathipala, M.; Liang, M
Li, M. M.; Huang, Y.; Sumathipala, M.; Liang, M. Q.; Valdeolivas, A.; Ananthakrishnan, A. N.; Liao, K.; Marbach, D.; and Zitnik, M. 2024. Contextual AI models for single-cell protein biology. Nature Methods, 21(8): 1546--1557
2024
-
[20]
M.; Cohen, T.; and Gavves, S
Lippe, P.; Magliacane, S.; L \"o we, S.; Asano, Y. M.; Cohen, T.; and Gavves, S. 2022. Citris: Causal identifiability from temporal intervened sequences. In International Conference on Machine Learning, 13557--13603. PMLR
2022
-
[21]
Massidda, R.; Geiger, A.; Icard, T.; and Bacciu, D. 2023. Causal abstraction with soft interventions. In Conference on Causal Learning and Reasoning, 68--87. PMLR
2023
-
[22]
Pearl, J. 2000. Causality: Models, Reasoning, and Inference. Cambridge University Press
2000
-
[23]
Raskutti, G.; and Uhler, C. 2018. Learning directed acyclic graph models based on sparsest permutations. Stat, 7(1): e183
2018
-
[24]
M.; Saunders, R
Replogle, J. M.; Saunders, R. A.; Pogson, A. N.; Hussmann, J. A.; Lenail, A.; Guna, A.; Mascibroda, L.; Wagner, E. J.; Adelman, K.; Lithwick-Yanai, G.; et al. 2022. Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seq. Cell, 185(14): 2559--2575
2022
-
[25]
Sanchez-Martin, P.; Rateike, M.; and Valera, I. 2021. Vaca: Design of variational graph autoencoders for interventional and counterfactual queries. arXiv preprint arXiv:2110.14690
2021 arXiv
-
[26]
Solus, L.; Wang, Y.; and Uhler, C. 2021. Consistency guarantees for greedy permutation-based causal inference algorithms. Biometrika, 108(4): 795--814
2021
-
[27]
S.; and Uhler, C
Squires, C.; Seigal, A.; Bhate, S. S.; and Uhler, C. 2023. Linear causal disentanglement via interventions. In International Conference on Machine Learning, 32540--32560. PMLR
2023
-
[28]
Suter, R.; Miladinovic, D.; Sch \"o lkopf, B.; and Bauer, S. 2019. Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness. In International Conference on Machine Learning, 6056--6065. PMLR
2019
-
[29]
Tao, H.; Yu, H.; and Li, J. 2024. DeepITE: Designing Variational Graph Autoencoders for Intervention Target Estimation. Advances in Neural Information Processing Systems, 37: 89978--90008
2024
-
[30]
Varici, B.; Acarturk, E.; Shanmugam, K.; Kumar, A.; and Tajer, A. 2023. Score-based causal representation learning with interventions. arXiv preprint arXiv:2301.08230
2023 arXiv
-
[31]
Veli c kovi \'c , P.; Cucurull, G.; Casanova, A.; Romero, A.; Lio, P.; and Bengio, Y. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903
2017 arXiv
-
[32]
Yang, M.; Liu, F.; Chen, Z.; Shen, X.; Hao, J.; and Wang, J. 2021. Causalvae: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9593--9602
2021
-
[33]
Zhang, J.; Greenewald, K.; Squires, C.; Srivastava, A.; Shanmugam, K.; and Uhler, C. 2023. Identifiability guarantees for causal disentanglement from soft interventions. Advances in Neural Information Processing Systems, 36: 50254--50292
2023
-
[34]
Zhu, X.; Xu, C.; and Tao, D. 2021. Commutative lie group vae for disentanglement learning. In International Conference on Machine Learning, 12924--12934. PMLR
2021
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.