REVIEW 2 major objections 4 minor 42 references
Factored space models: Towards causality between levels of abstraction
T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read One independence criterion is proven exact for deterministic variables.
desk verdict A careful finite-discrete theorem about structural vs conditional independence, with a real DAG-to-FSM construction; the continuous framing overstates the scope. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the history $H(X \mid C)$, the unique minimal subset $J$ of the factor index set such that the background variables $U_J$ determine $X$ on $C$ and $J$ disintegrates $C$, meaning $C$ factorizes as $C_J \times C_{I \setminus J}$. Structural independence compares these histories: $X \perp_\Omega Y \mid Z$ holds exactly when $H(X \mid z) \cap H(Y \mid z) = \emptyset$ for every value $z$. The completeness proof introduces the cohistory—the set of factors whose variation never changes $P(A \mid C)$ under factorizing distributions—and establishes that cohistory equals history, so independence in all factorizing distributions forces the relevant factor sets to be disjoint. A local-to-global lemma treats each conditional-independence statement as a polynomial that vanishes on an open set of distributions and therefore vanishes everywhere, which also yields the paper's strong completeness result.
What would settle it
A concrete refutation would be a finite factored space with events $A$, $B$, $C$ such that $A$ and $B$ are conditionally independent given $C$ under every factorizing distribution but $H(A \mid C) \cap H(B \mid C)$ is nonempty; the proof of Lemmas C.7 and C.8 says no such triple exists, so exhibiting one would overturn Theorem 6.2.
Extended reading notes
Core claim
The paper proves Theorem 6.2: for random variables $X$, $Y$, $Z$ on a finite factored space $\Omega = \times_{i \in I} \Omega_i$, $X$ and $Y$ are structurally independent given $Z$ if and only if $X$ and $Y$ are conditionally independent given $Z$ in every probability distribution $P$ that factorizes over $\Omega$. Structural independence means that for every value $z$, the history $H(X \mid z)$—the minimal set of factors needed to determine $X$ on the event $Z = z$—is disjoint from $H(Y \mid z)$. The theorem is proved first for events, with the history shown to contain exactly those factors that are probabilistically relevant to the event given the conditioning set. When the factored space is constructed from a causal directed acyclic graph, structural independence of node variables is equivalent to d-separation, so the classical soundness and completeness theorem for d-separation follows as a special case; additionally, some distributions have a perfect-map factored space but no perfect-map DAG, showing the framework is strictly more expressive.
Load-bearing premise
The theorem assumes finiteness throughout—finite index set, finite factors, and finite value spaces—and the proof uses that finiteness directly, so continuous and infinite settings are outside the stated result.
Editorial extensions
If this is right
- Any system modeled as independent sources of randomness gets a distribution-free independence test: two variables are independent in every factorizing distribution exactly when they read off disjoint sources, and this holds even when one variable is a deterministic function of another.
- The classical soundness and completeness theorem for d-separation in Bayesian networks becomes a special case, so factored space models inherit the independence guarantees of causal graphs while adding coverage of deterministic relationships.
- Factored space models are strictly more expressive than DAGs: there are distributions with a perfect-map factored space but no perfect-map causal graph, so the framework supports independence modeling where graph-based perfect maps do not exist.
- Structural time, defined by history inclusion, reproduces the ancestor relation for node variables in a constructed Bayesian network, giving a way to compare variables at different levels of abstraction by their sources of randomness.
- Because structural independence fails the intersection axiom while d-separation satisfies it, the framework's independence logic can represent deterministic constraints that causal graphs cannot.
Reading between the lines
- The theorem is stated for finite factored spaces only, so the motivating examples with continuous physical quantities, such as gas-particle kinetic energies, are not covered; a measure-theoretic or analytic extension is needed before those applications are justified.
- The equivalence suggests a discovery strategy: search for factorizations whose structural independences match observed conditional independences, in the same way causal discovery searches for DAGs; strong completeness says a match on any open neighborhood of factorizing distributions already certifies the structure.
- Structural time may give a formal handle on abstraction hierarchies, since a macro variable's history being a subset of a micro variable's history means the macro variable is determined no later than the micro variable in any process that reveals factors sequentially.
- The paper's speculation about self-referencing systems suggests a testable direction: represent a model's summaries of its own internal states as variables and ask whether their histories align with the flow of influence, which could be probed in language-model experiments.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. Factored space models (FSMs) represent a finite sample space as a product of independent factors and treat arbitrary variables as functions of the background factors. The paper defines histories H(X | C) and structural independence X ⊥_Ω Y | Z by disjointness of histories on each value of Z. Its central result, Theorem 6.2, states that for variables on a finite factored space, structural independence given Z holds if and only if conditional independence holds in every distribution that factorizes over the space. The proof is broken into soundness and completeness for events (Lemmas 6.3 and 6.4), with the completeness proof using 'cohistories' of probabilistically relevant factors, and a local-to-global strengthening (Proposition 6.6). The paper also proves that structural independence forms a compositional semigraphoid, constructs an FSM from any DAG, shows that node-level structural independence coincides with d-separation, and gives an example in which FSMs are strictly more expressive than DAGs as perfect maps.
Significance. If Theorem 6.2 holds, it gives a complete graph-free characterization of conditional independence in finite product spaces, including deterministic functions, and it recovers d-separation for DAG node variables as a special case. The proof in the appendix is detailed and largely self-contained, and the soundness/completeness argument via cohistories is convincing; I did not find a gap in the finite discrete setting. The local-to-global strengthening is a particularly clean addition. Two caveats temper significance: the theorem is confined to finite factored spaces, while several motivating examples are continuous; and the authors credit the original theorem to prior work [8], so the novelty lies in the proof and framing rather than in the statement itself.
major comments (2)
- [§1, §2.1, §4 Definition 4.2, §6 Definition 6.1] Theorem 6.2 is proved only for finite factored spaces (finite index set I, finite factors Ω_i, finite value spaces), as stated in Definition 4.2 and the preamble to Section 4. The motivating examples, however, are continuous or only countably infinite: temperature as a function of kinetic energies of gas particles (Section 1 and Section 2.1), object positions in images, and kinetic energies are real-valued. This is not a cosmetic mismatch. Under the paper's own Definition 6.1, conditional independence of events is defined by P(A∩C)P(B∩C)=P(A∩B∩C)P(C), with the convention that P(C)=0 makes the statement vacuously true. If Z is a non-atomic continuous variable, then P({Z=z})=0 for every factorizing distribution, so X ⊥⊥_P Y | Z holds for all X, Y, P, while structural independence generally fails. For example, on Ω=[0,1]^2 with uniform independent factors and X=Y=Z=U1+U2, the zero-probability convention makes the probabilistic independence vacuous, whereas H(X | z) ∩ H(Y | z) is nonempty for every z in (0,2). Thus the equivalence in Theorem 6.2 cannot be extended to the motivating examples without a measure-theoretic treatment of conditioning. The manuscript should either restrict all claims and examples to finite discrete systems or develop a conditional-independence notion for continuous variables (e.g., via regular conditional distributions) and prove the corresponding theorem.
- [§4 (p. 4), Theorem 6.2] The scope statement 'When we speak of variables in this paper, we always mean discrete random variables' is broader than what is proved. Definition 4.2 requires a finite index set and finite factors, and the proof of Lemma 4.9 sums over Val(X) and uses finiteness of Val(X); the interpolation arguments in Appendix C likewise use finiteness of I. Consequently the equivalence in Theorem 6.2 is established only for variables with finite value spaces. Countable discrete variables (e.g., integer-valued functions of the factors) are not covered, so the abstract's claim to generalize the d-separation theorem should be qualified to finite factored spaces, or an extension to countable or measure-theoretic settings should be supplied.
minor comments (4)
- [Appendix A, Lemma A.2 proof] In the proof of Lemma A.2, the sentence 'we have ω2 = ω' should read 'we have ω2 = ω′'; with the printed equality the subsequent inference is not valid.
- [Appendix C, proof of Lemma 6.5] In the proof of Lemma 6.5, 'Rλ → 0 as λ → 0+' should be 'Rλ → Q as λ → 0+'; as printed the convergence statement is nonsensical.
- [Definition 5.7] Definition 5.7(2) refers to independence of sets of variables X_W1, X_W2, X_W3 ⊆ X, but Section 4 defines structural and probabilistic independence only for (joint) random variables; the intended reduction to the joint variable (X_w)_{w∈W} should be stated explicitly.
- [Proposition 5.6 proof] There are spacing and typographical issues in the proof of Proposition 5.6 (e.g., '⇐ ⇒an' and the compressed equivalence chain); these should be cleaned up for readability.
Circularity Check
No circular derivation: Theorem 6.2 is proved from the paper's own definitions, with only a non-load-bearing self-citation to prior work by the first author.
full rationale
The central claim, Theorem 6.2, is derived from Definitions 4.6, 4.10, and 6.1 in a self-contained proof in Appendix C. The hard completeness direction is Lemma 6.4, which goes through the cohistory (Definition C.6) and proves the equality Cohistory(A | C) = H(A | C) in Lemma C.8 by a probabilistic relevance argument; it is not assumed by construction. The soundness direction, Lemma 6.3, is a direct factorization calculation from the history definitions. The theorem does not use the classical d-separation soundness/completeness theorem as an input; that theorem is used only externally in Proposition 5.5 to show that the FSM construction preserves d-separation for node variables, which is a derivation rather than a circular reduction. There are no fitted parameters, no predictions made from fitted subsets, and no imported uniqueness theorem. The only self-citation is reference [8] (Garrabrant), which is credited with the original factored-set framework and first proof of the theorem in the author contributions; however, the present paper restates all definitions and gives a new, self-contained proof in Appendix C, so the citation is not load-bearing. The finite/discrete scope is explicitly stated in Section 4 ('we always mean discrete random variables') and Definition 4.2 (finite index set and finite factors), so the continuous-counterexample concern raised by a skeptic is a scope caveat about the paper's framing, not a circularity in the derivation.
Assumptions & free parameters
assumptions (4)
- domain assumption All sample spaces, factor index sets, and value spaces are finite and discrete.
- domain assumption Distributions of interest factorize exactly as products of marginals over factors.
- domain assumption Background variables U_i are independent sources of randomness.
- standard math Soundness and completeness of d-separation for DAGs are taken as given.
Cite this review
Pith. "Pith review of Factored space models: Towards causality between levels of abstraction." pith.science (2026). https://pith.science/paper/Q6DKBT2J
@misc{pith2026241202579,
author = {Pith},
title = {Pith review of: Factored space models: Towards causality between levels of abstraction},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q6DKBT2J}},
note = {Machine review of arXiv:2412.02579}
}
read the original abstract
Causality plays an important role in understanding intelligent behavior, and there is a wealth of literature on mathematical models for causality, most of which is focused on causal graphs. Causal graphs are a powerful tool for a wide range of applications, in particular when the relevant variables are known and at the same level of abstraction. However, the given variables can also be unstructured data, like pixels of an image. Meanwhile, the causal variables, such as the positions of objects in the image, can be arbitrary deterministic functions of the given variables. Moreover, the causal variables may form a hierarchy of abstractions, in which the macro-level variables are deterministic functions of the micro-level variables. Causal graphs are limited when it comes to modeling this kind of situation. In the presence of deterministic relationships there is generally no causal graph that satisfies both the Markov condition and the faithfulness condition. We introduce factored space models as an alternative to causal graphs which naturally represent both probabilistic and deterministic relationships at all levels of abstraction. Moreover, we introduce structural independence and establish that it is equivalent to statistical independence in every distribution that factorizes over the factored space. This theorem generalizes the classical soundness and completeness theorem for d-separation.
Figures
Reference graph
Works this paper leans on
-
[8]
Temporal Inference with Finite Factored Sets
Scott Garrabrant. Temporal Inference with Finite Factored Sets . 2021. arXiv: 2109.11513 [cs.AI] . URL: https://arxiv.org/abs/2109.11513
work page Pith review arXiv 2021
-
[1]
Approximate Causal Abstractions
Sander Beckers, Frederick Eberhardt, and Joseph Y . Halpern. “Approximate Causal Abstractions”. In:Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25,
work page 2019
-
[2]
Causal models with constraints
Sander Beckers, Joseph Halpern, and Christopher Hitchcock. “Causal models with constraints”. In: Conference on Causal Learning and Reasoning. PMLR. 2023, pp. 866–879
work page 2023
-
[3]
Sander Beckers and Joseph Y . Halpern. “Abstracting Causal Models”. In:The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI. AAAI Press, 2019, pp. 2678–2685. DOI: 10.1609/AAAI.V33I01.33012678
-
[4]
Beyond structural causal models: Causal constraints models
Tineke Blom, Stephan Bongers, and Joris M Mooij. “Beyond structural causal models: Causal constraints models”. In: Uncertainty in Artificial Intelligence. PMLR. 2020, pp. 585–594
work page 2020
-
[5]
Daniel C. Dennett. The intentional stance. The MIT Press, 1989. ISBN : 978-0-262-54053-7
work page 1989
-
[6]
Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving
Aniket Didolkar et al. “Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving”. In: arXiv preprint arXiv:2405.12205 (2024)
arXiv 2024
-
[7]
Agent Incentives: A Causal Perspective
Tom Everitt et al. “Agent Incentives: A Causal Perspective”. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI. AAAI Press, 2021, pp. 11487–11495. DOI: 10.1609/AAAI.V35I13.17368
Show all 42 references
-
[9]
Causal abstraction for faithful model interpretation
Atticus Geiger, Chris Potts, and Thomas Icard. “Causal abstraction for faithful model interpretation”. In: arXiv preprint arXiv:2301.04709 (2023)
2023 arXiv
-
[10]
Causal abstractions of neural networks
Atticus Geiger et al. “Causal abstractions of neural networks”. In: Advances in Neural Information Processing Systems 34 (2021), pp. 9574–9586
2021
-
[11]
Identifying independence in Bayesian networks
Dan Geiger, Thomas Verma, and Judea Pearl. “Identifying independence in Bayesian networks”. In:Networks 20.5 (1990), pp. 507–534
1990
-
[12]
Reconstructing constructivism: causal models, Bayesian learning mechanisms, and the theory theory
Alison Gopnik and Henry M Wellman. “Reconstructing constructivism: causal models, Bayesian learning mechanisms, and the theory theory.” In: Psychological bulletin 138.6 (2012), p. 1085
2012
-
[13]
Language models (mostly) know what they know
Saurav Kadavath et al. “Language models (mostly) know what they know”. In: arXiv preprint arXiv:2207.05221 (2022)
2022 arXiv
-
[14]
Discovering agents
Zachary Kenton et al. “Discovering agents”. In: Artif. Intell. 322 (2023), p. 103963. DOI: 10.1016/J.ARTINT. 2023.103963
2023
-
[15]
Probabilistic graphical models: principles and techniques
Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. The MIT Press,
-
[16]
Causal Inference via Algebraic Geometry: Feasibility Tests for Functional Causal Structures with Two Binary Observed Variables
Ciarán M. Lee and Robert W. Spekkens. “Causal Inference via Algebraic Geometry: Feasibility Tests for Functional Causal Structures with Two Binary Observed Variables”. In: Journal of Causal Inference 5.2 (2017), p. 20160013. DOI: doi:10.1515/jci-2016-0013
2017 doi
-
[17]
Causality
Judea Pearl. Causality. Cambridge University Press, 2009. ISBN : 978-0-521-89560-6
2009
-
[18]
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann,
-
[19]
Graphoids: Graph-Based Logic for Reasoning about Relevance Relations or When would x tell you more about y if you already know z?
Judea Pearl and Azaria Paz. “Graphoids: Graph-Based Logic for Reasoning about Relevance Relations or When would x tell you more about y if you already know z?” In: Advances in Artificial Intelligence II, Seventh European Conference on Artificial Intelligence, ECAI 1986, Bright...
1986
-
[20]
Elements of causal inference: foundations and learning algorithms
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017. ISBN : 978-0-262-03731-0
2017
-
[21]
Robust agents learn causal world models
Jonathan Richens and Tom Everitt. “Robust agents learn causal world models”. In: International Conference on Learning Representations (2024)
2024
-
[22]
Causal Consistency of Structural Equation Models
Paul K. Rubenstein et al. “Causal Consistency of Structural Equation Models”. In: Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, UAI 2017, Sydney, Australia, August 11-15, 2017. Ed. by Gal Elidan, Kristian Kersting, and Alexander Ihler. A...
2017
-
[23]
Toward Causal Representation Learning
Bernhard Schölkopf et al. “Toward Causal Representation Learning”. In: Proc. IEEE 109.5 (2021), pp. 612–634. DOI: 10.1109/JPROC.2021.3058954
2021
-
[24]
Causality in thought
Steven A Sloman and David Lagnado. “Causality in thought”. In: Annual review of psychology 66.1 (2015), pp. 223–247
2015
-
[25]
Causation, prediction, and search
Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, prediction, and search. The MIT Press, 2000. ISBN : 978-0-262-28415-8
2000
-
[26]
Data mining: practical machine learning tools and techniques with Java implemen- tations
Ian H Witten and Eibe Frank. “Data mining: practical machine learning tools and techniques with Java implemen- tations”. In: Acm Sigmod Record 31.1 (2002), pp. 76–77
2002
-
[27]
Abstraction between Structural Causal Models: A Review of Definitions and Properties
Fabio Massimo Zennaro. “Abstraction between Structural Causal Models: A Review of Definitions and Properties”. In: CoRR abs/2207.08603 (2022). DOI: 10.48550/ARXIV.2207.08603. arXiv: 2207.08603. 14 Factored space models arxiv preprint A Proofs for Section 4 A.1 History as the m...
-
[31]
If supp(P ) ⊆ supp(Q), and P (C) > 0, then Q(C) > 0
-
[32]
If P = PJ ⊗ PJ, then, supp(P ) = supp(PJ ) × supp(PJ )
-
[33]
Note that P and Q do not have to factorize over Ω here
If P = PJ ⊗ PJ, Q = QJ ⊗ QJ, supp(PJ ) ⊆ supp(QJ ), and P (C) > 0, then Q(C) > 0. Note that P and Q do not have to factorize over Ω here. 24 Factored space models arxiv preprint Proof of Lemma C.11. 1. Let supp(P ) ⊆ supp(Q). We now prove the contraposition. If Q(C) = 0 , then...
-
[34]
Let P = PJ ⊗ PJ. For all α ∈ ΩJ and β ∈ ΩJ, we have α · β ∈ supp(P ) ⇐ ⇒ 0 < P(α · β) = PJ (α)PJ (β) ⇐ ⇒ PJ (α) > 0 and PJ (β) > 0 ⇐ ⇒ α ∈ supp(PJ ) and β ∈ supp(PJ ) ⇐ ⇒ α · β ∈ supp(PJ ) × supp(PJ )
-
[35]
By 2., we have supp(P ) = supp(PJ ) × supp(PJ ) ⊆ supp(QJ ) × supp(PJ ) = supp(Q)
Let P = PJ ⊗ PJ, and Q = QJ ⊗ PJ, let supp(PJ ) ⊆ supp(QJ ), and let P (C) > 0. By 2., we have supp(P ) = supp(PJ ) × supp(PJ ) ⊆ supp(QJ ) × supp(PJ ) = supp(Q) . (8) Since P (C) > 0, it follows from 1. that Q(C) > 0. This concludes the proof. We now use Lemma C.11 to show th...
-
[36]
For all i ∈ I, set P 0 i := Pi
-
[37]
, q} and j ∈ I, set P k j := P k−1 j if j ̸= jk Qj if j = jk
For all k ∈ {1, . . . , q} and j ∈ I, set P k j := P k−1 j if j ̸= jk Qj if j = jk
-
[38]
Note that by construction, we have that P q = Q, and also P k−1 and P k can only differ in jk
Set P k = N i∈I P k i . Note that by construction, we have that P q = Q, and also P k−1 and P k can only differ in jk. Next we show by induction that for all k ∈ {1, . . . , q}, we have P k(C) > 0. First, we note that since P 0 = P , we have P 0(C) > 0. We now assume that P k−...
-
[39]
P Dα) = PJ (α) · PJ Dα J . 2. δα ⊗ PJ (D) = PJ Dα J . Proof. Part 1 follows from Dα = {α} ×Dα J and from Lemma C.15. For part 2, we note δα ⊗ PJ (D) = X ω∈D (δα ⊗ PJ )(ω) = X ω∈D δα(ωJ )PJ (ωJ ) = X ω∈Dα PJ (ωJ ) = X ωJ ∈Dα J PJ (ωJ ) = PJ Dα J . Ultimately, the goal is to sho...
-
[40]
(δα ⊗ PJ ) ∈ △⊗ C (Ω)
-
[41]
(δα ⊗ PJ )i = Pi for all factors i that are relevant to A given C
-
[42]
For 2., we need to show that (δα ⊗ PJ ) ∈ △⊗(Ω) and (δα ⊗ PJ )(C) > 0
is satisfied since P ∈ △⊗(Ω) by our assumption, and P (C) > 0 because P (B ∩ C) > 0. For 2., we need to show that (δα ⊗ PJ ) ∈ △⊗(Ω) and (δα ⊗ PJ )(C) > 0. First, we note that δα = N i∈J δαi since for any α′ ∈ ΩJ we have Y i∈J δαi (α′ i) = 1 ⇐ ⇒α′ i = αi for all i ∈ J ⇐ ⇒α′ = ...
-
[1988]
13 Factored space models arxiv preprint
ISBN : 978-1-55860-479-7. 13 Factored space models arxiv preprint
-
[2009]
ISBN : 978-0-262-27738-9
-
[2019]
by Amir Globerson and Ricardo Silva
Ed. by Amir Globerson and Ricardo Silva. V ol. 115. Proceedings of Machine Learning Research. AUAI Press, 2019, pp. 606–615
2019
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.