MDL Meets Latent Confounders: LNML-based Causal Discovery

Kenji Yamanishi; Shin Matsushima; Zhongyi Que

REVIEW 4 major objections 5 minor 28 references

MDL Meets Latent Confounders: LNML-based Causal Discovery

T0 review · 4 major / 5 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An MDL score that treats latent common causes as first-class options lets nonlinear causal graphs be recovered without assuming causal sufficiency.

desk verdict Solid mid-subfield MDL method for nonlinear continuous data with latents; the residual proxy is openly heuristic and theory is linear-only, but the algorithm and results are still worth a look. read the letter →

arxiv 2607.04133 v1 pith:7MJ72U4X submitted 2026-07-05 cs.LG

Zhongyi Que , Shin Matsushima , Kenji Yamanishi This is my paper · ORCID

classification cs.LG

keywords causaldiscoverylatentconfoundersminimumdescriptionlengthLNMLGaussianprocessespseudo-collinearitynonlinearmechanisms

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Causal discovery usually either assumes every relevant variable is observed or forces mechanisms to be linear. This paper drops both restrictions by scoring every pair of variables under four competing models—A causes B, B causes A, a shared latent cause, or independence—using the luckiness-normalized maximum-likelihood code length of a Gaussian-process regression. When the two directed scores are almost equal, the gap is called Δ-pseudo-collinearity and is taken as evidence of a latent confounder rather than a direct edge. A greedy algorithm, PCG-CD, starts from a complete bidirected graph, orients or deletes edges according to these scores, and finally breaks cycles by the cheapest local change. On synthetic nonlinear data and a real automobile data set the method recovers directed edges and flags confounded pairs more reliably than the standard partial-ancestral-graph baseline. If the approach holds up, practitioners can search for causal structure in continuous, nonlinear systems even when unobserved common causes are known to exist.

What carries the argument

Δ-pseudo-collinearity: the absolute difference between the two directed residual code lengths; when it stays below a linear threshold the pair is declared bidirected (latent-confounded).

What would settle it

Generate nonlinear data from a known latent confounder whose mapping to the two children is highly non-invertible or high-noise; if the residual Δ-gap still falls below the threshold while a true directed edge produces a large gap, the criterion fails.

Watch

Extended reading notes

Core claim

Minimizing the LNML code length of Gaussian-process structural equations, together with a threshold on the pairwise directional gap Δ ≤ nε (Δ-pseudo-collinearity), identifies both directed causal edges and latent confounders without requiring linear mechanisms or causal sufficiency.

Load-bearing premise

The unobserved common cause can be replaced, for scoring purposes, by the residuals of the two observed variables themselves; the paper calls this substitution a heuristic.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The new piece here is a concrete greedy algorithm (PCG-CD) that scores GP mechanisms with LNML, decides among four pairwise relations via code-length, and flags latent confounding with a named Δ-pseudo-collinearity gap. That combination is not just a rehash of GLOBE, CoCa, or FCI; it is a practical deletion-based O(|V|²) search that starts from a complete bidirected graph and never re-adds edges.

What works: the GP LNML score under sufficiency is standard and cleanly written. The residual reformulation makes the latent-aware objective computable without sampling Z. Theorem 1 gives a usable linear-case bound that motivates the nε threshold. Synthetic pairwise AUCs look strong, multi-node Z-hit/E-hit beat FCI on the authors’ metrics, and the Auto-MPG example is readable. Code is linked. Citations cover the right MDL and latent-FCM lines.

Soft spots are real but not hidden. Section 4.1 itself calls the residual proxy L(A_res ↔ B_res) for the true latent score “heuristic.” Theory stops at linear mechanisms; the nonlinear discussion is informal. ε=5 is fixed from preliminary runs on similar graphs. Multi-node experiments stay at six nodes with one planted confounder and custom metrics that punish sparse FCI. No RCD/NOCADILAC/GES-style baselines, no uncertainty, no ε sensitivity. Those limits keep the abstract claim from being fully secured, but they do not make the method incoherent.

This is for people already working on MDL causal discovery or needing a nonlinear continuous method that can output bidirected edges. A serious referee should see it; the contribution is clear enough and the gaps are fixable. I would engage, cite the algorithm and the pseudo-collinearity idea if I am writing in this area, and bring it to reading group if we are covering latent-aware continuous methods.

Referee Report

4 major / 5 minor

Summary. The paper proposes an MDL framework for causal discovery under nonlinear mechanisms and latent confounders. Causal mechanisms are modeled by Gaussian processes; pairwise relationships among residualized variables are scored by an LNML-style code-length (Eqs. 5–8). Four candidate structures (A o B, A←B, A↔B, independence) are compared, and latent confounding is declared when the directional gap satisfies Δ-pseudo-collinearity (Eq. 10: |Δ|≤nε). A greedy algorithm (PCG-CD) starts from a complete bidirected graph, orients or removes edges by these scores, then breaks cycles. Theorem 1 bounds Δ for linear latent mechanisms; experiments on synthetic pairs/graphs and Auto MPG report improved Z-hit/E-hit versus FCI variants.

Significance. If the residual proxy and thresholding rule are reliable, the work would be a useful addition to MDL-based causal discovery: it is mechanism-agnostic (GP), explicitly handles latent confounders, and reduces edge evaluation complexity relative to GES-style search. Strengths include an explicit LNML formulation under GPs, a named detection criterion (Δ-pseudo-collinearity) with a linear-case bound, a concrete O(|V|²) greedy procedure, and public code. The contribution is incremental rather than foundational, but the combination of nonlinear mechanisms and latent-confounder scoring under MDL is of genuine interest to the causal-discovery community.

major comments (4)

§4.1 explicitly replaces the true latent score L(A_res|Z)+L(B_res|Z) by the residual pairwise proxy L(A_res↔B_res)=L(A_res|B_res)+L(B_res|A_res), calling the substitution “heuristic in nature” and stating it is used only to make the objective computable, not as a principled MDL decision. The abstract and §1.3 claim that PCG-CD “effectively detects latent confounders” under nonlinear mechanisms. Because the proxy is the sole operational definition of the confounder model, the central claim rests on an unvalidated approximation. A controlled check (e.g., synthetic nonlinear Z→A, Z→B with known ground-truth code-lengths, or comparison against a sampled latent model) is needed before the multi-node results can be read as confirmation.
Theorem 1 (§4.3) supplies an O(n C(|coefA−coefB|,σ_Z,…)) bound only for linear mechanisms. The nonlinear discussion is informal (“if approximately linear or range small”). Experiments in §5.2 use cubic polynomial mechanisms, yet no corresponding bound or diagnostic is given for when Δ remains O(n) under those mechanisms. Without this, the fixed threshold rule Δ≤nε (ε=5) lacks theoretical support precisely in the regime claimed by the abstract.
§5 sets ε=5 from “preliminary experiments on synthetic graphs with similar sparsity” and keeps it fixed for all sample sizes and both linear/nonlinear settings. Table 1 shows that AUC of Δ for distinguishing direct edges from confounding varies substantially with n and σ (e.g., linear n=1000, σ=0.2 yields AUC≈0.55). A sensitivity analysis or data-driven selection of ε is required; otherwise the reported Z-hit/E-hit in Table 2 may be tuned to the same synthetic family used to choose the threshold.
Table 2 evaluates only 6-node Erdős–Rényi graphs with a single latent confounder and 20 replicates. FCI is the sole baseline; stronger latent-aware nonlinear methods cited in §1.2 (e.g., NOCADILAC, CoCa) are not compared. Sim1/Sim2 favor sparse graphs, so FCI’s higher similarity scores are dismissed as “false negatives,” but no denser or multi-confounder regime is tested. The multi-node evidence is therefore too narrow to support the general claim of accurate recovery under nonlinear latent confounding.

minor comments (5)

§3.2 and Eq. (8) present an upper-bound Bayesian/GP score as “the resulting computable LNML objective.” Clarify the precise relationship to the LNML definition (Eqs. 1–2) and whether the parametric-complexity term is fully accounted for.
Algorithm 1 Stage 2 modifies edges inside cycles by “minimum loss” but does not specify how loss is computed when converting → to ↔ versus deleting the edge; a short formal definition would help reproducibility.
Figure 2 annotates Δ values on PCG-CD edges; the caption and text should state the sample size used for the Auto MPG run so that the threshold nε can be verified.
Notation for residual components (A_res) appears in §4 without an explicit formula; a one-line definition would improve readability.
Related-work coverage of MDL latent methods (DNML, CDHC) is brief; a sentence contrasting the GP residual proxy with those approaches would situate the novelty more clearly.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-by-construction; only mild hyperparameter selection of ε on similar synthetics and an acknowledged residual heuristic that does not force the graph decisions.

fitted input called prediction [§5 (Experiments, threshold paragraph)]
"The threshold parameter used in the pseudo-collinearity criterion is set to ε=5. This value was selected empirically through preliminary experiments on synthetic graphs with similar sparsity levels and varying sample sizes, and was kept fixed throughout all experiments."

The decision boundary that distinguishes directed edges from latent-confounder (bidirected) edges is tuned on synthetic graphs whose sparsity and size match the evaluation regime of Tables 1–2. Reported recovery metrics are therefore partially conditioned on a hyper-parameter fitted to the same distribution family, rather than being a pure out-of-sample consequence of the LNML scores alone.

full rationale

The paper's load-bearing steps are data-dependent LNML/GP code-length comparisons (Eqs. 5–8, 10) that select among four pairwise models and a greedy deletion procedure (Alg. 1). These comparisons are not algebraically forced to recover the target graph; they depend on the fitted residuals and kernel matrices of the observed sample. Theorem 1 supplies an independent linear-case bound motivating the form of the threshold, while the nonlinear case is left informal. The residual substitution L(A_res ↔ B_res) ≈ L(A|Z)+L(B|Z) is explicitly labeled heuristic and is stated not to be used for the MDL decision itself (only to render the global score computable). The sole mild circularity-adjacent practice is empirical choice of the fixed threshold ε=5 on preliminary synthetics of similar sparsity; this can inflate reported Z-hit/E-hit but does not make any equation reduce to its own input. No uniqueness theorem is imported from the authors, no self-citation is load-bearing for the central claim, and no known empirical pattern is merely renamed. Consequently the derivation chain remains self-contained against external benchmarks and scores only 2.

Assumptions & free parameters 3 free parameters · 6 assumptions · 2 invented entities

The central claim rests on GP SEM modeling, an LNML upper-bound score, a residual reduction of the graph, a heuristic latent code-length proxy, a linear-case gap bound used as motivation for a fixed threshold, and a greedy deletion search. Free parameters that affect decisions are mainly ε and GP kernel settings. Invented entities are the Δ-pseudo-collinearity notion and the PCG-CD procedure itself.

free parameters (3)

pseudo-collinearity threshold ε = 5
Fixed at ε=5 after preliminary synthetic experiments with similar sparsity; all orientation vs bidirected decisions use Δ ≤ nε (§5).
GP RBF length-scale and noise hyperparameters = init 1.0; optimized
Initialized at 1.0 and optimized by scikit-learn during training; they enter every LNML score (§5, §3.1).
T_max iteration cap
Algorithm 1 parameter that can stop skeleton refinement before convergence.

assumptions (6)

domain assumption Observed variables follow additive-noise SEMs with independent noise, possibly with latent parents.
Stated in §2.1 as the causal model class for both sufficiency and latent settings.
domain assumption Causal mechanisms are well-modeled by Gaussian processes with RBF kernels for LNML scoring.
§3.1 adopts GPs so that the Bayesian/penalized score of Kakade et al. can serve as LNML luckiness.
ad hoc to paper After residualizing known parents, pairwise relationships among residuals determine edge type without other observed nodes.
§4.1 residual reduction used to make latent-aware scoring tractable.
ad hoc to paper L(A_res ↔ B_res) approximates the true latent-confounder code-length by substituting Z with residuals of A and B.
Explicitly labeled heuristic in §4.1; not derived from a sampling-free exact LNML.
domain assumption Under linear latent confounding, Δ = O(n C(|coefA−coefB|, σZ, σA, σB)), justifying a linear-in-n threshold.
Theorem 1 (§4.3); nonlinear extension is only sketched.
ad hoc to paper A greedy deletion search from a complete bidirected graph that never reinserts edges yields a useful MDL minimizer.
Algorithm 1 design choice; no global optimality guarantee.

invented entities (2)

Δ-pseudo-collinearity
purpose: Operational criterion to keep a bidirected edge when directed code-lengths are nearly equal, interpreted as latent confounding.
Defined in §4.2–4.3; independent evidence is limited to synthetic AUC tables and one real-data qualitative check.
PCG-CD algorithm
purpose: Greedy two-stage procedure that builds a DAG with bidirected latent edges using LNML scores and Δ-thresholding.
Introduced in §4.4; evaluated only by the paper's own experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MDL Meets Latent Confounders: LNML-based Causal Discovery." pith.science (2026). https://pith.science/paper/7MJ72U4X

@misc{pith2026260704133,
  author       = {Pith},
  title        = {Pith review of: MDL Meets Latent Confounders: LNML-based Causal Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7MJ72U4X}},
  note         = {Machine review of arXiv:2607.04133}
}

abstract

Causal discovery with nonlinear mechanisms and latent confounders remains challenging. Existing methods often rely on either linear assumptions or causal sufficiency, limiting their applicability. We propose an MDL-based causal discovery framework that explicitly accounts for latent confounders while allowing flexible nonlinear mechanisms by minimizing the luckiness normalized maximum likelihood (LNML) code-length. The causal relationship between each variable pair is determined by selecting the shortest code-length of the causal model, and we introduce the notion of $\Delta$-pseudo-collinearity to identify dependencies induced by latent confounders. Based on these ideas, we develop a greedy algorithm, termed Pseudo-Collinearity Guided Causal Discovery (PCG-CD). Experiments on synthetic and real-world datasets demonstrate that the proposed method accurately recovers directed causal relationships and effectively detects latent confounders.

Figures

Figures reproduced from arXiv: 2607.04133 by the authors.

**Figure 2.** Causal discovery results on the Auto MPG dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 1 linked inside Pith

[1]

Knowledge and Information Systems56(2), 285–307 (2018) 3 https://archive.ics.uci.edu/dataset/9/auto+mpg MDL Meets Latent Confounders: LNML-based Causal Discovery 17

Budhathoki, K., Vreeken, J.: Origo: causal inference by compression. Knowledge and Information Systems56(2), 285–307 (2018) 3 https://archive.ics.uci.edu/dataset/9/auto+mpg MDL Meets Latent Confounders: LNML-based Causal Discovery 17

2018
[2]

Journal of machine learning research3(Nov), 507–554 (2002)

Chickering, D.M.: Optimal structure identification with greedy search. Journal of machine learning research3(Nov), 507–554 (2002)

2002
[3]

Erdös, P., Rènyi, A.: On random graphs I. Publ. math. debrecen6(290-297), 18 (1959)

1959
[4]

Grünwald,P.D.:Theminimumdescriptionlengthprinciple.MITpress,Cambridge, MA (2007)

2007
[5]

Advances in neural information processing systems21(2008)

Hoyer, P., Janzing, D., Mooij, J.M., Peters, J., Schölkopf, B.: Nonlinear causal discovery with additive noise models. Advances in neural information processing systems21(2008)

2008
[6]

Advances in neural information processing systems17(2004)

Kakade, S.M., Ng, A.: Online bounds for Bayesian algorithms. Advances in neural information processing systems17(2004)

2004
[7]

Advances in Neural Information Processing Systems18(2005)

Kakade, S.M., Seeger, M.W., Foster, D.P.: Worst-case bounds for Gaussian process models. Advances in Neural Information Processing Systems18(2005)

2005
[8]

In: Proceedings of the 2019 SIAM International Conference on Data Mining

Kaltenpoth, D., Vreeken, J.: We are not your real parents: Telling causal from confounded using mdl. In: Proceedings of the 2019 SIAM International Conference on Data Mining. pp. 199–207. SIAM (2019)

2019

Show all 28 references

[9]

In: Uncertainty in Artificial Intelligence

Kaltenpoth, D., Vreeken, J.: Causal discovery with hidden confounders using the algorithmic Markov condition. In: Uncertainty in Artificial Intelligence. pp. 1016–
[10]

In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J

Kaltenpoth, D., Vreeken, J.: Nonlinear causal discovery with latent confounders. In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J. (eds.) Proceedings of the 40th International Conference on Machine Learning. Proceed- ings of Machine Learning Rese...

2023
[11]

In: 2022 IEEE International Conference on Big Data

Kobayashi, M., Miyaguchi, K., Matsushima, S.: Detection of unob- served common cause in discrete data based on the MDL principle. In: 2022 IEEE International Conference on Big Data. pp. 45–54 (2022). https://doi.org/10.1109/BigData55660.2022.10020351

2022 doi
[12]

In: International Conference on artificial intelligence and statistics

Maeda, T.N., Shimizu, S.: Rcd: Repetitive causal discovery of linear non-Gaussian acyclic models with latent confounders. In: International Conference on artificial intelligence and statistics. pp. 735–745. PMLR (2020)

2020
[13]

In: 2017 IEEE international conference on data mining

Marx, A., Vreeken, J.: Telling cause from effect using MDL-based local and global regression. In: 2017 IEEE international conference on data mining. pp. 307–316. IEEE (2017)

2017
[14]

In: Proceedings of the AAAI Conference on Artificial Intelligence

Mian, O.A., Marx, A., Vreeken, J.: Discovering fully oriented causal networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 8975– 8982 (2021)

2021
[15]

Miyaguchi, K.: Normalized maximum likelihood with luckiness for multivariate normal distributions (2017),https://arxiv.org/abs/1708.01861

2017 arXiv
[16]

Cambridge University Press, Cambridge, UK, 1st edn

Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge, UK, 1st edn. (2000)

2000
[17]

Neural computation27(3), 771–799 (2015)

Peters, J., Bühlmann, P.: Structural intervention distance for evaluating causal graphs. Neural computation27(3), 771–799 (2015)

2015
[18]

International journal of data science and analytics3, 121–129 (2017)

Ramsey, J., Glymour, M., Sanchez-Romero, R., Glymour, C.: A million vari- ables and more: the fast greedy equivalence search algorithm for learning high- dimensional graphical causal models, with an application to functional magnetic resonance images. International journal of ...

2017
[19]

MIT Press, Cambridge, MA (2006),http://www.gaussianprocess.org/gpml/ 18 Z

Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA (2006),http://www.gaussianprocess.org/gpml/ 18 Z. Que et al

2006
[20]

Automatica14(5), 465–471 (1978)

Rissanen, J.: Modeling by shortest data description. Automatica14(5), 465–471 (1978)

1978
[21]

Journal of Machine Learning Research 7(10) (2006)

Shimizu, S., Hoyer, P.O., Hyvärinen, A., Kerminen, A., Jordan, M.: A linear non- Gaussian acyclic model for causal discovery. Journal of Machine Learning Research 7(10) (2006)

2006
[22]

Journal of Machine Learning Research12(Apr), 1225–1248 (2011)

Shimizu, S., Inazumi, T., Sogawa, Y., Hyvarinen, A., Kawahara, Y., Washio, T., Hoyer, P.O., Bollen, K., Hoyer, P.: DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model. Journal of Machine Learning Research12(Apr), 1225–1248 (2011)

2011
[23]

Problemy Peredachi Informatsii23(3), 3–17 (1987)

Shtar’kov, Y.M.: Universal sequential coding of single messages. Problemy Peredachi Informatsii23(3), 3–17 (1987)

1987
[24]

MIT Press, Cambridge, MA, 2nd edn

Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. MIT Press, Cambridge, MA, 2nd edn. (2000)

2000
[25]

Neural computation26(1), 57–83 (2014)

Tashiro, T., Shimizu, S., Hyvärinen, A., Washio, T.: ParceLiNGAM: A causal ordering method robust against latent confounders. Neural computation26(1), 57–83 (2014)

2014
[26]

Springer Nature, Berlin, Heidelberg (2023)

Yamanishi, K.: Learning with the Minimum Description Length Principle. Springer Nature, Berlin, Heidelberg (2023)

2023
[27]

Data Mining and Knowledge Discovery33(4), 1017–1058 (2019)

Yamanishi, K., Wu, T., Sugawara, S., Okada, M.: The decomposed normalized maximum likelihood code-length criterion for selecting hierarchical latent variable models. Data Mining and Knowledge Discovery33(4), 1017–1058 (2019)

2019
[28]

In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intel- ligence

Zhang, K., Hyvärinen, A.: On the identifiability of the post-nonlinear causal model. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intel- ligence. p. 647–655. UAI ’09 (2009)

2009

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.

[1] [1]

Knowledge and Information Systems56(2), 285–307 (2018) 3 https://archive.ics.uci.edu/dataset/9/auto+mpg MDL Meets Latent Confounders: LNML-based Causal Discovery 17

Budhathoki, K., Vreeken, J.: Origo: causal inference by compression. Knowledge and Information Systems56(2), 285–307 (2018) 3 https://archive.ics.uci.edu/dataset/9/auto+mpg MDL Meets Latent Confounders: LNML-based Causal Discovery 17

2018

[2] [2]

Journal of machine learning research3(Nov), 507–554 (2002)

Chickering, D.M.: Optimal structure identification with greedy search. Journal of machine learning research3(Nov), 507–554 (2002)

2002

[3] [3]

Erdös, P., Rènyi, A.: On random graphs I. Publ. math. debrecen6(290-297), 18 (1959)

1959

[4] [4]

Grünwald,P.D.:Theminimumdescriptionlengthprinciple.MITpress,Cambridge, MA (2007)

2007

[5] [5]

Advances in neural information processing systems21(2008)

Hoyer, P., Janzing, D., Mooij, J.M., Peters, J., Schölkopf, B.: Nonlinear causal discovery with additive noise models. Advances in neural information processing systems21(2008)

2008

[6] [6]

Advances in neural information processing systems17(2004)

Kakade, S.M., Ng, A.: Online bounds for Bayesian algorithms. Advances in neural information processing systems17(2004)

2004

[7] [7]

Advances in Neural Information Processing Systems18(2005)

Kakade, S.M., Seeger, M.W., Foster, D.P.: Worst-case bounds for Gaussian process models. Advances in Neural Information Processing Systems18(2005)

2005

[8] [8]

In: Proceedings of the 2019 SIAM International Conference on Data Mining

Kaltenpoth, D., Vreeken, J.: We are not your real parents: Telling causal from confounded using mdl. In: Proceedings of the 2019 SIAM International Conference on Data Mining. pp. 199–207. SIAM (2019)

2019