Pith. sign in

REVIEW 3 major objections 4 minor 99 references

When Shift Happens - Confounding Is to Blame

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Under hidden confounding shift, out-of-distribution generalization provably requires environment-specific relationships rather than invariant ones alone.

desk verdict The paper has a clean information decomposition and a lot of empirical work, but the central theorem rests on a contradictory pair of inequalities, so the proof claim in the abstract fails. read the letter →

arxiv 2505.21422 v1 pith:U4JDLAED submitted 2025-05-27 cs.LG

Abbavaram Gowtham Reddy , Celia Rubio-Madrigal , Rebekka Burkholz , Krikamol Muandet This is my paper
classification cs.LG
keywords out-of-distributiongeneralizationhiddenconfoundingpredictiveinformationcausalinvariancedistributionshiftcovariateselectionempiricalriskminimizationproxyvariables
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Conventional out-of-distribution (OOD) generalization wisdom says to learn causal invariant features, yet standard empirical risk minimization (ERM) often matches or beats specialized invariance methods, and using all covariates instead of only causal ones helps. This paper argues that hidden confounding is the reason. When an unobserved variable drives both the input and the outcome, and its distribution shifts across environments, the usual invariance assumptions break; the paper proves that the predictive information between predictions and labels then equals conditional informativeness minus residual information, so effective generalization requires modeling environment-specific relationships. It also shows that informative non-causal covariates act as proxies for the hidden confounder and improve OOD accuracy. If correct, this reframes covariate selection and justifies methods that use environment information rather than only hunting invariant features.

What carries the argument

The carrying object is the predictive-information decomposition: an identity that rewrites $I(Y;\hat{Y})$, how much the prediction tells about the label, as a signed sum of conditional informativeness, variation, label shift, feature shift, concept shift, and residual. Under hidden confounding with $X \to Y$ or $Y \to X$, the four shift terms cancel, leaving the reduced identity $I(Y;\hat{Y}) = I(\phi(X);Y|E) - I(\phi(X);Y|\hat{Y})$, where $I(\phi(X);Y|E)$ is the information the features provide about the label within each environment and $I(\phi(X);Y|\hat{Y})$ is the information the features carry about the label beyond what the prediction already captures. This identity does the work: it converts the question of what to learn into a two-term objective and shows why environment-specific information and proxy covariates matter.

What would settle it

Train a binary classifier under a hidden confounding shift with $U \to X$ and $U \to Y$ and $X \to Y$, where the model deliberately outputs the label opposite to the true one; it achieves maximum $I(Y;\hat{Y}) = 1$ bit yet 0 percent accuracy. If the paper's own conditional-informativeness-minus-residual objective ranks this predictor as optimal, that would falsify the claim that maximizing predictive information is the right OOD generalization objective.

Watch

Extended reading notes

Core claim

The central claim is that hidden confounding shift flips the goal of OOD generalization. For causal graphs where an unobserved confounder $U$ causes both $X$ and $Y$ and either $X$ causes $Y$ or $Y$ causes $X$, the paper derives the identity $I(Y;\hat{Y}) = I(\phi(X);Y|E) - I(\phi(X);Y|\hat{Y})$: predictive information is exactly conditional informativeness minus residual. The variation, label-shift, feature-shift, and concept-shift terms from the general decomposition cancel. Consequently maximizing predictive information requires maximizing the information the representation gives about the label within each environment while minimizing what remains unexplained by the prediction, which means invariant representations alone are insufficient and environment-specific input-output mappings are needed. The paper further proves that adding informative covariates that are not causally related to $Y$ increases conditional informativeness and reduces concept shift, explaining why all-covariate models can beat causal-subset models.

Load-bearing premise

The whole argument treats how much the prediction tells you about the true label as the same thing as generalizing well; the paper itself notes that a predictor which always chooses the wrong label can get maximum mutual information but zero accuracy, so if that identification fails, the conclusion does not follow.

Editorial extensions

If this is right

  • If correct, the superior OOD performance of ERM observed in benchmarks is not an accident: under hidden confounding shift, pooling all data and fitting environment-specific structure is the mathematically justified strategy, which may explain why ERM often beats invariance-constrained methods.
  • Methods that enforce invariant representations will underperform when hidden confounding shift is present, because they suppress the very environment-specific information the identity says is needed.
  • Adding non-causal covariates informative about the hidden confounder should improve OOD performance, giving a principled covariate-selection rule: prefer covariates that raise conditional informativeness and reduce concept shift, even if they are not causes of the outcome.
  • Mixture-of-experts and other environment-conditional models receive theoretical backing, and the paper extends this rationale to settings where confounding support does not overlap and the confounder is continuous-valued.
  • Practitioners should treat environment labels or environment statistics as a resource for OOD generalization rather than as noise to be removed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the decomposition suggests a practical diagnostic: compute conditional informativeness minus residual on validation environments to predict which model will generalize and to decide whether a candidate covariate is worth adding before running a full training experiment.
  • Beyond the paper, when an adversary can choose test environments with extreme confounder values, maximizing conditional informativeness might encourage fitting environment artifacts, so the strategy may need to be paired with regularization on the residual or on feature shift.
  • Beyond the paper, the identity suggests that the distinction between causation and prediction blurs under hidden confounding: variables that are merely informative about the hidden confounder are as useful as causal parents, which could reshape feature-engineering practice in tabular machine learning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper studies out-of-distribution generalization under hidden confounding shifts. It proposes an information-theoretic decomposition of predictive information I(Y;Ŷ) into conditional informativeness, variation, label shift, feature shift, concept shift, and residual terms (Proposition 4.1), and claims that under hidden confounding with either X→Y or Y→X this reduces to I(φ(X);Y|E) - I(φ(X);Y|Ŷ) (Proposition 4.2). From this, the authors conclude that effective OOD generalization requires environment-specific relationships rather than invariant ones, and that adding informative non-causal covariates as proxies for hidden confounders improves generalization (Proposition 4.3). The paper supports these claims with real-world tabular benchmarks (TableShift) and synthetic experiments showing correlations between the information-theoretic components and ID/OOD accuracy.

Significance. If the theoretical results were sound, the paper would make a valuable conceptual contribution to the OOD generalization debate by explaining the empirical success of ERM and all-covariate models in the presence of hidden confounding. It would also provide practical guidance for covariate selection and for methods that learn environment-specific experts. The authors should be credited for a clean derivation of Proposition 4.1, for a transparent empirical protocol that reports mutual information estimates alongside accuracies, and for including an explicit limitation paragraph in Appendix C. However, the central proof of Proposition 4.2 is internally inconsistent; the main theoretical claim is therefore not established, and the interpretation of the information-theoretic identities as statements about OOD risk is not justified. The current contribution, while empirically suggestive, does not meet the bar for a theoretical claim of proof.

major comments (3)
  1. [Appendix A, Proposition 4.2, Case 1 (inequalities (8) and (9))] The proof of Proposition 4.2 is invalid: inequalities (8) and (9) cannot both hold in general. For arbitrary variables φ, E, Y, the identity I(φ;E|Y)-I(φ;E)=I(Y;E|φ)-I(Y;E) holds, since both sides equal I(φ;Y|E)-I(φ;Y). Hence (8), I(φ;E|Y)≥I(φ;E), is equivalent to I(Y;E|φ)≥I(Y;E), which is the exact reverse of (9), I(Y;E)≥I(Y;E|φ). Both can hold only when the differences are zero, which is not an assumed condition. The same contradiction appears in Case 2 between inequalities (10) and (11). A concrete linear-Gaussian example with E∼N(0,1), U=E+N(0,1), X=U+N(0,1), and Y=X-2U+N(0,1) satisfies the graph U→X, U→Y, X→Y but gives I(X;E|Y)≈0.16<I(X;E)≈0.20 when φ is the identity map, directly falsifying inequality (8). Consequently, the sandwich argument in the proof collapses and Proposition 4.2 is not established; the abstract's claim that generalization is proven to require environment-specific relationships is unsupported.
  2. [Section 4 and Appendix C (limitations of predictive information decomposition)] The central interpretive step from Equation (4) to OOD generalization is not justified. The theorem, even if correct, concerns the mutual information I(Y;Ŷ), not the OOD risk Re(f∘ϕ). The manuscript itself concedes in Appendix C that a binary classifier can achieve maximal I(Y;Ŷ) by perfectly anti-correlating with Y while having 0% accuracy. No bound or monotonicity relation between I(Y;Ŷ) and expected accuracy or loss is provided. Therefore maximizing I(φ(X);Y|E)-I(φ(X);Y|Ŷ) does not entail better OOD generalization, and the conclusion that effective generalization requires environment-specific relationships does not follow from the information-theoretic identities.
  3. [Appendix A, Proposition 4.3 (iii)] Proposition 4.3(iii) is not rigorously established. The proof relies on the statement that 'conditioning usually reduces mutual information unless the additional conditioning variable opens any collider paths,' but mutual information can increase under conditioning in non-collider settings depending on the joint distribution; no formal condition (e.g., faithfulness or a specific conditional independence) is given. Since the covariate-selection guidance in Section 4.3 depends on part (iii), this inequality needs either a proper proof under explicit assumptions or should be relabeled as an empirical observation.
minor comments (4)
  1. [Section 5, Figure 4] The text says 'the first six datasets... listed in Table 2,' but Table 2 lists eight datasets; please clarify which datasets are used and why.
  2. [Section 5] The sentence 'The relative ID/OOD performance of the methods shown in Figure 1' should refer to Figure 4, not Figure 1.
  3. [Section 4.2, Equation (4)] The sentence 'the predictive information can be maximized by maximizing the difference: conditional informativeness - residual' is tautological given the equality; the substantive claim about environment-specific relationships requires a formal argument that invariant representations cannot achieve this difference.
  4. [Definition 4.3] The sentence 'any covariate that is informative to U, is also informative to Y and vice-versa' is not generally implied by U→Y and needs assumptions or should be softened.
Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The theoretical results assume a specific causal graph (U affects both X and Y, with X and Y causally oriented), and the proofs of Proposition 4.2 rely on mutual information inequalities that are argued heuristically rather than formally established. No code or data artifacts are shipped. No new entities such as particles or forces are introduced.

assumptions (3)
  • domain assumption Causal graph with hidden confounder U causing both X and Y, environment variable E shifting P(U), and either X -> Y or Y -> X.
    This graph is assumed throughout Section 4, especially in Proposition 4.2 and the inequalities in (2) and (3). It is not learned from data; it is a modeling choice.
  • ad hoc to paper Mutual information inequalities in (2) and (3): conditioning on Y or phi(X) changes MI in a specific direction (e.g., I(phi; E | Y) >= I(phi; E) when X -> Y).
    These inequalities are asserted via collider reasoning but are not generally true for all distributions; no formal proof is provided. Proposition 4.2's equality relies on them.
  • standard math Standard chain rules of mutual information and d-separation facts.
    Used in the proofs of Propositions 4.1, 4.2, and 4.3. These are accepted background results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Shift Happens - Confounding Is to Blame." pith.science (2026). https://pith.science/paper/U4JDLAED

@misc{pith2026250521422,
  author       = {Pith},
  title        = {Pith review of: When Shift Happens - Confounding Is to Blame},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U4JDLAED}},
  note         = {Machine review of arXiv:2505.21422}
}
read the original abstract

Distribution shifts introduce uncertainty that undermines the robustness and generalization capabilities of machine learning models. While conventional wisdom suggests that learning causal-invariant representations enhances robustness to such shifts, recent empirical studies present a counterintuitive finding: (i) empirical risk minimization (ERM) can rival or even outperform state-of-the-art out-of-distribution (OOD) generalization methods, and (ii) its OOD generalization performance improves when all available covariates, not just causal ones, are utilized. Drawing on both empirical and theoretical evidence, we attribute this phenomenon to hidden confounding. Shifts in hidden confounding induce changes in data distributions that violate assumptions commonly made by existing OOD generalization approaches. Under such conditions, we prove that effective generalization requires learning environment-specific relationships, rather than relying solely on invariant ones. Furthermore, we show that models augmented with proxies for hidden confounders can mitigate the challenges posed by hidden confounding shifts. These findings offer new theoretical insights and practical guidance for designing robust OOD generalization algorithms and principled covariate selection strategies.

Figures

Figures reproduced from arXiv: 2505.21422 by the authors.

Figure 1
Figure 1. Causal graphs underlying distribution shifts. Several existing methods for OOD generaliza￾tion assume certain causal relationships among U, X, Y that guarantee specific invariances. For instance, when U → Y → X ( [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. We evaluate four linear regression (L.R.) models in an OOD setting characterized by hidden confounding shifts and minimal environment overlap (i.e., distant µe). (i) A model trained solely on X learns an incorrect relationship with Y , illustrating Simpson’s paradox. (ii) Using environment-specific summary statistics of X, denoted as E, recovers the correct relationship but remains limited in representation power. (… view at source ↗
Figure 3
Figure 3. Bi-directed arrow between X and Y indicate that some covariates of X can cause Y , and some may be caused by Y . Now consider the predictive information I(Y ; Yˆ ), where the predictions Yˆ = (f ◦ ϕ)(X) are based on a learned represen￾tations ϕ(X). We model the underlying causal relationships among X, Y, U, ϕ(X), Yˆ and E as shown in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The difference conditional informativeness − residual in the plots is positively correlated with the average ID and OOD test accuracy over the eight datasets shown in the table on the right. 5 Experimental results We conduct experiments on both real-world and synthetic…
Figure 5
Figure 5. Figure 5: Adding more proxy variables XI of U that are informative to Y helps in reducing MSE, increasing conditional informativeness and feature shift while reducing concept shift. Informative covariates vs. accuracy: We now study how inclusion of informative covariates helps i…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

99 extracted references · 54 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    A reductions approach to fair classification

    Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International conference on machine learning, pages 60–69. PMLR, 2018

    2018

  3. [3]

    Domain-adversarial neural networks

    Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, and Mario Marchand. Domain-adversarial neural networks. arXiv preprint arXiv:1412.4446, 2014

    arXiv 2014

  4. [4]

    Kusner, Stephen R

    Ibrahim Alabdulmohsin, Nicole Chiou, Alexander D’Amour, Arthur Gretton, Sanmi Koyejo, Matt J. Kusner, Stephen R. Pfohl, Olawale Salaudeen, Jessica Schrouff, and Katherine Tsai. Adapting to latent subgroup shifts via concepts and proxies. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceeding...

    2023

  5. [5]

    Maximum likelihood with bias- corrected calibration is hard-to-beat at label shift adaptation

    Amr Alexandari, Anshul Kundaje, and Avanti Shrikumar. Maximum likelihood with bias- corrected calibration is hard-to-beat at label shift adaptation. In International Conference on Machine Learning, pages 222–232. PMLR, 2020

    2020

  6. [6]

    Invariant risk mini- mization

    Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXiv:1907.02893, 2019

    arXiv 1907

  7. [7]

    Robust supervised learning

    J Andrew Bagnell. Robust supervised learning. In AAAI, pages 714–719, 2005

    2005

  8. [8]

    From detection of individual metastases to classification of lymph node status at the patient level: The CAMELYON17 challenge

    Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, Quanzheng Li, Farhad Ghazvinian Zanjani, Svitlana Zinger, Keisuke Fukuta, Daisuke Komura, Vlado Ovtcharov, Shenghua Cheng, Shaoqun Zeng, Jeppe Thagaard, Anders B Dahl, Huangjing Lin, Hao ...

    2019

Show all 99 references
  1. [9]

    Robust optimization–methodology and applications

    Aharon Ben-Tal and Arkadi Nemirovski. Robust optimization–methodology and applications. Mathematical programming, 92:453–480, 2002

    2002

  2. [10]

    Rethinking early stopping: Refine, then calibrate

    Eugene Berta, David Holzmüller, Michael I Jordan, and Francis Bach. Rethinking early stopping: Refine, then calibrate. arXiv preprint arXiv:2501.19195, 2025

    2025 arXiv

  3. [11]

    Discriminative learning under covariate shift

    Steffen Bickel, Michael Brückner, and Tobias Scheffer. Discriminative learning under covariate shift. Journal of Machine Learning Research, 10(9), 2009

    2009

  4. [12]

    Reliability, sufficiency, and the decomposition of proper scores

    Jochen Bröcker. Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 135:1512–1519, 2009

    2009

  5. [13]

    Experimental and quasi-experimental designs for research

    Donald T Campbell and Julian C Stanley. Experimental and quasi-experimental designs for research. Ravenio books, 2015. 10

    2015

  6. [14]

    Mea- surement error in nonlinear models: a modern perspective

    Raymond J Carroll, David Ruppert, Leonard A Stefanski, and Ciprian M Crainiceanu. Mea- surement error in nonlinear models: a modern perspective. Chapman and Hall/CRC, 2006

    2006

  7. [15]

    Xgboost: A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining , pages 785–794, 2016

    2016

  8. [16]

    Robust covariate shift regression

    Xiangli Chen, Mathew Monfort, Anqi Liu, and Brian D Ziebart. Robust covariate shift regression. In Artificial Intelligence and Statistics, pages 1270–1279. PMLR, 2016

    2016

  9. [17]

    Invariant causal mechanisms through distribution matching

    Mathieu Chevalley, Charlotte Bunne, Andreas Krause, and Stefan Bauer. Invariant causal mechanisms through distribution matching. arXiv preprint arXiv:2206.11646, 2022

    2022 arXiv

  10. [18]

    Semi-supervised learning of class balance under class-prior change by distribution matching

    Marthinus Christoffel Du Plessis and Masashi Sugiyama. Semi-supervised learning of class balance under class-prior change by distribution matching. Neural Networks, 50:110–119, 2014

    2014

  11. [19]

    Statistics of robust optimization: A generalized empirical likelihood approach

    John C Duchi, Peter W Glynn, and Hongseok Namkoong. Statistics of robust optimization: A generalized empirical likelihood approach. Mathematics of Operations Research, 46(3): 946–969, 2021

    2021

  12. [20]

    The algorithmic foundations of differential privacy

    Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9:211–407, 2014

    2014

  13. [21]

    Spuriosity didn’t kill the classifier: Using invariant predictions to harness spurious features

    Cian Eastwood, Shashank Singh, Andrei L Nicolicioiu, Marin Vlastelica Pogan ˇci´c, Julius von Kügelgen, and Bernhard Schölkopf. Spuriosity didn’t kill the classifier: Using invariant predictions to harness spurious features. Advances in Neural Information Processing Systems, 2023

    2023

  14. [22]

    An information-theoretic approach to distribution shifts

    Marco Federici, Ryota Tomioka, and Patrick Forré. An information-theoretic approach to distribution shifts. In A. Beygelzimer, Y . Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021

    2021

  15. [23]

    Domain-adversarial training of neural networks

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of machine learning research, 17:1–35, 2016

    2016

  16. [24]

    Benchmarking distribution shift in tabular data with tableshift

    Joshua P Gardner, Zoran Popovi, and Ludwig Schmidt. Benchmarking distribution shift in tabular data with tableshift. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023

    2023

  17. [25]

    A unified view of label shift estimation

    Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan, and Zachary Lipton. A unified view of label shift estimation. Advances in Neural Information Processing Systems, 33:3290–3300, 2020

    2020

  18. [26]

    Maximum likelihood estimation is all you need for well-specified covariate shift

    Jiawei Ge, Shange Tang, Jianqing Fan, Cong Ma, and Chi Jin. Maximum likelihood estimation is all you need for well-specified covariate shift. In The Twelfth International Conference on Learning Representations, 2024

    2024

  19. [27]

    Revisiting deep learning models for tabular data

    Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. Advances in neural information processing systems , 34: 18932–18943, 2021

    2021

  20. [28]

    Covariate shift by kernel mean matching

    Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, Bernhard Schölkopf, et al. Covariate shift by kernel mean matching. Dataset shift in machine learning, 3(4):5, 2009

    2009

  21. [29]

    Sebastian Gregor Gruber and Francis R. Bach. Optimizing estimators of squared calibration errors in classification. Transactions on Machine Learning Research, 2025

    2025

  22. [30]

    In search of lost domain generalization

    Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In Interna- tional Conference on Learning Representations, 2021

    2021

  23. [31]

    Invariant causal prediction for nonlinear models

    Christina Heinze-Deml, Jonas Peters, and Nicolai Meinshausen. Invariant causal prediction for nonlinear models. Journal of Causal Inference, 6:20170016, 2018. 11

    2018

  24. [32]

    A baseline for detecting misclassified and out-of- distribution examples in neural networks

    Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of- distribution examples in neural networks. In International Conference on Learning Rep- resentations, 2017

    2017

  25. [33]

    Support and invertibility in domain- invariant representations

    Fredrik D Johansson, David Sontag, and Rajesh Ranganath. Support and invertibility in domain- invariant representations. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 527–536, 2019

    2019

  26. [34]

    Lightgbm: A highly efficient gradient boosting decision tree

    Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017

    2017

  27. [35]

    Robust and optimal control, volume 2

    IS Khalil, JC Doyle, and K Glover. Robust and optimal control, volume 2. Prentice hall New York, 1996

    1996

  28. [36]

    Federated optimization: Distributed optimization beyond the datacenter

    Jakub Koneˇcn`y, Brendan McMahan, and Daniel Ramage. Federated optimization: Distributed optimization beyond the datacenter. arXiv preprint arXiv:1511.03575, 2015

    2015 arXiv

  29. [37]

    Estimating mutual information

    Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical Review E, 2004

    2004

  30. [38]

    Out-of-distribution generalization via risk extrapolation (rex)

    David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapolation (rex). In International conference on machine learning, 2021

    2021

  31. [39]

    Measurement bias and effect restoration in causal inference

    Manabu Kuroki and Judea Pearl. Measurement bias and effect restoration in causal inference. Biometrika, 101, 2014

    2014

  32. [40]

    Robust text classification under confounding shift

    Virgile Landeiro and Aron Culotta. Robust text classification under confounding shift. Journal of Artificial Intelligence Research, 63:391–419, 2018

    2018

  33. [41]

    A simple unified framework for detecting out-of-distribution samples and adversarial attacks

    Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018

    2018

  34. [42]

    Large-scale methods for distributionally robust optimization

    Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distributionally robust optimization. Advances in Neural Information Processing Systems, 33: 8847–8860, 2020

    2020

  35. [43]

    Domain generalization with adversarial feature learning

    Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5400–5409, 2018

    2018

  36. [44]

    Federated domain generalization: A survey

    Ying Li, Xingwei Wang, Rongfei Zeng, Praveen Kumar Donta, Ilir Murturi, Min Huang, and Schahram Dustdar. Federated domain generalization: A survey. arXiv preprint arXiv:2306.01334, 2023

    2023 arXiv

  37. [45]

    Foogd: Federated collaboration for both out-of- distribution generalization and detection

    Xinting Liao, Weiming Liu, Pengyang Zhou, Fengyuan Yu, Jiahe Xu, Jun Wang, Wenjie Wang, Chaochao Chen, and Xiaolin Zheng. Foogd: Federated collaboration for both out-of- distribution generalization and detection. Advances in Neural Information Processing Systems, 37:132908–132...

    2024

  38. [46]

    Zin: When and how to learn invariance without environment partition? Advances in Neural Information Processing Systems , 35: 24529–24542, 2022

    Yong Lin, Shengyu Zhu, Lu Tan, and Peng Cui. Zin: When and how to learn invariance without environment partition? Advances in Neural Information Processing Systems , 35: 24529–24542, 2022

    2022

  39. [47]

    Detecting and correcting for label shift with black box predictors

    Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In International conference on machine learning , pages 3122–3130. PMLR, 2018

    2018

  40. [48]

    Statistical analysis with missing data

    Roderick JA Little and Donald B Rubin. Statistical analysis with missing data. John Wiley & Sons, 2019

    2019

  41. [49]

    Towards out-of-distribution generalization: A survey

    Jiashuo Liu, Zheyan Shen, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624, 2021. 12

    2021 arXiv

  42. [50]

    On the need for a language describing distribution shifts: Illustrations on tabular datasets

    Jiashuo Liu, Tianyu Wang, Peng Cui, and Hongseok Namkoong. On the need for a language describing distribution shifts: Illustrations on tabular datasets. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023

    2023

  43. [51]

    Conditional adversarial domain adaptation

    Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31, 2018

    2018

  44. [52]

    Causal effect inference with deep latent-variable models

    Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. Advances in neural information processing systems, 30, 2017

    2017

  45. [53]

    Causality inspired representation learning for domain generalization

    Fangrui Lv, Jian Liang, Shuang Li, Bin Zang, Chi Harold Liu, Ziteng Wang, and Di Liu. Causality inspired representation learning for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8046– 8056, June 2022

    2022

  46. [54]

    Communication-efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017

    2017

  47. [55]

    Identifying causal effects with proxy variables of an unmeasured confounder

    Wang Miao, Zhi Geng, and Eric J Tchetgen Tchetgen. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, 105:987–993, 2018

    2018

  48. [56]

    Domain generalization via invariant feature representation

    Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 10–18, 2013

    2013

  49. [57]

    Do causal predictors generalize better to new domains? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    Vivian Yvonne Nastl and Moritz Hardt. Do causal predictors generalize better to new domains? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  50. [58]

    Causality

    Judea Pearl. Causality. Cambridge university press, 2009

    2009

  51. [59]

    Causal inference by using invariant prediction: identification and confidence intervals

    Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology, 78:947–1012, 2016

    2016

  52. [60]

    Space-for-time substitution as an alternative to long-term studies

    Steward TA Pickett. Space-for-time substitution as an alternative to long-term studies. In Long-term studies in ecology: approaches and alternatives, pages 110–135. Springer, 1989

    1989

  53. [61]

    Neural oblivious decision ensembles for deep learning on tabular data

    Sergei Popov, Stanislav Morozov, and Artem Babenko. Neural oblivious decision ensembles for deep learning on tabular data. arXiv preprint arXiv:1909.06312, 2019

    1909 arXiv

  54. [62]

    Scalable out-of-distribution robustness in the presence of unobserved confounders

    Parjanya Prajakta Prashant, Seyedeh Baharan Khatami, Bruno Ribeiro, and Babak Salimi. Scalable out-of-distribution robustness in the presence of unobserved confounders. In The 28th International Conference on Artificial Intelligence and Statistics, 2025

    2025

  55. [63]

    Learning counterfactually invariant predictors

    Francesco Quinzan, Cecilia Casolo, Krikamol Muandet, Yucen Luo, and Niki Kilbertus. Learning counterfactually invariant predictors. Transactions on Machine Learning Research,

  56. [64]

    Detecting and measuring confounding using causal mechanism shifts

    Abbavaram Gowtham Reddy and Vineeth N Balasubramanian. Detecting and measuring confounding using causal mechanism shifts. Advances in Neural Information Processing Systems, 37:61677–61699, 2024

    2024

  57. [65]

    On causally disentangled representations

    Abbavaram Gowtham Reddy, Benin L Godfrey, and Vineeth N Balasubramanian. On causally disentangled representations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8089–8097, 2022

    2022

  58. [66]

    Invariant models for causal transfer learning

    Mateo Rojas-Carulla, Bernhard Schölkopf, Richard Turner, and Jonas Peters. Invariant models for causal transfer learning. Journal of Machine Learning Research, 19:1–34, 2018

    2018

  59. [67]

    Domain-adjusted regression or: Erm may already learn features sufficient for out-of-distribution generalization

    Elan Rosenfeld, Pradeep Ravikumar, and Andrej Risteski. Domain-adjusted regression or: Erm may already learn features sufficient for out-of-distribution generalization. arXiv preprint arXiv:2202.06856, 2022. 13

    2022 arXiv

  60. [68]

    Anchor regression: Heterogeneous data meet causality

    Dominik Rothenhäusler, Nicolai Meinshausen, Peter Bühlmann, and Jonas Peters. Anchor regression: Heterogeneous data meet causality. Journal of the Royal Statistical Society Series B: Statistical Methodology, 83:215–246, 01 2021

    2021

  61. [69]

    Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019

    1911 arXiv

  62. [70]

    Improving robustness against common corruptions by covariate shift adapta- tion

    Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adapta- tion. Advances in neural information processing systems, 33:11539–11551, 2020

    2020

  63. [71]

    Causality for machine learning

    Bernhard Schölkopf. Causality for machine learning. In Probabilistic and causal inference: The works of Judea Pearl, pages 765–804, 2022

    2022

  64. [72]

    Diagnosing failures of fairness transfer across distribution shift in real-world medical settings

    Jessica Schrouff, Natalie Harris, Oluwasanmi O Koyejo, Ibrahim Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong, Alexander Brown, Subhrajit Roy, Diana Mincu, Chrsitina Chen, Awa Dieng, Yuan Liu, Vivek Natarajan, Alan Karthikesalingam, Katherine A Heller, Silvia Chiappa, and Alex...

    2022

  65. [73]

    E. H. Simpson. The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society: Series B (Methodological), 13(2):238–241, 2018

    2018

  66. [74]

    Domain generalisa- tion via imprecise learning

    Anurag Singh, Siu Lun Chau, Shahine Bouabid, and Krikamol Muandet. Domain generalisa- tion via imprecise learning. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 45544–45570. PMLR, 2024

    2024

  67. [75]

    Certifying some distributional robustness with principled adversarial training.arXiv preprint arXiv:1710.10571, 2017

    Aman Sinha, Hongseok Namkoong, Riccardo V olpi, and John Duchi. Certifying some distributional robustness with principled adversarial training.arXiv preprint arXiv:1710.10571, 2017

    2017 arXiv

  68. [76]

    Saint: Improved neural networks for tabular data via row attention and contrastive pre-training

    Gowthami Somepalli, Micah Goldblum, Avi Schwarzschild, C Bayan Bruss, and Tom Gold- stein. Saint: Improved neural networks for tabular data via row attention and contrastive pre-training. arXiv preprint arXiv:2106.01342, 2021

    2021 arXiv

  69. [77]

    Information transfer in social media, 2011

    Greg Ver Steeg and Aram Galstyan. Information transfer in social media, 2011

    2011

  70. [78]

    Information-theoretic measures of influence based on content dynamics, 2013

    Greg Ver Steeg and Aram Galstyan. Information-theoretic measures of influence based on content dynamics, 2013

    2013

  71. [79]

    When training and test sets are different: characterizing learning transfer

    Amos Storkey et al. When training and test sets are different: characterizing learning transfer. Dataset shift in machine learning, 30(3-28):6, 2009

    2009

  72. [80]

    Machine learning in non-stationary environments: Introduction to covariate shift adaptation

    Masashi Sugiyama and Motoaki Kawanabe. Machine learning in non-stationary environments: Introduction to covariate shift adaptation. MIT press, 2012

    2012

  73. [82]

    Deep coral: Correlation alignment for deep domain adaptation

    Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Computer vision–ECCV 2016 workshops: Amsterdam, the Netherlands, October 8-10 and 15-16, 2016, proceedings, part III 14, pages 443–450. Springer, 2016

    2016

  74. [83]

    Domain adaptation with conditional distribution matching and generalized label shift

    Remi Tachet des Combes, Han Zhao, Yu-Xiang Wang, and Geoffrey J Gordon. Domain adaptation with conditional distribution matching and generalized label shift. Advances in Neural Information Processing Systems, 33:19276–19289, 2020

    2020

  75. [84]

    Csi: Novelty detection via contrastive learning on distributionally shifted instances

    Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems, 33:11839–11852, 2020. 14

    2020

  76. [85]

    Proxy methods for domain adaptation

    Katherine Tsai, Stephen R Pfohl, Olawale Salaudeen, Nicole Chiou, Matt Kusner, Alexander D’Amour, Sanmi Koyejo, and Arthur Gretton. Proxy methods for domain adaptation. In International Conference on Artificial Intelligence and Statistics, pages 3961–3969. PMLR, 2024

    2024

  77. [86]

    The nature of statistical learning theory

    Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999

    1999

  78. [87]

    Shanmukha Ramakrishna Vedantam, David Lopez-Paz, and David J. Schwab. An empirical investigation of domain generalization with empirical risk minimizers. In A. Beygelzimer, Y . Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021

    2021

  79. [88]

    Generalizing to unseen domains: A survey on domain generalization

    Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and S Yu Philip. Generalizing to unseen domains: A survey on domain generalization. IEEE transactions on knowledge and data engineering, 35:8052–8072, 2022

    2022

  80. [89]

    Online adaptation to label distribu- tion shift

    Ruihan Wu, Chuan Guo, Yi Su, and Kilian Q Weinberger. Online adaptation to label distribu- tion shift. Advances in Neural Information Processing Systems, 34:11340–11351, 2021

    2021

  81. [90]

    Adversarial domain adaptation with domain mixup

    Minghao Xu, Jian Zhang, Bingbing Ni, Teng Li, Chengjie Wang, Qi Tian, and Wenjun Zhang. Adversarial domain adaptation with domain mixup. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 6502–6509, 2020

    2020

  82. [91]

    Improve unsupervised domain adaptation with mixup training

    Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, and Liu Ren. Improve unsupervised domain adaptation with mixup training. arXiv preprint arXiv:2001.00677, 2020

    2001 arXiv

  83. [92]

    Generalized out-of-distribution detection: A survey

    Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. International Journal of Computer Vision, 132(12):5635–5662, 2024

    2024

  84. [93]

    Towards a theoretical framework of out-of-distribution generalization

    Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li, Zhenguo Li, and Liwei Wang. Towards a theoretical framework of out-of-distribution generalization. In A. Beygelzimer, Y . Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021

    2021

  85. [94]

    Coping with label shift via distributionally robust optimisation

    Jingzhao Zhang, Aditya Menon, Andreas Veit, Srinadh Bhojanapalli, Sanjiv Kumar, and Suvrit Sra. Coping with label shift via distributionally robust optimisation. arXiv preprint arXiv:2010.12230, 2020

    2010 arXiv

  86. [95]

    Coping with label shift via distributionally robust optimisation

    Jingzhao Zhang, Aditya Krishna Menon, Andreas Veit, Srinadh Bhojanapalli, Sanjiv Kumar, and Suvrit Sra. Coping with label shift via distributionally robust optimisation. InInternational Conference on Learning Representations, 2021

    2021

  87. [96]

    Domain adaptation under target and conditional shift

    Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In International conference on machine learning, pages 819–827. Pmlr, 2013

    2013

  88. [97]

    Federated learning with domain generalization

    Liling Zhang, Xinyu Lei, Yichun Shi, Hongyu Huang, and Chao Chen. Federated learning with domain generalization. arXiv preprint arXiv:2111.10487, 2021

    2021 arXiv

  89. [98]

    On learning invariant representations for domain adaptation

    Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In International conference on machine learning, pages 7523–7532, 2019

    2019

  90. [99]

    Fundamental limits and tradeoffs in invariant representation learning

    Han Zhao, Chen Dan, Bryon Aragam, Tommi S Jaakkola, Geoffrey J Gordon, and Pradeep Ravikumar. Fundamental limits and tradeoffs in invariant representation learning. In Journal of machine learning research, volume 23, pages 1–49, 2022

    2022

  91. [100]

    Domain generalization: A survey

    Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45:4396–4415, 2022. 15 Appendix A Proofs of propositions Proposition 4.1. For a covariate vector X, label Y , with...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.