Pith. sign in

REVIEW 3 major objections 4 minor 72 references

Towards Understanding Extrapolation: a Causal Lens

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Extrapolation is achievable, under a minimal-change latent model, by identifying an invariant variable that stays on-support.

desk verdict A real contribution to single-sample extrapolation theory, but the dense-shift theorem has a genuine proof gap that needs fixing before I'd trust it. read the letter →

arxiv 2501.09163 v1 pith:EPWIIK2W submitted 2025-01-15 cs.LG cs.AIstat.ML

Lingjing Kong , Guangyi Chen , Petar Stojanov , Haoxuan Li , Eric P. Xing , Kun Zhang This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords extrapolationdistributionshiftlatentvariableidentificationminimalchangeprincipletest-timeadaptationcausalgenerativemodelidentifiabilitysparseshifts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that extrapolation, predicting labels for a target sample outside the training support, is possible when the shift obeys a minimal change principle: only a latent subspace $s$ changes and moves off-support, while an invariant latent subspace $c$ stays on-support and carries the label. It casts extrapolation as identifying $c$, and gives two identification theorems. If the shift is dense, identification holds when $c$ is discrete, the class manifolds are separated, and the target $s$ is within a distance bound set by the manifold separation and the smoothness of the generating map. If the shift is sparse, identification holds with no bound on the off-support distance, under an informativeness and a Jacobian-rank dependence condition. This would let a classifier trained on labeled source data be applied directly to off-support target samples.

What carries the argument

The load-bearing object is the latent-variable generative model $x=g(c,s)$, $y=g_y(c)$, where $z=[c,s]$ is partitioned into an invariant variable $c$ and a changing variable $s$, with $g$ smooth and invertible. The estimation objective is maximum likelihood on the source density matched to $p(x)$, plus maximization of the target likelihood $p(c_{\mathrm{tgt}})$; in the dense case an additional term minimizes the off-support distance of the estimated $s$. The proof uses block-wise identifiability of $c$, meaning estimated and true $c$ values correspond bijectively, which is established on the source support and then extended off support either by the distance bound in the dense case or by identifying the $c$-only-influenced dimensions through Jacobian rank conditions in the sparse case.

What would settle it

Exhibit one target sample satisfying Assumption 4.1, with a smooth invertible $g$, compact source support, finite $c$, continuous $p(s|c)$, and $s_{\mathrm{tgt}}$ within the stated distance bound, for which every model solving the objective in Eq. 2 assigns the target to the wrong $c$; such a case would refute Theorem 4.2. Conversely, for the sparse case, a generative process satisfying Assumption 4.3 in which Eq. 3 maps two distinct true $c$ values to the same estimated $c$ would refute Theorem 4.4.

Watch

Extended reading notes

Core claim

Under the generative model $x=g(c,s)$, $y=g_y(c)$, with $c$ the invariant variable and $s$ the changing variable, extrapolation succeeds exactly when the invariant variable $c$ is identifiable for the target sample. Theorem 4.2 shows that under dense influence of $s$, maximizing the target likelihood subject to matching the source density and minimizing the off-support distance of $s$ identifies $c$, provided $c$ is finite-valued, the manifolds $g(c,\cdot)$ are separated, and $s_{\mathrm{tgt}}$ is close enough to the source support. Theorem 4.4 shows that when $s$ influences at most $d_s$ dimensions and the $c$-only-influenced dimensions are informative and mechanistically dependent, the same identification holds with no restriction on how far $s_{\mathrm{tgt}}$ moves. The label then transfers because $c_{\mathrm{tgt}}$ remains inside the source support, so a classifier $f: c \mapsto y$ trained on the source applies to the target.

Load-bearing premise

The target sample is out of support only because a separate latent factor $s$ moves outside its training range, while the label-relevant factor $c$ remains inside the training range and the label depends only on $c$.

Editorial extensions

If this is right

  • With as few as one out-of-support target sample, a classifier trained on $c \mapsto y$ from the labeled source can transfer without retraining on the target.
  • Dense global shifts are extrapolable only up to a severity governed by the ratio between class-manifold separation and the generating map's Jacobian norm.
  • Sparse local shifts are extrapolable at any severity, provided the $c$-only-influenced dimensions remain informative and mechanistically dependent.
  • The theory motivates practical test-time adaptation: entropy minimization acts as target-likelihood maximization and sparsity constraints act as minimal-change regularization, and the paper reports gains on both.
  • The trade-off between shift scope and severity is testable: classification error should stay roughly constant across severity levels for sparse shifts but degrade sharply for dense shifts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The dense-shift bound suggests a practical diagnostic: estimate the between-manifold separation and the Jacobian norm, then predict the maximum safe shift before extrapolation collapses; the paper does not develop this diagnostic.
  • If the latent decomposition is correct, extrapolation failure should be predictable from whether the shift also moves $c$ or overlaps label-relevant dimensions, so measuring that overlap could indicate when a target prediction can be trusted.
  • The sparse-shift rank-dependence condition may extend naturally to object-centric and compositional generalization, where $c$ is an object identity and $s$ is a pose or background variable, a connection the paper leaves implicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies extrapolation, i.e., prediction on a target sample lying outside the support of the training distribution. It proposes a latent-variable model embodying a "minimal change principle": the target is out-of-support only through a changing latent variable s, while an invariant latent variable c remains inside the source support and the label depends only on c. Extrapolation is then reduced to identifying c. The authors give two identifiability results: Theorem 4.2 for dense shifts (all dimensions of x may be affected by s, but the off-support distance of the target's s is bounded) and Theorem 4.4 for sparse shifts (only few dimensions are affected, with no bound on the off-support distance). They also propose practical test-time adaptation regularizers motivated by the theory and report synthetic and real-world experiments.

Significance. If the two identification theorems were correct, the paper would make a valuable theoretical contribution: it would provide conditions under which latent-variable identification and subsequent classification are possible with a single off-support target sample, without overlapping supports or a full target distribution. The connection between the theory and practical TTA algorithms is interesting, and the empirical gains on CIFAR-C and ImageNet-C, while modest, support the practical relevance. The paper also has the strength of addressing a clearly defined and practically important problem, and it explicitly discusses limitations. However, the central proof of the dense-shift theorem has gaps that are currently load-bearing, and the sparse-shift proof depends on assumptions whose satisfaction by the stated objective is not fully justified. The manuscript is therefore not yet ready for acceptance.

major comments (3)
  1. [Appendix A2, Eqs. (4)-(5), Theorem 4.2] The derivation of the lower bound Ju(||h|| + ||h'||) >= D(ck, ck') is not justified. The bound ||Jg(z)|| <= Ju is introduced in Section 4.1 as holding "on the support," but the path integrals in Eqs. (4)-(5) run from s_src and s'_src, which lie on the boundary of S_src, to stgt and s'_tgt, which lie outside S_src; the paths immediately leave the region where the bound is assumed. Moreover, Eq. (4) applies Jg to the alternative generating function g', even though g' is only assumed to equal g on C times S_src and may have an arbitrarily large Jacobian off the support. Consequently, the inequality leading to max{||h||, ||h'||} >= D(ck, ck')/(2Ju) and the subsequent exclusion of alternative models in Eq. (6) do not follow. This gap is load-bearing for the entire dense-shift theorem; it could be repaired by assuming a uniform global Jacobian bound on both g and all admissible g' over the relevant extrapolation region, but that is not assumed in the paper.
  2. [Objective (2) and Appendix A2, proof of Theorem 4.2] The proof restricts the alternative generating function g' to satisfy g' = g over C times S_src, but the optimization in Eq. (2) only enforces the marginal equality p_hat(x) = p(x) on X_src. This does not force g_hat to agree with g on the support; any invertible reparameterization s_hat = phi(s) with g_hat(c, s_hat) = g(c, phi^{-1}(s_hat)) also matches the source distribution. Since Lemma A1 identifies c but not s, the estimated source support S_hat_src and the distance D(s_hat, S_hat_src) depend on the coordinate choice for s. An alternative model can therefore compress the s-coordinate and make its off-support distance arbitrarily small, undermining the "minimal off-support distance" selection step. The proof needs an additional condition that fixes the coordinate system or a model class for which distances are comparable across admissible models.
  3. [Appendix A2, Lemma A1 (from [38])] The proof of Theorem 4.2 begins with Lemma A1, which asserts source-support identification of the discrete invariant variable c and is imported verbatim from the authors' earlier preprint [38] without proof. This lemma is the foundation of the on-support identification step, and the paper gives no argument for it. Because the result is self-cited and not available in the present manuscript, the paper is not self-contained. Please include a proof of Lemma A1 in the appendix or state it as an additional assumption.
minor comments (4)
  1. [Appendix A3, Lemma A1 (Brady et al.)] The statement of Lemma A1 says "S subset of R^ds," but in the proof the object [Jg(z)]_S is used as a submatrix indexed by a set of output dimensions, e.g., Ic\s(z) subset of [dx]. The statement should be corrected to "S subset of [dx]" to match its use.
  2. [Section 4.1, definition of D(c1, c2)] The definition D(c1, c2) := inf_{s1,s2 in Bd(S_src)} ||g(c1, s1) - g(c2, s2)|| assumes the boundary of the source support is nonempty and the infimum is attained; this is not discussed, though it follows from compactness of the boundary under Assumption 4.1-ii together with continuity of g. A brief comment would help.
  3. [Section 6, Tables 2 and 4] The improvements reported in Table 2 (e.g., TeSLA-s from 12.1 to 11.7 on CIFAR10-C) are small and the standard deviations are not reported for the +SC variants in the same way as for the baselines; please clarify the number of seeds and whether the differences are statistically significant.
  4. [Section 5, synthetic experiments] The synthetic experiments use a VAE-based estimator, which does not satisfy the invertibility assumption (Assumption 4.1-i) exactly. The paper should acknowledge this gap between the theoretical conditions and the experimental implementation, or justify why the VAE approximately respects the condition.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the identification theorems are argued from explicit assumptions, and the imported Lemma A1 concerns only source-support identification; the noted Theorem 4.2 Jacobian gap is a correctness issue, not a reduction of the conclusion to the inputs.

full rationale

The claimed derivation is not circular. The latent-variable model in Eq. (1) and the minimal-change conditions are stated assumptions, not conclusions derived from the estimation objective. Theorem 4.2 and Theorem 4.4 then prove sufficient conditions under which the estimated model recovers the invariant variable, so the theorems do not merely restate the generative model. The main imported ingredient is Lemma A1 from Kong et al. [38], which is a same-author prior result, but it is used only to identify the discrete variable c on the source support; its assumptions (Assumption 4.1 i-iv) do not include the off-support target sample, and the target identification is argued separately through the distance bound in Assumption 4.1-v and the off-support minimization in Eq. (2), or through the index-set and rank arguments in Appendix A3. That citation is therefore independent support for a sub-claim, not an unverified premise that already contains the theorem. One non-circular concern should be flagged: the proof of Theorem 4.2 applies the support-only Jacobian bound J_u to off-support path integrals and to alternative functions g' that are unconstrained off support, so the proof has a genuine gap; however, a proof gap is a soundness issue, not circularity, because the conclusion is not equivalent to the assumptions by construction. The empirical validation is external and the synthetic experiments are illustrative rather than the source of the theoretical claim.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The central theorems rest on the minimal-change generative model and on technical identifiability assumptions. The most consequential imported item is Lemma A1 from the authors' own prior work, which is not proved in this text; the remaining assumptions are standard within the partial-disentanglement and nonlinear-ICA literatures.

free parameters (1)
  • Empirical hyperparameters (lambda_KL, lambda_s_distance, LoRA rank, sparsity ratio) = lambda_s_distance=0.01; lambda_KL in {1e-1, 1e-2, 1e-3}; Table A2
    Hand-set for the synthetic and real-world experiments. They do not enter the identification theorems, but the empirical demonstrations of the theory depend on them.
assumptions (6)
  • domain assumption Minimal change principle: x = g(c,s), y = gy(c), ctgt in Csrc, stgt outside Ssrc.
    Section 3, Eq. 1. Central modeling premise; if the semantic content itself shifts, extrapolation is impossible by construction.
  • domain assumption Assumption 4.1-i: g is smooth and invertible with smooth inverse everywhere.
    Used to bound the effect of latent shifts by the Jacobian norm Ju and to invoke source identifiability. Standard in the iVAE family but not guaranteed for real image generators.
  • domain assumption Lemma A1 (source discrete subspace identification, from Kong et al. [38]).
    Imported, not proved here; this is the step that identifies c on the source support, and Theorem 4.2's target step builds on it.
  • domain assumption Assumption 4.1-v: distance of stgt from Ssrc bounded by min_{c != ctgt} D(ctgt,c) / (2 Ju).
    Load-bearing for dense shifts: the target must be close enough that only the correct manifold can explain it. Hard to verify without knowing the true generator and manifold separation.
  • domain assumption Assumption 4.3-iv: mechanistic dependence of c-only dimensions, rank([Jg]_{Ic\s}) < rank([Jg]_{P1}) + rank([Jg]_{P2}).
    The sparse theorem uses this Jacobian-rank condition to locate the unaffected image dimensions; it is a strong structural assumption and hard to verify in practice.
  • domain assumption Assumption 4.3-ii: the c-only image dimensions uniquely determine c: [x]_{Ic\s(c,s1)} != [x]_{Ic\s(c*,s2)} for c* != c.
    Ensures the sparse target's unaffected region cannot be produced by another class; required for the final step of Theorem 4.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Understanding Extrapolation: a Causal Lens." pith.science (2026). https://pith.science/paper/EPWIIK2W

@misc{pith2026250109163,
  author       = {Pith},
  title        = {Pith review of: Towards Understanding Extrapolation: a Causal Lens},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EPWIIK2W}},
  note         = {Machine review of arXiv:2501.09163}
}
read the original abstract

Canonical work handling distribution shifts typically necessitates an entire target distribution that lands inside the training distribution. However, practical scenarios often involve only a handful of target samples, potentially lying outside the training support, which requires the capability of extrapolation. In this work, we aim to provide a theoretical understanding of when extrapolation is possible and offer principled methods to achieve it without requiring an on-support target distribution. To this end, we formulate the extrapolation problem with a latent-variable model that embodies the minimal change principle in causal mechanisms. Under this formulation, we cast the extrapolation problem into a latent-variable identification problem. We provide realistic conditions on shift properties and the estimation objectives that lead to identification even when only one off-support target sample is available, tackling the most challenging scenarios. Our theory reveals the intricate interplay between the underlying manifold's smoothness and the shift properties. We showcase how our theoretical results inform the design of practical adaptation algorithms. Through experiments on both synthetic and real-world data, we validate our theoretical findings and their practical implications.

Figures

Figures reproduced from arXiv: 2501.09163 by the authors.

Figure 1
Figure 1. Illustration of extrapolation and our theoretical conditions. The horizontal axis represents the changing variable s, ranging from the source support to out-of-support regions. The vertical axis represents the observed data x living on the manifolds indexed by different values of the invariant variable c. Figure (a) demonstrates that given a point out of support it is unclear which class manifolds it belongs to. Fig… view at source ↗
Figure 2
Figure 2. The data-generating process. The invariant latent variable c and the changing latent variable s jointly generate the observed variable x. The dashed line indicates potential statistical dependence. In this process, the latent space z ∈ Z ⊂ R dz comprises two subspaces: the invariant variable c ∈ C ⊂ R dc and the changing variable s ∈ S ⊂ R ds . We define Z := Zsrc ∪ {ztgt} as the source support augmented with the ta… view at source ↗
Figure 3
Figure 3. TTA classification errors under different levels of shift severity levels and scopes. To investigate the trade-off between the shift scope (dense vs. sparse) and severity, we sim￾ulate different levels of corruption severity and corrupted region sizes and evaluate a classical TTA method TENT [15] on these configurations. Following [45], we inject impulse noise to the CIFAR10 dataset, with noise levels ranging from 1… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 43 canonical work pages

  1. [38]

    Learning Discrete Concepts in Latent Hierarchical Models

    Lingjing Kong, Guangyi Chen, Biwei Huang, Eric P Xing, Yuejie Chi, and Kun Zhang. Learning discrete concepts in latent hierarchical models. arXiv preprint arXiv:2406.00519, 2024

    work page Pith review arXiv 2024

  2. [1]

    Measuring robustness to natural distribution shifts in image classification

    Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems, 33:18583–18599, 2020

    2020

  3. [2]

    Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pages 5389–5400

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pages 5389–5400. PMLR, 2019

    2019

  4. [3]

    Wilds: A benchmark of in-the-wild distribution shifts

    Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pages 5637–5664. PMLR, 2021

    work page 2021

  5. [4]

    In search of lost domain generalization

    Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint arXiv:2007.01434, 2020

    arXiv 2007

  6. [5]

    Improving predictive inference under covariate shift by weighting the log-likelihood function

    Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, 90(2):227–244, 2000

    2000

  7. [6]

    Correcting sample selection bias by unlabeled data

    Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf, and Alex Smola. Correcting sample selection bias by unlabeled data. Advances in Neural Information Processing Systems, 19, 2006

    work page 2006

  8. [7]

    Direct importance estimation for covariate shift adaptation

    Masashi Sugiyama, Taiji Suzuki, Shinichi Nakajima, Hisashi Kashima, Paul V on Bünau, and Motoaki Kawanabe. Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics, 60:699–746, 2008

    work page 2008

Show all 72 references
  1. [8]

    Domain adaptation under target and conditional shift

    Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In International Conference on Machine Learning, pages 819–827, 2013

    2013

  2. [9]

    Low-dimensional density ratio estimation for covariate shift correction

    Petar Stojanov, Mingming Gong, Jaime Carbonell, and Kun Zhang. Low-dimensional density ratio estimation for covariate shift correction. In The 22nd international conference on artificial intelligence and statistics, pages 3449–3458. PMLR, 2019

    2019

  3. [10]

    The risks of invariant risk minimiza- tion

    Elan Rosenfeld, Pradeep Ravikumar, and Andrej Risteski. The risks of invariant risk minimiza- tion. In International Conference on Learning Representations, volume 9, 2021

    2021

  4. [11]

    Generalizing to unseen domains via distribution matching

    Isabela Albuquerque, João Monteiro, Mohammad Darvishi, Tiago H Falk, and Ioannis Mitliagkas. Generalizing to unseen domains via distribution matching. arXiv preprint arXiv:1911.00804, 2019. 11

    1911 arXiv

  5. [12]

    Domain generalization by marginal transfer learning

    Gilles Blanchard, Aniket Anand Deshmukh, Urun Dogan, Gyemin Lee, and Clayton Scott. Domain generalization by marginal transfer learning. Journal of machine learning research, 22 (2):1–55, 2021

    2021

  6. [13]

    Test-time training with self-supervision for generalization under distribution shifts

    Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning, pages 9229–9248. PMLR, 2020

    2020

  7. [14]

    Ttt++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34, 2021

    Yuejiang Liu, Parth Kothari, Bastien van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Ttt++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34, 2021

    2021

  8. [15]

    Tent: Fully test-time adaptation by entropy minimization

    Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726, 2020

    2006 arXiv

  9. [16]

    Think locally, act globally: Federated learning with local and global representations

    Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.01523, 2020

    2001 arXiv

  10. [17]

    A comprehensive survey on test-time adaptation under distribution shifts

    Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribution shifts. arXiv preprint arXiv:2303.15361, 2023

    2023 arXiv

  11. [18]

    Partial disentanglement for domain adaptation

    Lingjing Kong, Shaoan Xie, Weiran Yao, Yujia Zheng, Guangyi Chen, Petar Stojanov, Victor Akinwande, and Kun Zhang. Partial disentanglement for domain adaptation. In International Conference on Machine Learning, pages 11455–11472. PMLR, 2022

    2022

  12. [19]

    Subspace identification for multi-source domain adaptation

    Zijian Li, Ruichu Cai, Guangyi Chen, Boyang Sun, Zhifeng Hao, and Kun Zhang. Subspace identification for multi-source domain adaptation. Advances in Neural Information Processing Systems, 36, 2023

    2023

  13. [20]

    Test-time training with masked autoencoders

    Yossi Gandelsman, Yu Sun, Xinlei Chen, and Alexei Efros. Test-time training with masked autoencoders. Advances in Neural Information Processing Systems, 35:29374–29385, 2022

    2022

  14. [21]

    Tesla: Test- time self-learning with automatic adversarial augmentation

    Devavrat Tomar, Guillaume Vray, Behzad Bozorgtabar, and Jean-Philippe Thiran. Tesla: Test- time self-learning with automatic adversarial augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20341–20350, 2023

    2023

  15. [22]

    A theory of learning from different domains

    Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79: 151–175, 2010

    2010

  16. [23]

    Covariate shift adaptation by importance weighted cross validation

    Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007

    2007

  17. [24]

    Additive decoders for latent variables identification and cartesian-product extrapolation

    Sébastien Lachapelle, Divyat Mahajan, Ioannis Mitliagkas, and Simon Lacoste-Julien. Additive decoders for latent variables identification and cartesian-product extrapolation. Advances in Neural Information Processing Systems, 36, 2023

    2023

  18. [25]

    Com- positional generalization from first principles

    Thaddäus Wiedemer, Prasanna Mayilvahanan, Matthias Bethge, and Wieland Brendel. Com- positional generalization from first principles. Advances in Neural Information Processing Systems, 36, 2023

    2023

  19. [26]

    Provable compositional generalization for object-centric learning

    Thaddäus Wiedemer, Jack Brady, Alexander Panfilov, Attila Juhos, Matthias Bethge, and Wieland Brendel. Provable compositional generalization for object-centric learning. arXiv preprint arXiv:2310.05327, 2023

    2023 arXiv

  20. [27]

    Toward compositional generalization in object-oriented world modeling

    Linfeng Zhao, Lingzhi Kong, Robin Walters, and Lawson LS Wong. Toward compositional generalization in object-oriented world modeling. In International Conference on Machine Learning, pages 26841–26864. PMLR, 2022

    2022

  21. [28]

    Learning to extrapolate: A transductive approach

    Aviv Netanyahu, Abhishek Gupta, Max Simchowitz, Kaiqing Zhang, and Pulkit Agrawal. Learning to extrapolate: A transductive approach. In The Eleventh International Conference on Learning Representations, 2022. 12

    2022

  22. [29]

    Engression: Extrapolation for nonlinear regression? In 2023 IMS International Conference on Statistics and Data Science (ICSDS), page 232, 2023

    Xinwei Shen and Nicolai Meinshausen. Engression: Extrapolation for nonlinear regression? In 2023 IMS International Conference on Statistics and Data Science (ICSDS), page 232, 2023

    2023

  23. [30]

    First steps toward understanding the extrapolation of nonlinear models to unseen domains

    Kefan Dong and Tengyu Ma. First steps toward understanding the extrapolation of nonlinear models to unseen domains. In The Eleventh International Conference on Learning Representa- tions, 2022

    2022

  24. [31]

    Identifying representations for intervention extrapolation

    Sorawit Saengkyongam, Elan Rosenfeld, Pradeep Kumar Ravikumar, Niklas Pfister, and Jonas Peters. Identifying representations for intervention extrapolation. In The Twelfth International Conference on Learning Representations, 2023

    2023

  25. [32]

    Independent component analysis: algorithms and applications

    Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):411–430, 2000

    2000

  26. [33]

    Variational au- toencoders and nonlinear ica: A unifying framework

    Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational au- toencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pages 2207–2217. PMLR, 2020

    2020

  27. [34]

    Kingma, and Aapo Hyvarinen

    Ilyes Khemakhem, Ricardo Pio Monti, Diederik P. Kingma, and Aapo Hyvarinen. Ice-beem: Identifiable conditional energy-based deep models based on nonlinear ica, 2020

    2020

  28. [35]

    Unsupervised feature extraction by time-contrastive learning and nonlinear ica

    Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems, 29, 2016

    2016

  29. [36]

    Nonlinear ica using auxiliary variables and generalized contrastive learning

    Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859–868. PMLR, 2019

    2019

  30. [37]

    Self-supervised learning with data augmentations provably isolates content from style, 2021

    Julius von Kugelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style, 2021

    2021

  31. [39]

    Provably learning object-centric representations

    Jack Brady, Roland S Zimmermann, Yash Sharma, Bernhard Schölkopf, Julius V on Kügelgen, and Wieland Brendel. Provably learning object-centric representations. In International Conference on Machine Learning, pages 3038–3062. PMLR, 2023

    2023

  32. [40]

    A dirt-t approach to unsupervised domain adaptation

    Rui Shu, Hung Bui, Hirokazu Narui, and Stefano Ermon. A dirt-t approach to unsupervised domain adaptation. In International Conference on Learning Representations, 2018

    2018

  33. [41]

    Semi-supervised learning by entropy minimization

    Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in neural information processing systems, 17, 2004

    2004

  34. [42]

    Evaluating prediction-time batch normalization for robustness under covariate shift

    Zachary Nado, Shreyas Padhy, D Sculley, Alexander D’Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006.10963, 2020

    2006 arXiv

  35. [43]

    Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation

    Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, pages 6028–6039. PMLR, 2020

    2020

  36. [44]

    Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering

    Yongyi Su, Xun Xu, and Kui Jia. Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering. Advances in Neural Information Processing Systems, 35:17543–17555, 2022

    2022

  37. [45]

    Benchmarking neural network robustness to common corruptions and perturbations

    Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019

    1903 arXiv

  38. [46]

    Contrastive multiview coding

    Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In European conference on computer vision, pages 776–794. Springer, 2020. 13

    2020

  39. [47]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  40. [48]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021

    2021

  41. [49]

    Towards understanding gd with hard and conjugate pseudo-labels for test-time adaptation

    Jun-Kun Wang and Andre Wibisono. Towards understanding gd with hard and conjugate pseudo-labels for test-time adaptation. In The Eleventh International Conference on Learning Representations, 2023

    2023

  42. [50]

    Test time adaptation via conjugate pseudo-labels

    Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and J Zico Kolter. Test time adaptation via conjugate pseudo-labels. Advances in Neural Information Processing Systems, 35:6204–6218, 2022

    2022

  43. [51]

    Ttn: A domain-shift aware batch normalization in test-time adaptation

    Hyesu Lim, Byeonggeun Kim, Jaegul Choo, and Sungha Choi. Ttn: A domain-shift aware batch normalization in test-time adaptation. In The Eleventh International Conference on Learning Representations, 2023

    2023

  44. [52]

    Learning to generalize across domains on single test samples

    Zehao Xiao, Xiantong Zhen, Ling Shao, and Cees GM Snoek. Learning to generalize across domains on single test samples. arXiv preprint arXiv:2202.08045, 2022

    2022 arXiv

  45. [53]

    Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization

    Junha Song, Jungsoo Lee, In So Kweon, and Sungha Choi. Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11920–11929, 2023

    2023

  46. [54]

    Test- time adaptation of discriminative models via diffusion generative feedback

    Mihir Prabhudesai, Tsung-Wei Ke, Alex Li, Deepak Pathak, and Katerina Fragkiadaki. Test- time adaptation of discriminative models via diffusion generative feedback. Advances in Neural Information Processing Systems, 36, 2023

    2023

  47. [55]

    Swapprompt: Test-time prompt adaptation for vision-language models

    Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu. Swapprompt: Test-time prompt adaptation for vision-language models. Advances in Neural Information Processing Systems, 36, 2023

    2023

  48. [56]

    Actmad: Activation matching to align distributions for test-time-training

    Muhammad Jehanzeb Mirza, Pol Jané Soneira, Wei Lin, Mateusz Kozinski, Horst Possegger, and Horst Bischof. Actmad: Activation matching to align distributions for test-time-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2415...

    2023

  49. [57]

    On the robustness of open-world test-time training: Self-training with dynamic prototype expansion

    Yushu Li, Xun Xu, Yongyi Su, and Kui Jia. On the robustness of open-world test-time training: Self-training with dynamic prototype expansion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11836–11846, 2023

    2023

  50. [58]

    Feature alignment and uniformity for test time adaptation

    Shuai Wang, Daoan Zhang, Zipei Yan, Jianguo Zhang, and Rui Li. Feature alignment and uniformity for test time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20050–20060, 2023

    2023

  51. [59]

    Test-time adaptation via self-training with nearest neighbor information

    Minguk Jang, Sae-Young Chung, and Hye Won Chung. Test-time adaptation via self-training with nearest neighbor information. arXiv preprint arXiv:2207.10792, 2022

    2022 arXiv

  52. [60]

    Improved test-time adaptation for domain generalization

    Liang Chen, Yong Zhang, Yibing Song, Ying Shan, and Lingqiao Liu. Improved test-time adaptation for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24172–24182, 2023

    2023

  53. [61]

    Efficient test-time model adaptation without forgetting

    Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International confer- ence on machine learning, pages 16888–16905. PMLR, 2022

    2022

  54. [62]

    Delta: Degradation-free fully test-time adaptation

    Bowen Zhao, Chen Chen, and Shu-Tao Xia. Delta: Degradation-free fully test-time adaptation. In The Eleventh International Conference on Learning Representations, 2023

    2023

  55. [63]

    Towards stable test-time adaptation in dynamic wild world

    Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In The Eleventh International Conference on Learning Representations, 2023. 14

    2023

  56. [64]

    Test-time classifier adjustment module for model-agnostic domain generalization

    Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems, 34:2427–2440, 2021

    2021

  57. [65]

    Contrastive test-time adaptation

    Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 295–305, 2022

    2022

  58. [66]

    On pitfalls of test-time adaptation

    Hao Zhao, Yuejiang Liu, Alexandre Alahi, and Tao Lin. On pitfalls of test-time adaptation. In International Conference on Machine Learning, pages 42058–42080. PMLR, 2023

    2023

  59. [67]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  60. [68]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations, 2014

    2014

  61. [69]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  62. [70]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020

    2003 arXiv

  63. [71]

    Towards Understanding Extrapolation: a Causal Lens

    Yongcan Yu, Lijun Sheng, Ran He, and Jian Liang. Benchmarking test-time adaptation against distribution shifts in image classification. arXiv preprint arXiv:2307.03133, 2023. 15 Appendix for “Towards Understanding Extrapolation: a Causal Lens ” Table of Contents A1 Related Wor...

    2023 arXiv

  64. [72]

    In this stage, only the encoder is updated

    Stage One: We follow the MAE-TTT approach by inputting masked images and training the model using reconstruction loss. In this stage, only the encoder is updated. 2) Stage Two: We input full images (32 in a batch) and optimize the model with the entropy minimization loss follo...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.