Pith. sign in

REVIEW 3 major objections 3 minor 64 references

Moment Alignment: Unifying Gradient and Hessian Matching for Domain Generalization

T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves that aligning the derivatives of the loss across training domains bounds the error on an unseen target, unifying invariant risk minimization, gradient matching, and Hessian matching as special cases of moment alignment.

desk verdict The unification idea and CMA are worth a look, but the central bound is false as stated, so the theory needs major repair before it can be trusted. read the letter →

arxiv 2506.07378 v1 pith:IDMLOJ3W submitted 2025-06-09 cs.LG stat.ML

Yuen Chen , Haozhe Si , Guojun Zhang , Han Zhao This is my paper
classification cs.LGstat.ML MSC 68T0768Q32
keywords domaingeneralizationmomentalignmenttransfermeasuregradientmatchingHessianinvariantriskminimizationclosed-formdistributionshift
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Domain generalization—training a model on several labeled domains so it works on an unseen related domain—is, the paper argues, a moment-alignment problem. The target error of a predictor is bounded, under a mixture assumption on the target, by the transfer measure between the source domains, and that transfer measure is in turn bounded by the maximum pairwise difference of the domains' higher-order loss derivatives with respect to the classifier. Gradient differences and Hessian differences are just the first two terms in this derivative expansion, so IRM penalties, gradient matching, and Hessian matching become truncations of a single objective. The paper's Closed-Form Moment Alignment algorithm computes the gradient and Hessian alignment terms analytically for a linear classifier head, eliminating the repeated backpropagation or sampling-based Hessian estimation that made earlier second-order methods costly. Experiments report performance comparable to existing moment-matching methods, with better worst-group accuracy on several datasets.

What carries the argument

The machinery is the extended transfer measure, built from a minimax center $\mu^* := \arg\min_\mu \max_i T_{\Gamma}(\mu_i\|\mu)$, together with Proposition 2's bound reducing the multi-source transfer measure to the maximum pairwise source-to-source transfer measure. Theorems 1 and 3 then expand each domain loss in a Taylor series around a common point $\theta^*$; every term is a difference of $n$th-order derivatives, so the bound says “align derivatives.” For the algorithm, the central object is the closed-form softmax Hessian $H = (\operatorname{diag}(p) - pp^{\top}) \otimes (xx^{\top})$ for a linear classifier head, whose gradient is $(p-y)x$; the Frobenius-norm penalty is evaluated without forming the Kronecker product via $\operatorname{tr}(\operatorname{diag}(p) - pp^{\top}) \operatorname{tr}(xx^{\top})$, which is what makes CMA's exact Hessian matching computationally practical.

What would settle it

Take three source distributions and one target in their convex hull, compute the exact transfer measures $T_{\Gamma}(\mu_i\|\mu_j)$ and $T_{\Gamma}(S\|T)$ for a simple loss class (for example, linear classifiers on Gaussians), and check the inequality $T_{\Gamma}(S\|T) \le \tfrac12 \max_{i\neq j} T_{\Gamma}(\mu_j\|\mu_i)$. A numerical counterexample—or a direct demonstration that no midpoint $\mu_{\mathrm{mid}}$ can satisfy $T_{\Gamma}(\mu_{\mathrm{mid}}\|\mu_i) \ge T_{\Gamma}(\mu^*\|\mu_i)$ for all $i$—would refute Proposition 2 and with it the theory's main bound.

Watch

Extended reading notes

Core claim

The central claim is that if the unseen target distribution lies in the convex hull of the source distributions, then the one-sided transfer measure $T_{\Gamma}(S\|T)$—the worst-case excess-loss gap between a virtual center domain $\mu^*$ and the target—satisfies $T_{\Gamma}(S\|T) \le \tfrac12 \max_{i\neq j} T_{\Gamma}(\mu_j\|\mu_i)$, and that this pairwise quantity is controlled by the differences of the higher-order derivatives of the domain losses. Under the IRM assumption (an invariant optimal predictor exists), the bound involves only second- and higher-order derivative differences, so Hessian and higher-moment alignment is sufficient. Without the IRM assumption, the bound also includes the per-domain optimality gaps and, when gradients are bounded, a gradient-norm term, which is why gradient matching matters. The paper reads IRMv1, gradient matching (Fish, IGA, AND-Mask), and Hessian matching (Fishr, HGP, Hutchinson, CORAL) as special cases of aligning feature-moment or loss-derivative moments, and proves a duality: matching the $n$th moment of features is equivalent to matching the $n$th derivative of the classifier head.

Load-bearing premise

The paper's central bound assumes the unseen target is a mixture (convex combination) of the training distributions, and the proof of Proposition 2 further assumes a midpoint distribution satisfying $T_{\Gamma}(\mu_{\mathrm{mid}}\|\mu_i) \ge T_{\Gamma}(\mu^*\|\mu_i)$ for all $i$, a step that the minimax definition does not by itself guarantee.

Editorial extensions

If this is right

  • Matching gradients and Hessians is not a heuristic: under the paper's assumptions it directly reduces an upper bound on the unseen-target error, so the two penalties should act additively.
  • IRM, gradient matching, and Hessian matching are the first-order, first-moment, and second-moment truncations of one moment-alignment objective, so any future domain-generalization method that aligns derivatives is automatically a moment-alignment method.
  • Because feature moments and classifier derivatives are dual, feature-covariance matching (as in CORAL) is approximately Hessian matching, and gradient penalties such as IRMv1 are approximately feature-mean alignment.
  • Exact Hessian matching no longer requires sampling or repeated backpropagation for a linear classifier head, and the memory-efficient trace form makes it feasible when the full $dC \times dC$ Hessian would not fit in memory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Testable implication the paper leaves implicit: on targets near the boundary of the source convex hull, the Hessian-difference term enters with $\delta$ while the gradient term enters with $\sqrt{\delta}$, so second-moment alignment should matter more than first-moment alignment in that regime; synthetic Gaussian-domain experiments varying the mixture weights could verify this ordering.
  • Beyond the paper's closed-form derivation, the separability of the softmax Hessian into a class-probability factor and a feature-covariance factor points to a cheaper approximation: aligning feature second moments and class-prediction confusions separately may substitute for full Hessian alignment.
  • The paper notes the per-domain optimality gap in its non-IRM bound but does not discuss its training dynamics; a corollary of the bound is that gradient and Hessian penalties should be annealed in together with the ERM loss, which is exactly the schedule CMA uses in experiments.
  • Because the feature–parameter duality is stated as exact for softmax classifiers, any new feature-moment alignment method should have a parameter-space dual; constructing such dual penalties could yield new domain-generalization algorithms without extra backpropagation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper develops a theory of 'moment alignment' for domain generalization, built on the transfer-measure framework of Zhang et al. (2021). It extends transfer measures to multiple source domains, proposes an upper bound on the target transfer measure in terms of pairwise domain transfer measures (Proposition 2), and then derives bounds showing that aligning higher-order derivatives of the loss (gradients, Hessians, etc.) across domains controls the transfer measure, both under an invariant-optimal-predictor assumption (Theorem 1) and without it (Theorem 3). The paper further argues that IRM, gradient matching, and Hessian matching are special cases of moment alignment, introduces the CMA algorithm that computes gradient and Hessian penalties in closed form for a linear classifier head, and reports linear-probing and fine-tuning experiments on several DG benchmarks.

Significance. If the theory were correct, the paper would provide a genuinely unifying account of several seemingly disconnected DG heuristics, and the closed-form Hessian computation in CMA would be a practically useful contribution over sampling- or backpropagation-based Hessian matching. The paper is also honest about several limitations, including the convex-hull assumption and the restricted hyperparameter search in the DomainBed experiments. However, the central theoretical result, Proposition 2, is false as stated, and since Theorems 1 and 3 and their corollaries all invoke Proposition 2 at their first step, the paper's main theoretical claim is unsupported. The empirical results are presented as proof-of-concept and show CMA roughly on par with Fishr, but they cannot compensate for the failure of the paper's headline theory.

major comments (3)
  1. [Section 3.2 and Supplementary A, Eq. (13)] Proposition 2 is false as stated. Consider binary classification with 0–1 loss, predictor class Γ={h1,h2} where hc is the constant predictor of class c, source domains µ1 putting all mass on label 1 and µ2 all mass on label 2, and target T=µ1. The balanced distribution µ*=(µ1+µ2)/2 makes both predictors optimal (loss 1/2), so TΓ(µ1∥µ*)=TΓ(µ2∥µ*)=0 and µ* is a valid minimax center in Definition 3. Then TΓ(S∥T)=TΓ(µ*∥µ1)=1, while max_{i≠j} TΓ(µj∥µi)=1, so the right-hand side of Eq. (3) is 1/2, contradicting the proposition. The proof's midpoint step in Eq. (13) asserts both the existence of µ_mid satisfying TΓ(µjmax∥µi)=TΓ(µjmax∥µmid)+TΓ(µmid∥µi) and the inequality TΓ(µmid∥µi)≥TΓ(µ*∥µi); the latter does not follow from the minimax definition of µ*, which only controls max_i TΓ(µi∥µ) and does not control the reversed argument order TΓ(µ∥µi). Since Theorem 1, Theorem 3, Corollary 2, Corollary 4, Proposition 3, and Proposition 4 all begin by applying Proposition 2, the central claim that aligning derivatives bounds the transfer measure is unsupported.
  2. [Supplementary B, Eq. (28), and Corollary 4, Eq. (9)] The proof of Theorem 3 is inconsistent with its statement. The theorem statement (Eq. (7)) contains the term −(Lµi(θ*)−Lµi(θ*i)), but the displayed bound in Eq. (28) of the proof replaces this with +Lµi(θ*)−Lµi(θ*i). In addition, Corollary 4's Eq. (9) writes the first-order term as ∥∇θLµj(θ*)−∇θLµj(θ*)∥2, where the second index should be i (i.e., ∇θLµi(θ*)). These are not merely typographical: the first error changes the claimed upper bound in a way that cannot be derived from the preceding Taylor expansion, and the second makes the displayed corollary vacuous as written.
  3. [Section 5.2, Eq. (11)] The memory-efficient Hessian formula is incorrect for multiclass problems. For H=(diag(p)−pp^T)⊗(xx^T), the Frobenius norm squared is ∥diag(p)−pp^T∥_F^2 ∥xx^T∥_F^2 = tr((diag(p)−pp^T)^2) tr((xx^T)^2), not tr(diag(p)−pp^T) tr(xx^T) as stated in Eq. (11). The two expressions coincide for binary classification (C=2) and for the rank-one factor xx^T, but for C>2 they differ; for example, with p=(0.5,0.3,0.2), tr(A)^2=0.3844 while tr(A^2)=0.2044 for A=diag(p)−pp^T. Since the experiments include multiclass datasets (MultiNLI and DomainBed), the implementation based on Eq. (11) would not be minimizing the stated Hessian-matching objective on those datasets.
minor comments (3)
  1. [Theorem 1 proof, Eq. (19)] The proof writes sup_{θ∈Γ} ... = sup_{∥θ−θ*∥₂²≤δ} ..., but Γ is only a subset of the ball F2, so the correct relation is ≤, not equality. The final bound still follows if the inequality is used, but the displayed equality is unjustified.
  2. [Headers, Corollary 2 and Corollary 4] The word 'Hessian' is capitalized in the main text but appears as 'hessian' in the corollary headings; please standardize.
  3. [Section 6.2 and Table 3] The main-text DomainBed results use test-domain (oracle) model selection; the training-domain selection results in H.3 show CMA underperforming Fishr on ColoredMNIST (51.4 vs. 52.2) and only tying on the other datasets. The text should state more prominently that the 'state-of-the-art' claim depends on the model-selection protocol.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the derivative-alignment bounds are derived from the transfer-measure definition by Taylor expansion, and CMA directly minimizes the resulting upper-bound terms.

full rationale

The paper's central theoretical claims (Theorems 1 and 3) are derived from Definition 3 and Proposition 2 via Taylor expansion of the per-domain losses around an invariant or weakly Pareto-optimal parameter. The transfer-measure framework is attributed to Zhang et al. (2021), which shares two authors with the present paper, but it is used as an external, published starting point rather than as an unverified self-citation that forecloses alternatives. Proposition 1 is likewise cited from prior work and then extended. The unification of IRM, gradient matching, and Hessian matching is presented as explicit algebraic equivalence: Eq. (45) rewrites the gradient-variance penalty, and Sections F.1-F.3 show that setting β=0 or α=0 in the CMA objective recovers the corresponding existing objectives. These are identities by construction, not predictions that reduce to fitted inputs. The CMA algorithm minimizes the gradient and Hessian discrepancy terms that appear in the upper bounds, which is a direct design choice rather than a circular prediction. The paper acknowledges in Section 7 that Assumption 2 (target in the convex hull of sources) is restrictive and may not hold in practice; that is an honest limitation statement, not circularity. A possible mathematical gap exists in the proof of Proposition 2 (the midpoint inequality in Eq. (13) does not follow from the minimax definition of μ*), but a proof gap or even a false lemma is a correctness issue, not a circularity: the bound does not assume its own conclusion. Overall, no step in the derivation chain reduces, by construction or by self-citation, to its own inputs.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central theory depends on the convex-hull target assumption, strong convexity of the head loss, and an unstated midpoint property in the proof of Proposition 2. The algorithm relies on per-dataset hyperparameters α, β, and annealing iterations. No new physical or mathematical entities are invented.

free parameters (3)
  • α (gradient matching weight) = 10 (Waterbirds), 5000 (CelebA), 5000 (MultiNLI); random search on DomainBed
    Trade-off between ERM loss and gradient variance penalty; tuned on validation sets as reported in Table 6 and Section H.1.
  • β (Hessian matching weight) = 1000 (Waterbirds), 100 (CelebA), 1 (MultiNLI); random search on DomainBed
    Trade-off between ERM loss and Hessian Frobenius penalty; tuned per dataset.
  • penalty annealing iterations = 2100 (Waterbirds), 4000 (CelebA), 0 (MultiNLI)
    Number of updates before gradient and Hessian penalties are activated; selected by grid search.
assumptions (6)
  • domain assumption The target domain μ_T is a convex combination of the source domains (Assumption 2).
    Used in Propositions 2-3 and Theorems 1 and 3. The paper acknowledges in Section 7 that this may not hold or be verifiable in practice.
  • domain assumption There exists an invariant optimal predictor across source domains (Assumption 1, IRM assumption).
    Used in Theorem 1 and Corollary 2. In the linear probing experiments this is enforced by applying the authors' ISR method to the features before training the head.
  • domain assumption Source losses are ν-strongly convex and M-times differentiable with respect to the classifier head.
    Required for the δ = 2δ_S/ν radius and for the Taylor expansion. Cross-entropy is only strongly convex with L2 regularization, which the paper assumes.
  • domain assumption Bounded maximum gradient norm: min_θ max_i ||∇Lμi(θ)||_2 ≤ g (Assumption 3).
    Used only in the second half of Theorem 3 to remove the first-order term. The value of g is not identified in practice.
  • ad hoc to paper The transfer measure admits a midpoint distribution satisfying the equality and inequality in the proof of Proposition 2 (Supplementary A).
    Not stated as an assumption in the main text, but required for the 1/2 max-pairwise bound. The existence and the inequality direction are not proven.
  • standard math The transfer-measure definitions of Zhang et al. (2021) are taken as given.
    The paper's bounds are expressed in this framework. It is prior work by the authors themselves, used as an external starting point.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Moment Alignment: Unifying Gradient and Hessian Matching for Domain Generalization." pith.science (2026). https://pith.science/paper/IDMLOJ3W

@misc{pith2026250607378,
  author       = {Pith},
  title        = {Pith review of: Moment Alignment: Unifying Gradient and Hessian Matching for Domain Generalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IDMLOJ3W}},
  note         = {Machine review of arXiv:2506.07378}
}
read the original abstract

Domain generalization (DG) seeks to develop models that generalize well to unseen target domains, addressing the prevalent issue of distribution shifts in real-world applications. One line of research in DG focuses on aligning domain-level gradients and Hessians to enhance generalization. However, existing methods are computationally inefficient and the underlying principles of these approaches are not well understood. In this paper, we develop the theory of moment alignment for DG. Grounded in \textit{transfer measure}, a principled framework for quantifying generalizability between two domains, we first extend the definition of transfer measure to domain generalization that includes multiple source domains and establish a target error bound. Then, we prove that aligning derivatives across domains improves transfer measure both when the feature extractor induces an invariant optimal predictor across domains and when it does not. Notably, moment alignment provides a unifying understanding of Invariant Risk Minimization, gradient matching, and Hessian matching, three previously disconnected approaches to DG. We further connect feature moments and derivatives of the classifier head, and establish the duality between feature learning and classifier fitting. Building upon our theory, we introduce \textbf{C}losed-Form \textbf{M}oment \textbf{A}lignment (CMA), a novel DG algorithm that aligns domain-level gradients and Hessians in closed-form. Our method overcomes the computational inefficiencies of existing gradient and Hessian-based techniques by eliminating the need for repeated backpropagation or sampling-based Hessian estimation. We validate the efficacy of our approach through two sets of experiments: linear probing and full fine-tuning. CMA demonstrates superior performance in both settings compared to Empirical Risk Minimization and state-of-the-art algorithms.

Figures

Figures reproduced from arXiv: 2506.07378 by the authors.

Figure 1
Figure 1. Hessian Penalty and worst-case accuracy on [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Comparison of first and second-moment differ [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 24 canonical work pages

  1. [1]

    Varshney, and Amit Dhurandhar

    Kartik Ahuja, Karthikeyan Shanmugam, Kush R. Varshney, and Amit Dhurandhar. Invariant Risk Minimization Games , March 2020. URL http://arxiv.org/abs/2002.04692. arXiv:2002.04692 [cs, stat]

    arXiv 2020

  2. [2]

    Invariance Principle Meets Information Bottleneck for Out -of- Distribution Generalization , November 2022 a

    Kartik Ahuja, Ethan Caballero, Dinghuai Zhang, Jean-Christophe Gagnon-Audet, Yoshua Bengio, Ioannis Mitliagkas, and Irina Rish. Invariance Principle Meets Information Bottleneck for Out -of- Distribution Generalization , November 2022 a . URL http://arxiv.org/abs/2106.06607. arXiv:2106.06607 [cs, stat]

    arXiv 2022

  3. [3]

    Empirical or Invariant Risk Minimization? A Sample Complexity Perspective

    Kartik Ahuja, Jun Wang, Amit Dhurandhar, Karthikeyan Shanmugam, and Kush R. Varshney. Empirical or Invariant Risk Minimization ? A Sample Complexity Perspective , August 2022 b . URL http://arxiv.org/abs/2010.16412. arXiv:2010.16412 [cs, stat]

    work page Pith review arXiv 2022

  4. [4]

    AlBadawy, Ashirbani Saha, and Maciej A

    Ehab A. AlBadawy, Ashirbani Saha, and Maciej A. Mazurowski. Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing. Medical Physics, 45 0 (3): 0 1150--1158, March 2018. ISSN 2473-4209. doi:10.1002/mp.12752

    doi:10.1002/mp.12752 2018

  5. [5]

    Invariant Risk Minimization , March 2020

    Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant Risk Minimization , March 2020. URL http://arxiv.org/abs/1907.02893. arXiv:1907.02893 [cs, stat]

    arXiv 2020

  6. [6]

    Recognition in Terra Incognita

    Sara Beery, Grant van Horn, and Pietro Perona. Recognition in Terra Incognita , July 2018. URL http://arxiv.org/abs/1807.04975. arXiv:1807.04975 [cs, q-bio]

    work page Pith review arXiv 2018

  7. [7]

    Bekas, E

    C. Bekas, E. Kokiopoulou, and Y. Saad. An estimator for the diagonal of a matrix. Applied Numerical Mathematics, 57 0 (11): 0 1214--1229, November 2007. ISSN 0168-9274. doi:10.1016/j.apnum.2007.01.003. URL https://www.sciencedirect.com/science/article/pii/S0168927407000244

    doi:10.1016/j.apnum.2007.01.003 2007

  8. [8]

    A theory of learning from different domains

    Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79 0 (1): 0 151--175, May 2010. ISSN 1573-0565. doi:10.1007/s10994-009-5152-4. URL https://doi.org/10.1007/s10994-009-5152-4

    doi:10.1007/s10994-009-5152-4 2010

Show all 64 references
  1. [9]

    Generalizing from Several Related Classification Tasks to a New Unlabeled Sample

    Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from Several Related Classification Tasks to a New Unlabeled Sample . In Advances in Neural Information Processing Systems , volume 24. Curran Associates, Inc., 2011. URL https://papers.nips.cc/paper_files/paper/201...

    2011

  2. [10]

    Chapter 19 - Multiobjective Optimization and Advanced Topics

    Kuang-Hua Chang. Chapter 19 - Multiobjective Optimization and Advanced Topics . In Kuang-Hua Chang, editor, e- Design , pages 1105--1173. Academic Press, Boston, January 2015. ISBN 978-0-12-382038-9. doi:10.1016/B978-0-12-382038-9.00019-3. URL https://www.sciencedirect.com/sci...

    2015 doi

  3. [11]

    Functional Map of the World , April 2018

    Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional Map of the World , April 2018. URL http://arxiv.org/abs/1711.07846. arXiv:1711.07846 [cs]

    2018 arXiv

  4. [12]

    Dark Model Adaptation : Semantic Image Segmentation from Daytime to Nighttime , October 2018

    Dengxin Dai and Luc Van Gool. Dark Model Adaptation : Semantic Image Segmentation from Daytime to Nighttime , October 2018. URL http://arxiv.org/abs/1810.02575. arXiv:1810.02575 [cs]

    2018 arXiv

  5. [13]

    An Image is Worth 16x16 Words : Transformers for Image Recognition at Scale , June 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An Image is Worth 16x16 Words : Transformers for Image Recognition a...

    2021 arXiv

  6. [14]

    Rockmore

    Chen Fang, Ye Xu, and Daniel N. Rockmore. Unbiased Metric Learning : On the Utilization of Multiple Datasets and Web Images for Softening Bias . In 2013 IEEE International Conference on Computer Vision , pages 1657--1664, Sydney, Australia, December 2013. IEEE. ISBN 978-1-4799...

    2013

  7. [15]

    Domain- Adversarial Training of Neural Networks , May 2016

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain- Adversarial Training of Neural Networks , May 2016. URL http://arxiv.org/abs/1505.07818. arXiv:1505.07818 [cs, stat]

    2016 arXiv

  8. [16]

    Bastiaan Kleijn, Mengjie Zhang, and David Balduzzi

    Muhammad Ghifary, W. Bastiaan Kleijn, Mengjie Zhang, and David Balduzzi. Domain Generalization for Object Recognition with Multi -task Autoencoders , August 2015. URL http://arxiv.org/abs/1508.07680. arXiv:1508.07680 [cs, stat]

    2015 arXiv

  9. [17]

    Are Vision Transformers Robust to Spurious Correlations ?, March 2022

    Soumya Suvra Ghosal, Yifei Ming, and Yixuan Li. Are Vision Transformers Robust to Spurious Correlations ?, March 2022. URL http://arxiv.org/abs/2203.09125. arXiv:2203.09125 [cs]

    2022 arXiv

  10. [18]

    In Search of Lost Domain Generalization , July 2020

    Ishaan Gulrajani and David Lopez-Paz. In Search of Lost Domain Generalization , July 2020. URL http://arxiv.org/abs/2007.01434. arXiv:2007.01434 [cs, stat]

    2020 arXiv

  11. [19]

    Bowman, and Noah A

    Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, and Noah A. Smith. Annotation Artifacts in Natural Language Inference Data , April 2018. URL http://arxiv.org/abs/1803.02324. arXiv:1803.02324 [cs]

    2018 arXiv

  12. [20]

    M. C. Hansen, P. V. Potapov, R. Moore, M. Hancher, S. A. Turubanova, A. Tyukavina, D. Thau, S. V. Stehman, S. J. Goetz, T. R. Loveland, A. Kommareddy, A. Egorov, L. Chini, C. O. Justice, and J. R. G. Townshend. High-resolution global maps of 21st-century forest cover change. S...

    2013 doi

  13. [21]

    Invariant Causal Prediction for Nonlinear Models

    Christina Heinze-Deml, Jonas Peters, and Nicolai Meinshausen. Invariant Causal Prediction for Nonlinear Models . Journal of Causal Inference, 6 0 (2): 0 20170016, September 2018. ISSN 2193-3685, 2193-3677. doi:10.1515/jci-2017-0016. URL https://www.degruyter.com/document/doi/1...

    2018 doi

  14. [22]

    Understanding Hessian Alignment for Domain Generalization

    Sobhan Hemati, Guojun Zhang, Amir Estiri, and Xi Chen. Understanding Hessian Alignment for Domain Generalization . In 2023 IEEE / CVF International Conference on Computer Vision ( ICCV ) , pages 18958--18968, Paris, France, October 2023. IEEE. ISBN 9798350307184. doi:10.1109/I...

    2023

  15. [23]

    Efros, and Trevor Darrell

    Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A. Efros, and Trevor Darrell. CyCADA : Cycle - Consistent Adversarial Domain Adaptation , December 2017. URL http://arxiv.org/abs/1711.03213. arXiv:1711.03213 [cs]

    2017 arXiv

  16. [24]

    Does Distributionally Robust Supervised Learning Give Robust Classifiers ?, July 2018

    Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. Does Distributionally Robust Supervised Learning Give Robust Classifiers ?, July 2018. URL http://arxiv.org/abs/1611.02041. arXiv:1611.02041 [stat]

    2018 arXiv

  17. [25]

    Causal-based Time Series Domain Generalization for Vehicle Intention Prediction , December 2021

    Yeping Hu, Xiaogang Jia, Masayoshi Tomizuka, and Wei Zhan. Causal-based Time Series Domain Generalization for Vehicle Intention Prediction , December 2021. URL http://arxiv.org/abs/2112.02093. arXiv:2112.02093 [cs, stat]

    2021 arXiv

  18. [26]

    Winning Prize Comes from Losing Tickets : Improve Invariant Learning by Exploring Variant Parameters for Out -of- Distribution Generalization

    Zhuo Huang, Muyang Li, Li Shen, Jun Yu, Chen Gong, Bo Han, and Tongliang Liu. Winning Prize Comes from Losing Tickets : Improve Invariant Learning by Exploring Variant Parameters for Out -of- Distribution Generalization . International Journal of Computer Vision, 133 0 (1): 0 ...

    2025 doi

  19. [27]

    Sutherland, and Nathan Srebro

    Pritish Kamath, Akilesh Tangella, Danica J. Sutherland, and Nathan Srebro. Does Invariant Risk Minimization Capture Invariance ?, February 2021. URL http://arxiv.org/abs/2101.01134. arXiv:2101.01134 [cs, stat]

    2021 arXiv

  20. [28]

    Out-of- Distribution Generalization with Maximal Invariant Predictor

    Masanori Koyama and Shoichiro Yamaguchi. Out-of- Distribution Generalization with Maximal Invariant Predictor . October 2020. URL https://openreview.net/forum?id=FzGiUKN4aBp

    2020

  21. [29]

    When is invariance useful in an Out -of- Distribution Generalization problem ?, November 2021

    Masanori Koyama and Shoichiro Yamaguchi. When is invariance useful in an Out -of- Distribution Generalization problem ?, November 2021. URL http://arxiv.org/abs/2008.01883. arXiv:2008.01883 [cs, stat]

    2021 arXiv

  22. [30]

    Out-of- Distribution Generalization via Risk Extrapolation ( REx ), February 2021

    David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of- Distribution Generalization via Risk Extrapolation ( REx ), February 2021. URL http://arxiv.org/abs/2003.00688. arXiv:2003.00688 [cs, stat]

    2021 arXiv

  23. [31]

    MNIST handwritten digit database, 2010

    Yann LeCun, Corinna Cortes, and Chris Burges. MNIST handwritten digit database, 2010. URL http://yann.lecun.com/exdb/mnist/

    2010

  24. [32]

    Hospedales

    Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Deeper, Broader and Artier Domain Generalization , October 2017. URL http://arxiv.org/abs/1710.03077. arXiv:1710.03077 [cs]

    2017 arXiv

  25. [33]

    Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C. Kot. Domain Generalization with Adversarial Feature Learning . In 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition , pages 5400--5409, Salt Lake City, UT, June 2018. IEEE. ISBN 978-1-5386-6420-9. doi:1...

    2018

  26. [34]

    Deep Learning Face Attributes in the Wild , September 2015

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep Learning Face Attributes in the Wild , September 2015. URL http://arxiv.org/abs/1411.7766. arXiv:1411.7766 [cs] version: 3

    2015 arXiv

  27. [35]

    Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. Learning Transferable Features with Deep Adaptation Networks , May 2015. URL http://arxiv.org/abs/1502.02791. arXiv:1502.02791 [cs]

    2015 arXiv

  28. [36]

    Domain Generalization via Invariant Feature Representation , January 2013

    Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain Generalization via Invariant Feature Representation , January 2013. URL http://arxiv.org/abs/1301.2115. arXiv:1301.2115 [cs, stat]

    2013 arXiv

  29. [37]

    Learning explanations that are hard to vary, October 2020

    Giambattista Parascandolo, Alexander Neitz, Antonio Orvieto, Luigi Gresele, and Bernhard Schölkopf. Learning explanations that are hard to vary, October 2020. URL http://arxiv.org/abs/2009.00329. arXiv:2009.00329 [cs, stat]

    2020 arXiv

  30. [38]

    Moment Matching for Multi - Source Domain Adaptation , August 2019

    Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment Matching for Multi - Source Domain Adaptation , August 2019. URL http://arxiv.org/abs/1812.01754. arXiv:1812.01754 [cs]

    2019 arXiv

  31. [39]

    Causal inference using invariant prediction: identification and confidence intervals, November 2015

    Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference using invariant prediction: identification and confidence intervals, November 2015. URL http://arxiv.org/abs/1501.01332. arXiv:1501.01332 [stat]

    2015 arXiv

  32. [40]

    Fishr: Invariant Gradient Variances for Out -of- Distribution Generalization

    Alexandre Rame, Corentin Dancette, and Matthieu Cord. Fishr: Invariant Gradient Variances for Out -of- Distribution Generalization . In Proceedings of the 39th International Conference on Machine Learning , pages 18347--18377. PMLR, June 2022. URL https://proceedings.mlr.press...

    2022

  33. [41]

    The Risks of Invariant Risk Minimization , March 2021

    Elan Rosenfeld, Pradeep Ravikumar, and Andrej Risteski. The Risks of Invariant Risk Minimization , March 2021. URL http://arxiv.org/abs/2010.05761. arXiv:2010.05761 [cs, stat]

    2021 arXiv

  34. [42]

    Hashimoto, and Percy Liang

    Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally Robust Neural Networks for Group Shifts : On the Importance of Regularization for Worst - Case Generalization , April 2020. URL http://arxiv.org/abs/1911.08731. arXiv:1911.08731 [cs, stat]

    2020 arXiv

  35. [43]

    BREEDS : Benchmarks for Subpopulation Shift , August 2020

    Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. BREEDS : Benchmarks for Subpopulation Shift , August 2020. URL http://arxiv.org/abs/2008.04859. arXiv:2008.04859 [cs, stat]

    2020 arXiv

  36. [44]

    Do Image Classifiers Generalize Across Time ?, December 2019

    Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, and Ludwig Schmidt. Do Image Classifiers Generalize Across Time ?, December 2019. URL http://arxiv.org/abs/1906.02168. arXiv:1906.02168 [cs, stat]

    2019 arXiv

  37. [45]

    Yuge Shi, Jeffrey Seely, Philip H. S. Torr, N. Siddharth, Awni Hannun, Nicolas Usunier, and Gabriel Synnaeve. Gradient Matching for Domain Generalization , July 2021. URL http://arxiv.org/abs/2104.09937. arXiv:2104.09937 [cs, stat]

    2021 arXiv

  38. [46]

    How to train your ViT ? Data , Augmentation , and Regularization in Vision Transformers , June 2022

    Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your ViT ? Data , Augmentation , and Regularization in Vision Transformers , June 2022. URL http://arxiv.org/abs/2106.10270. arXiv:2106.10270 [cs]

    2022 arXiv

  39. [47]

    Self- Distilled Vision Transformer for Domain Generalization

    Maryam Sultana, Muzammal Naseer, Muhammad Haris Khan, Salman Khan, and Fahad Shahbaz Khan. Self- Distilled Vision Transformer for Domain Generalization . pages 3068--3085, 2022. URL https://openaccess.thecvf.com/content/ACCV2022/html/Sultana_Self-Distilled_Vision_Transformer_f...

    2022

  40. [48]

    Deep CORAL : Correlation Alignment for Deep Domain Adaptation , July 2016

    Baochen Sun and Kate Saenko. Deep CORAL : Correlation Alignment for Deep Domain Adaptation , July 2016. URL http://arxiv.org/abs/1607.01719. arXiv:1607.01719 [cs]

    2016 arXiv

  41. [49]

    Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology

    David Tellez, Geert Litjens, Péter Bándi, Wouter Bulten, John-Melle Bokhorst, Francesco Ciompi, and Jeroen van der Laak. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Medical Image Analy...

    2019

  42. [50]

    Evading the Simplicity Bias : Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization , September 2022

    Damien Teney, Ehsan Abbasnejad, Simon Lucey, and Anton van den Hengel. Evading the Simplicity Bias : Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization , September 2022. URL http://arxiv.org/abs/2105.05612. arXiv:2105.05612 [cs]

    2022 arXiv

  43. [51]

    Adversarial Discriminative Domain Adaptation , February 2017

    Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial Discriminative Domain Adaptation , February 2017. URL http://arxiv.org/abs/1702.05464. arXiv:1702.05464 [cs]

    2017 arXiv

  44. [52]

    An overview of statistical learning theory

    Vladimir N Vapnik. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10 0 (5): 0 988--999, September 1999. ISSN 1941-0093. doi:10.1109/72.788640. URL https://ieeexplore.ieee.org/document/788640. Conference Name: IEEE Transactions on Neural Networks

    1999 doi

  45. [53]

    Detect and correct bias in multi-site neuroimaging datasets

    Christian Wachinger, Anna Rieckmann, Sebastian Pölsterl, and Alzheimer’s Disease Neuroimaging Initiative and the Australian Imaging Biomarkers and Lifestyle flagship study of ageing . Detect and correct bias in multi-site neuroimaging datasets. Medical Image Analysis, 67: 0 10...

    2021

  46. [54]

    The Caltech - UCSD Birds -200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech - UCSD Birds -200-2011 dataset. Technical report, California Institute of Technology, 2011. URL https://paperswithcode.com/dataset/cub-200-2011

    2011

  47. [55]

    Provable Domain Generalization via Invariant - Feature Subspace Recovery , July 2022

    Haoxiang Wang, Haozhe Si, Bo Li, and Han Zhao. Provable Domain Generalization via Invariant - Feature Subspace Recovery , July 2022. URL http://arxiv.org/abs/2201.12919. arXiv:2201.12919 [cs, stat]

    2022 arXiv

  48. [56]

    Invariant- Feature Subspace Recovery : A New Class of Provable Domain Generalization Algorithms , November 2023

    Haoxiang Wang, Gargi Balasubramaniam, Haozhe Si, Bo Li, and Han Zhao. Invariant- Feature Subspace Recovery : A New Class of Provable Domain Generalization Algorithms , November 2023. URL http://arxiv.org/abs/2311.00966. arXiv:2311.00966 [cs, stat]

    2023 arXiv

  49. [57]

    PyTorch Image Models , 2019

    Ross Wightman. PyTorch Image Models , 2019. URL https://github.com/google-research/vision_transformer. original-date: 2020-10-21T12:35:02Z

    2019

  50. [58]

    A Broad - Coverage Challenge Corpus for Sentence Understanding through Inference

    Adina Williams, Nikita Nangia, and Samuel Bowman. A Broad - Coverage Challenge Corpus for Sentence Understanding through Inference . In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for C...

    2018 doi

  51. [59]

    Central Moment Discrepancy ( CMD ) for Domain - Invariant Representation Learning , May 2019

    Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Platz. Central Moment Discrepancy ( CMD ) for Domain - Invariant Representation Learning , May 2019. URL http://arxiv.org/abs/1702.08811. arXiv:1702.08811 [cs, stat]

    2019 arXiv

  52. [60]

    Quantifying and Improving Transferability in Domain Generalization , November 2021

    Guojun Zhang, Han Zhao, Yaoliang Yu, and Pascal Poupart. Quantifying and Improving Transferability in Domain Generalization , November 2021. URL http://arxiv.org/abs/2106.03632. arXiv:2106.03632 [cs, stat]

    2021 arXiv

  53. [61]

    Zhang, Kaican Li, Han Gao, Weiyan Xie, Zhi Lin, Zhenguo Li, Luning Wang, and Yongxiang Huang

    Nevin L. Zhang, Kaican Li, Han Gao, Weiyan Xie, Zhi Lin, Zhenguo Li, Luning Wang, and Yongxiang Huang. A Causal Framework to Unify Common Domain Generalization Approaches , July 2023. URL http://arxiv.org/abs/2307.06825. arXiv:2307.06825 [cs]

    2023 arXiv

  54. [62]

    On Learning Invariant Representations for Domain Adaptation

    Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On Learning Invariant Representations for Domain Adaptation . In Proceedings of the 36th International Conference on Machine Learning , pages 7523--7532. PMLR, May 2019. URL https://proceedings.mlr.press/v97/zha...

    2019

  55. [63]

    Prompt Vision Transformer for Domain Generalization , August 2022

    Zangwei Zheng, Xiangyu Yue, Kai Wang, and Yang You. Prompt Vision Transformer for Domain Generalization , August 2022. URL http://arxiv.org/abs/2208.08914. arXiv:2208.08914 [cs]

    2022 arXiv

  56. [64]

    Places: A 10 Million Image Database for Scene Recognition

    Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 Million Image Database for Scene Recognition . IEEE Transactions on Pattern Analysis and Machine Intelligence, 40 0 (6): 0 1452--1464, June 2018. ISSN 0162-8828, 2160-9292, 1939-3539. do...

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.