Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Consistent flat minima, enforced by refining loss landscapes with soft labels, improve domain generalization by up to 2.6 percentage points over sharpness-aware minimization.

desk verdict Solid empirical DG paper with a genuinely new label-refining mechanism; the theory overclaims transfer, but the experiments stand on their own. read the letter →

arxiv 2412.13573 v2 pith:6DTL5EHR submitted 2024-12-18 cs.CV cs.AI

Aodi Li , Liansheng Zhuang , Xiao Long , Minghong Yao , Shafei Wang This is my paper · ORCID
classification cs.CVcs.AI
keywords domaingeneralizationflatminimalosslandscapesharpness-awareminimizationsoftlabelsself-feedbacktrainingPAC-BayesiananalysisBed
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that for domain generalization, finding a flat minimum in the training loss is not enough: the minimum must be flat in the same way across all training domains, so that it also sits in a flat region of unseen test domains. It proposes Self-Feedback Training (SFT), an iterative loop that measures how much the loss-sharpness values of two training domains disagree, and then trains a small landscape refiner network to generate soft labels that reduce that disagreement while keeping labels correct. Over five DomainBed benchmarks, SFT raises average out-of-domain accuracy to 67.1% with ResNet-50, 2.6 points above sharpness-aware minimization (SAM), and to 77.8% with ViT-B/16, 1.5 points above SAM. The intended upshot is that landscape consistency, not flatness alone, is what transfers to unseen domains.

What carries the argument

The machinery is the self-feedback loop between a model $f_\theta$ and a landscape refiner $g_\phi$. In the feedback phase, two training domains $D_d$ and $D_{d'}$ are picked; the model is updated on $D_d$ with a SAM-style loss using soft labels $\tilde{y} = g_\phi(x)$, and the difference in loss sharpness $|L^{\mathrm{CE-SL}}_{D_d}(\theta,\phi) - L^{\mathrm{CE-SL}}_{D_{d'}}(\theta,\phi)|$ is computed as the inconsistency signal. In the refinement phase, $g_\phi$ is updated to minimize projection cross-entropy (PCE), which projects soft labels onto the label space $\{q_1 \geq \alpha q_k,\ \sum_k q_k = 1\}$ via a fast KL-minimization algorithm, plus penalties on sharpness and on the sharpness difference. This progressively reshapes the loss landscapes so that their flat minima coincide; a PAC-Bayesian bound (Theorem 2 in the supplementary) is used to argue that test-domain sharpness is controlled by training-domain sharpness when domain shifts are drawn independently from a fixed distribution.

What would settle it

Take the best SFT configuration and rerun training with the sharpness-difference term in Eq. (13) set to zero while keeping the soft-label refiner and the SAM step; if out-of-domain accuracy on a test domain whose shift was not present among training domains does not drop, then the consistency term is not the carrier of the reported gains.

Watch

Extended reading notes

Core claim

Flat minima help domain generalization only when they are consistent across domains. SFT's central discovery is that loss landscapes can be actively reshaped during training: a refiner network produces dynamic soft labels, and a feedback signal defined as the absolute difference between SAM sharpness values on two training domains tells the refiner how to change those labels so the landscapes in different domains become both flatter and more alike. The refined landscapes then share a flat basin, and the model trained inside that basin generalizes better out of domain. The paper reports consistent gains over SAM and over other sharpness-aware methods, including GAM, GSAM, FAD, and SAGM, on five DomainBed benchmarks, with the largest improvements on TerraIncognita and PACS.

Load-bearing premise

The theoretical transfer guarantee assumes that the shifts between training domains are independent draws from the same distribution $W$ that also produces the unseen test domains; on the real benchmarks the test domains are fixed, so the bound does not directly cover the evaluated setting.

Editorial extensions

If this is right

  • SFT can be layered on top of sharpness-aware training and improves out-of-domain accuracy by 2.6 points with ResNet-50 and 1.5 points with ViT-B/16 averaged over five DomainBed benchmarks.
  • Because the consistency penalty is separate from the choice of sharpness optimizer, replacing the basic SAM step with stronger variants such as SAGM should give further gains; the paper names this as future work.
  • The projection cross-entropy algorithm converges in at most $N$ steps, where $N$ is the number of classes, and runs about 10 times faster than a generic convex solver, making landscape refinement cheap enough for large-scale models.
  • The theory predicts that reducing sharpness discrepancy among training domains bounds the sharpness on unseen test domains, which is the formal sense in which landscape consistency transfers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the mechanism would use the same soft-label refiner but with a random or sign-flipped feedback signal: if the gain persists, the consistency term itself is not what helps.
  • The refiner is a second network with the same architecture as the model; a smaller or shared refiner could reduce compute while retaining most of the consistency benefit.
  • The feedback signal uses zero-order sharpness; plugging first-order flatness measures, as in GAM or FAD, into the same loop might yield sharper reductions in landscape discrepancy.
  • The consistency-refinement view suggests a new interpretation of soft labels as data-dependent, geometry-aware label smoothing, which connects SFT to knowledge distillation and to label-noise-robust training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Self-Feedback Training (SFT), an iterative two-phase method for domain generalization. In the feedback phase, SFT trains the model on one sampled domain with sharpness-aware minimization under soft labels, then measures landscape inconsistency as the absolute difference of sharpness between two training domains. In the refinement phase, it updates a landscape refiner g_phi using a loss that combines a projection cross-entropy term, a sharpness term, and the sharpness-difference consistency term. The authors claim that this procedure seeks flat minima that are consistent across domains, enabling better transfer to unseen test domains. They support this with a PAC-Bayesian theorem in the supplementary material, a toy experiment visualizing loss surfaces, and extensive DomainBed experiments with ResNet-50, ViT-B/16, and ViT-L/14, reporting average gains of +2.6% over SAM with ResNet-50 and +1.5% with ViT-B/16.

Significance. If the central claim holds, the paper makes a useful contribution: it reframes domain generalization as a problem of refining loss landscapes for consistency, introduces a differentiable soft-label refiner, and provides a complete, efficient algorithm for the projection cross-entropy optimization. The empirical evaluation is broad, covering five DomainBed benchmarks and three backbones, and the reported gains over SAM and its variants are consistent. The paper also ships a detailed reproducibility section and a self-contained derivation of Algorithm 2. However, the theoretical transfer guarantee, which is the main justification for the core claim that landscape consistency transfers to unseen domains, rests on an i.i.d. assumption about domain shifts that does not hold under the DomainBed protocol. The manuscript is therefore technically sound in its empirical parts but has a load-bearing gap in its theoretical argument.

major comments (3)
  1. [Supplementary Section 6.2, Eq. (22)] The transfer step of Theorem 2 is not established under the DomainBed protocol. Eq. (22) applies McAllester's bound with the p-1 held-out training domains D_d' treated as i.i.d. draws from the domain-shift distribution W, and the test domain omega(D_d) as having the same law. In the actual experiments, the test domains are fixed, disjoint datasets, and the held-out domains are the other training domains, not random samples from a distribution over shifts. The proof does not include a term measuring the mismatch between the empirical domain distribution and W, or between W and the true test distribution. Consequently, the conclusion that 'the consistency of flat minima achieved in the training domains can be transferred to unseen test domains' does not follow from the supplied argument. The paper should either prove a bound with an explicit mismatch or distribution-shift term, or substantially weaken the claim to a conditional statement that is clearly satisfied by the experimental protocol.
  2. [Section 3.2.2, Eq. (13) and Theorem 2 condition Eq. (16)] The connection between the optimized objective and the theorem's premise is loose. Theorem 2 assumes that the expected sharpness difference is at most epsilon for all hyper-posterior distributions Q that satisfy Eq. (16). However, the training loss in Eq. (13) penalizes the sharpness difference for a single randomly selected domain pair with a finite weight lambda2, and also includes a separate sharpness term lambda1 that encourages flatness but does not directly enforce consistency. There is no argument that minimizing Eq. (13) drives the expectation in Eq. (16) below epsilon, nor is there a statement about how the finite lambda2 and the optimization trajectory relate to the theorem's uniform condition. The paper should provide an explicit optimization-to-theorem link, or state the condition as an assumption whose verification is left to the experiments.
  3. [Supplementary Section 6.2, Lemma 1 and Eq. (9)] McAllester's bound in Lemma 1 requires a bounded loss taking values in [0,1], but the sharpness quantity used throughout the paper, defined in Eq. (9) as a difference of cross-entropy losses, is not obviously bounded and can exceed one or become negative for soft labels. If the theorem is meant to bound the sharpness of the test loss, the paper must specify the normalization or clipping used to make the sharpness function satisfy the lemma's conditions, or prove that the sharpness difference lies in [0,1] under the stated assumptions. Without this, the PAC-Bayesian estimate terms in Eq. (17) do not formally apply.
minor comments (5)
  1. [Table 10 caption] The caption of Table 10 says 'TerraIncognita' but the table reports DomainNet results; it should read 'DomainNet'.
  2. [Figure 3] The axis labels in Figure 3 are duplicated, and the hyperparameter in the legend appears as 'λ' rather than 'lambda2'; this makes the figure harder to read.
  3. [Algorithm 2, line 5] The expression 'p α 1' in the condition on line 5 is ambiguous in the main text; it should be written as p_1^alpha (or the product) with clear parentheses to avoid confusion with the subscript notation used elsewhere.
  4. [Section 3.2.1] The description of the landscape refiner says it 'shares the same architecture with the model f_theta' but does not specify how its output is normalized or whether it has a separate output head for the N-class soft labels; a short clarification would improve reproducibility.
  5. [Supplementary Section 7.1.1] In the KKT derivation, the notation n and N are both used for the number of classes (e.g., 'j = 2, ..., n' and '(q1, ..., qN)'); the paper should use one symbol consistently.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-referential diagnostic: the reported improvement in landscape consistency is measured by the same sharpness-difference term that SFT's refinement loss minimizes by construction; the central held-out accuracy comparisons remain independent evidence.

  1. self definitional [Section 4.3 (Sharpness Analysis) and Eq. (13) in Section 3.2.2 (Refinement Phase)]
    "SFT quantifies landscape consistency through domain-specific sharpness differences. In Figure 4, we find that SFT reduces domain-specific sharpness values and their differences compared to SAM, while simultaneously attaining higher classification accuracy."

    The 'landscape consistency' used as evidence is defined as the sharpness difference |L_CES-SL_Dd(θ,ϕ) − L_CES-SL_Dd′(θ,ϕ)|, which is exactly the λ2 term in L_Refine (Eq. 13); the sharpness value itself is also penalized via the λ1 term. Therefore, reporting that SFT reduces these quantities compared with SAM largely restates the training objective and is guaranteed by optimization, rather than an independent empirical confirmation of the benefit of consistency. The held-out DG accuracy gains are external evidence and do not reduce to the objective, so this is a minor self-referential step rather than a collapse of the central claim.

full rationale

Apart from the self-referential sharpness diagnostic, the paper's derivation chain is not circular. The SFT objective (Eq. 13) combines a projection cross-entropy term, a sharpness regularizer, and a sharpness-difference penalty; the PCE projection is an exact convex optimization solved by Algorithm 2 with a complete KKT derivation, independent of the generalization claim. The central empirical claim is measured on fixed held-out test domains under the DomainBed protocol, not on the training objective itself, so it is not statistically forced by the fitted hyperparameters. The transfer theorem (Supplementary Theorem 2) is conditional on the explicitly stated assumption that domain shifts ω are i.i.d. from a distribution W; whether the fixed DomainBed domains satisfy that assumption is a validity gap in the proof, not a circular reduction, because the theorem does not define its conclusion into the assumption. The only self-citation (Ref. [34] for adopting VPT) is a protocol choice and is not load-bearing. Overall, the core SFT method and its main accuracy comparisons have independent content; the sole self-referential element is the Section 4.3 'confirmation' of consistency, which is why the score is 2 rather than 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the empirical effectiveness of the refiner and on the theoretical transfer bound. The free parameters are the SAM perturbation radius and the two refinement weights plus the label ratio, all selected on validation data. The theory assumes a distribution over domain shifts and a bounded sharpness measure. The refiner is a learned component introduced by the paper with no independent external validation.

free parameters (4)
  • ρ (SAM perturbation strength) = searched per dataset; 0.3 used in Table 4 ablation
    Controls the radius for sharpness computation in Eqs. 2-4 and 8; affects the magnitude of the sharpness penalty and the feedback signal.
  • λ1 (sharpness weight) = 0.5 in Table 4, searched U[0,1]
    Weights the sharpness term in the refinement loss (Eq. 13); trades off flatness against label correctness.
  • λ2 (consistency weight) = 0.7 in Table 4, searched U[0,1]
    Weights the cross-domain sharpness difference in Eq. 13; directly controls the strength of the consistency feedback.
  • α (label space ratio) = 10^U[0.5,3] in search, i.e., about 3.16 to 1000
    Defines the minimum ratio between the true-class probability and others in the label space C1 (Eq. 10); balances label correctness and softness.
assumptions (4)
  • standard math McAllester's PAC-Bayes bound (Lemma 1)
    Used as the core tool in Theorem 2 to bound expected test sharpness by empirical sharpness plus KL terms; stated in Supplementary Section 6.2.
  • domain assumption Domain shifts are i.i.d. from a distribution W over dataset-to-distribution mappings
    Stated in Supplementary Section 6.1: 'We assume that the domain shifts ω follow a specific distribution W.' This is needed to treat held-out domains as samples from W and to transfer the consistency bound to unseen test domains.
  • domain assumption Sharpness loss ℓ(f,z) is bounded in [0,1]
    Lemma 1 requires a bounded loss, but the sharpness defined in Eq. 9 can be arbitrarily large for cross-entropy with small perturbations; the paper does not discuss truncation or normalization.
  • domain assumption Flatness is a valid proxy for out-of-domain generalization error
    The paper relies on prior work (SAM, SWAD, GAM) for the link between sharpness and generalization rather than deriving it; the theorem bounds sharpness, not accuracy.
invented entities (1)
  • Landscape refiner gφ
    purpose: Generates soft labels that reshape the loss landscape to make flat minima more consistent across domains
    A neural network introduced specifically for SFT; its behavior is only evaluated through the paper's experiments and loss-surface visualizations, with no external benchmark or pretrained checkpoints provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes." pith.science (2026). https://pith.science/paper/6DTL5EHR

@misc{pith2026241213573,
  author       = {Pith},
  title        = {Pith review of: Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6DTL5EHR}},
  note         = {Machine review of arXiv:2412.13573}
}
read the original abstract

Domain generalization aims to learn a model from multiple training domains and generalize it to unseen test domains. Recent theory has shown that seeking the deep models, whose parameters lie in the flat minima of the loss landscape, can significantly reduce the out-of-domain generalization error. However, existing methods often neglect the consistency of loss landscapes in different domains, resulting in models that are not simultaneously in the optimal flat minima in all domains, which limits their generalization ability. To address this issue, this paper proposes an iterative Self-Feedback Training (SFT) framework to seek consistent flat minima that are shared across different domains by progressively refining loss landscapes during training. It alternatively generates a feedback signal by measuring the inconsistency of loss landscapes in different domains and refines these loss landscapes for greater consistency using this feedback signal. Benefiting from the consistency of the flat minima within these refined loss landscapes, our SFT helps achieve better out-of-domain generalization. Extensive experiments on DomainBed demonstrate superior performances of SFT when compared to state-of-the-art sharpness-aware methods and other prevalent DG baselines. On average across five DG benchmarks, SFT surpasses the sharpness-aware minimization by 2.6% with ResNet-50 and 1.5% with ViT-B/16, respectively. The code will be available soon.

Figures

Figures reproduced from arXiv: 2412.13573 by the authors.

Figure 1
Figure 1. Loss Landscapes without and with consistency. The left subplot (a) illustrates the inconsistency of loss landscapes across different domains, which arise due to domain shifts. This paper proposes refining these landscapes to achieve improved con￾sistency, as demonstrated in the right subplot (b). lem, abundant domain generalization (DG) algorithms have been developed from different perspectives, e.g., invariant or c… view at source ↗
Figure 2
Figure 2. 2D visualization of loss surfaces at each domain with/without landscape refinement. The first row shows the in￾consistency of loss surfaces using one-hot labels (without refine￾ment); the second row shows the improved landscape consistency using soft labels generated by the landscape refiner. The final well￾trained model is marked by “+”. Here, “SL” represents the utilization of soft labels. L CE−SL Dd and ¯ϵ CE−SL … view at source ↗
Figure 3
Figure 3. Performances of classification with the varied hyper￾parameter λ. The left (a) and right subplots (b) show the perfor￾mance on the training and test domains, respectively. 4.2. Experiments on Real Dataset 4.2.1. Experimental setting Dataset and protocol. We evaluate our method on five widely-used and challenging datasets for domain general￾ization, including PACS [36] (4 domains, 7 classes, and 9, 991 examples), VLC… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of model sharpness and DG accuracy across different training strategies on the OfficeHome dataset. (where ρ ̸= 0) and label smoothing. When comparing the six experiments in the lower part, we observe that incor￾porating all components of the SFT framework to…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Harmonizing and Merging Source Models for CLIP-based Domain Generalization

    cs.CV 2025-06 conditional novelty 6.0 of 10

    HAM trains per-domain CLIP encoders, enriches them with confident cross-domain samples, aligns their update directions, and merges them with redundancy trimming, reaching 79.0% average accuracy on five DG benchmarks w...

Reference graph

Works this paper leans on

86 extracted references · 51 canonical work pages · cited by 1 Pith paper

  1. [1]

    Invariant risk minimization

    Martin Arjovsky, L ´eon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019. 1, 2, 7

    arXiv 1907

  2. [2]

    Sharpness- aware minimization improves language model generaliza- tion

    Dara Bahri, Hossein Mobahi, and Yi Tay. Sharpness- aware minimization improves language model generaliza- tion. arXiv preprint arXiv:2110.08529, 2021. 2

    arXiv 2021

  3. [3]

    Decaug: Out-of-distribution generalization via decomposed feature representation and semantic augmentation

    Haoyue Bai, Rui Sun, Lanqing Hong, Fengwei Zhou, Nanyang Ye, Han-Jia Ye, S-H Gary Chan, and Zhenguo Li. Decaug: Out-of-distribution generalization via decomposed feature representation and semantic augmentation. In Pro- ceedings of the AAAI Conference on Artificial Intelligence , pages 6705–6713, 2021. 1

    2021

  4. [4]

    Nas-ood: Neural ar- chitecture search for out-of-distribution generalization

    Haoyue Bai, Fengwei Zhou, Lanqing Hong, Nanyang Ye, S-H Gary Chan, and Zhenguo Li. Nas-ood: Neural ar- chitecture search for out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8320–8329, 2021. 2

    2021

  5. [5]

    Metareg: Towards domain generalization using meta- regularization

    Yogesh Balaji, Swami Sankaranarayanan, and Rama Chel- lappa. Metareg: Towards domain generalization using meta- regularization. Advances in neural information processing systems, 31, 2018. 1

    2018

  6. [6]

    Recognition in terra incognita

    Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European confer- ence on computer vision (ECCV), pages 456–473, 2018. 6

    2018

  7. [7]

    Domain generalization by marginal transfer learning

    Gilles Blanchard, Aniket Anand Deshmukh, ¨Urun Dogan, Gyemin Lee, and Clayton Scott. Domain generalization by marginal transfer learning. The Journal of Machine Learn- ing Research, 22(1):46–100, 2021. 7

    2021

  8. [8]

    Ex- ploiting domain-specific features to enhance domain gener- alization

    Manh-Ha Bui, Toan Tran, Anh Tran, and Dinh Phung. Ex- ploiting domain-specific features to enhance domain gener- alization. Advances in Neural Information Processing Sys- tems, 34:21189–21201, 2021. 7

    2021

Show all 86 references
  1. [9]

    Swad: Domain generalization by seeking flat minima

    Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems , 34: 22405–22418, 2021. 2, 3, 7

    2021

  2. [10]

    Domain generalization by mutual-information regularization with pre-trained models

    Junbum Cha, Kyungjae Lee, Sungrae Park, and Sanghyuk Chun. Domain generalization by mutual-information regularization with pre-trained models. arXiv preprint arXiv:2203.10789, 2022. 7

    2022 arXiv

  3. [11]

    Entropy-sgd: Bias- ing gradient descent into wide valleys

    Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-sgd: Bias- ing gradient descent into wide valleys. Journal of Statisti- cal Mechanics: Theory and Experiment , 2019(12...

    2019

  4. [12]

    Sharpness-aware training for free

    Jiawei Du, Daquan Zhou, Jiashi Feng, Vincent Tan, and Joey Tianyi Zhou. Sharpness-aware training for free. Advances in Neural Information Processing Systems , 35: 23439–23451, 2022. 2

    2022

  5. [13]

    Learning to learn with variational information bottleneck for domain general- ization

    Yingjun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees GM Snoek, and Ling Shao. Learning to learn with variational information bottleneck for domain general- ization. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings...

    2020

  6. [14]

    Computing nonvacuous generalization bounds for deep (stochastic) neu- ral networks with many more parameters than training data

    Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neu- ral networks with many more parameters than training data. arXiv preprint arXiv:1703.11008, 2017. 1, 2

    2017 arXiv

  7. [15]

    Unbiased met- ric learning: On the utilization of multiple datasets and web images for softening bias

    Chen Fang, Ye Xu, and Daniel N Rockmore. Unbiased met- ric learning: On the utilization of multiple datasets and web images for softening bias. In Proceedings of the IEEE Inter- national Conference on Computer Vision, pages 1657–1664,

  8. [16]

    Sharpness-aware minimization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412,

    2010 arXiv

  9. [17]

    Domain-adversarial train- ing of neural networks

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pas- cal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial train- ing of neural networks. The journal of machine learning research, 17(1):2096–2030, 2016. 1, 7

    2016

  10. [18]

    Are vi- sion transformers robust to spurious correlations? arXiv preprint arXiv:2203.09125, 2022

    Soumya Suvra Ghosal, Yifei Ming, and Yixuan Li. Are vi- sion transformers robust to spurious correlations? arXiv preprint arXiv:2203.09125, 2022. 2

    2022 arXiv

  11. [19]

    In search of lost domain generalization

    Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint arXiv:2007.01434 ,

    2007 arXiv

  12. [20]

    Simplifying neu- ral nets by discovering flat minima

    Sepp Hochreiter and J ¨urgen Schmidhuber. Simplifying neu- ral nets by discovering flat minima. Advances in neural in- formation processing systems, 7, 1994. 1

    1994

  13. [21]

    Flat minima

    Sepp Hochreiter and J ¨urgen Schmidhuber. Flat minima. Neural computation, 9(1):1–42, 1997. 2

    1997

  14. [22]

    Self-challenging improves cross-domain generalization

    Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. Self-challenging improves cross-domain generalization. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 124–140. Springer, 2020. 7

    2020

  15. [23]

    Averaging weights leads to wider optima and better generalization

    Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv:1803.05407, 2018. 2

    2018 arXiv

  16. [24]

    A single-step, sharpness- aware minimization is all you need to achieve efficient and accurate sparse training

    Jie Ji, Gen Li, Jingjing Fu, Fatemeh Afghah, Linke Guo, Xiaoyong Yuan, and Xiaolong Ma. A single-step, sharpness- aware minimization is all you need to achieve efficient and accurate sparse training. Advances in Neural Information Processing Systems, 37:44269–44290, 2024. 2

    2024

  17. [25]

    Vi- sual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. arXiv preprint arXiv:2203.12119, 2022. 7

    2022 arXiv

  18. [26]

    An adaptive policy to employ sharpness-aware minimization

    Weisen Jiang, Hansi Yang, Yu Zhang, and James Kwok. An adaptive policy to employ sharpness-aware minimization. arXiv preprint arXiv:2304.14647, 2023. 2

    2023 arXiv

  19. [27]

    Fantastic generalization measures and where to find them

    Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. arXiv preprint arXiv:1912.02178, 2019. 1, 2 9

    1912 arXiv

  20. [28]

    When do flat minima optimizers work? Advances in Neural Information Processing Systems , 35:16577–16595,

    Jean Kaddour, Linqing Liu, Ricardo Silva, and Matt J Kus- ner. When do flat minima optimizers work? Advances in Neural Information Processing Systems , 35:16577–16595,

  21. [29]

    Deep learn- ing for NLP and speech recognition

    Uday Kamath, John Liu, and James Whitaker. Deep learn- ing for NLP and speech recognition. Springer, 2019. 1

    2019

  22. [30]

    On large- batch training for deep learning: Generalization gap and sharp minima

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large- batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836, 2016. 1, 2

    2016 arXiv

  23. [31]

    Selfreg: Self-supervised contrastive regu- larization for domain generalization

    Daehee Kim, Youngjun Yoo, Seunghyun Park, Jinkyu Kim, and Jaekoo Lee. Selfreg: Self-supervised contrastive regu- larization for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9619–9628, 2021. 7

    2021

  24. [32]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

    arXiv

  25. [33]

    Out-of-distribution general- ization via risk extrapolation (rex)

    David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution general- ization via risk extrapolation (rex). In International Confer- ence on Machine Learning, pages 5815–5826. PMLR, 2021. 2, 7

    2021

  26. [34]

    Learning common and specific visual prompts for domain generalization

    Aodi Li, Liansheng Zhuang, Shuo Fan, and Shafei Wang. Learning common and specific visual prompts for domain generalization. In Proceedings of the Asian Conference on Computer Vision, pages 4260–4275, 2022. 8

    2022

  27. [35]

    Invariant informa- tion bottleneck for domain generalization

    Bo Li, Yifei Shen, Yezhen Wang, Wenzhen Zhu, Dong- sheng Li, Kurt Keutzer, and Han Zhao. Invariant informa- tion bottleneck for domain generalization. InProceedings of the AAAI Conference on Artificial Intelligence, pages 7399– 7407, 2022. 7

    2022

  28. [36]

    Deeper, broader and artier domain generaliza- tion

    Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generaliza- tion. In Proceedings of the IEEE international conference on computer vision, pages 5542–5550, 2017. 6

    2017

  29. [37]

    Learning to generalize: Meta-learning for do- main generalization

    Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta-learning for do- main generalization. In Proceedings of the AAAI conference on artificial intelligence, 2018. 1, 7

    2018

  30. [38]

    Domain generalization with adversarial feature learning

    Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5400–5409, 2018. 7

    2018

  31. [39]

    Visualizing the loss landscape of neural nets.Ad- vances in neural information processing systems , 31, 2018

    Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets.Ad- vances in neural information processing systems , 31, 2018. 1

    2018

  32. [40]

    Domain generalization via conditional invari- ant representations

    Ya Li, Mingming Gong, Xinmei Tian, Tongliang Liu, and Dacheng Tao. Domain generalization via conditional invari- ant representations. In Proceedings of the AAAI conference on artificial intelligence, 2018. 7

    2018

  33. [41]

    Adapting neural architectures between domains

    Yanxi Li, Zhaohui Yang, Yunhe Wang, and Chang Xu. Adapting neural architectures between domains. Advances in Neural Information Processing Systems , 33:789–798,

  34. [42]

    Internal consistency and self-feedback in large language models: A survey

    Xun Liang, Shichao Song, Zifan Zheng, Hanyu Wang, Qingchen Yu, Xunkai Li, Rong-Hua Li, Feiyu Xiong, and Zhiyu Li. Internal consistency and self-feedback in large language models: A survey. arXiv preprint arXiv:2407.14507, 2024. 2

    2024 arXiv

  35. [43]

    Deep learning applied to nlp

    Marc Moreno Lopez and Jugal Kalita. Deep learning applied to nlp. arXiv preprint arXiv:1703.03091, 2017. 1

    2017 arXiv

  36. [44]

    Self-refine: It- erative refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hal- linan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: It- erative refinement with self-feedback. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  37. [45]

    Training recurrent neural networks by dif- fusion

    Hossein Mobahi. Training recurrent neural networks by dif- fusion. arXiv preprint arXiv:1601.04114, 2016. 1, 2

    2016 arXiv

  38. [46]

    When does label smoothing help? Advances in neural infor- mation processing systems, 32, 2019

    Rafael M ¨uller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in neural infor- mation processing systems, 32, 2019. 2, 5

    2019

  39. [47]

    Reducing domain gap via style- agnostic networks

    Hyeonseob Nam, HyunJae Lee, Jongchan Park, Wonjun Yoon, and Donggeun Yoo. Reducing domain gap via style- agnostic networks. arXiv preprint arXiv:1910.11645, 2(7): 8, 2019. 1, 7

    1910 arXiv

  40. [48]

    Learn- ing explanations that are hard to vary

    Giambattista Parascandolo, Alexander Neitz, Antonio Orvi- eto, Luigi Gresele, and Bernhard Sch ¨olkopf. Learn- ing explanations that are hard to vary. arXiv preprint arXiv:2009.00329, 2020. 3, 7

    2009 arXiv

  41. [49]

    Moment matching for multi-source domain adaptation

    Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 1406–1415,

  42. [50]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  43. [51]

    Distributionally robust neural networks for group shifts: On the importance of regularization for worst- case generalization

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst- case generalization. arXiv preprint arXiv:1911.08731, 2019. 1, 7

    1911 arXiv

  44. [52]

    Gradi- ent matching for domain generalization

    Yuge Shi, Jeffrey Seely, Philip HS Torr, N Siddharth, Awni Hannun, Nicolas Usunier, and Gabriel Synnaeve. Gradi- ent matching for domain generalization. arXiv preprint arXiv:2104.09937, 2021. 7

    2021 arXiv

  45. [53]

    Multi- dataset co-training with sharpness-aware optimization for audio anti-spoofing

    Hye-jin Shim, Jee-weon Jung, and Tomi Kinnunen. Multi- dataset co-training with sharpness-aware optimization for audio anti-spoofing. arXiv preprint arXiv:2305.19953 ,

    arXiv

  46. [54]

    Deep coral: Correlation alignment for deep domain adaptation

    Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European con- ference on computer vision, pages 443–450. Springer, 2016. 7

    2016

  47. [55]

    Statistical learning theory

    Vladimir Vapnik. Statistical learning theory. (No Title),

  48. [56]

    Deep hashing network for 10 unsupervised domain adaptation

    Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for 10 unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 5018–5027, 2017. 6

    2017

  49. [57]

    Generalizing to unseen domains via adversarial data augmentation

    Riccardo V olpi, Hongseok Namkoong, Ozan Sener, John C Duchi, Vittorio Murino, and Silvio Savarese. Generalizing to unseen domains via adversarial data augmentation. Ad- vances in neural information processing systems , 31, 2018. 1

    2018

  50. [58]

    Deep learning for computer vision: A brief review.Computational intelligence and neuroscience, 2018, 2018

    Athanasios V oulodimos, Nikolaos Doulamis, Anastasios Doulamis, and Eftychios Protopapadakis. Deep learning for computer vision: A brief review.Computational intelligence and neuroscience, 2018, 2018. 1

    2018

  51. [59]

    Generalizing to unseen domains: A survey on domain generalization

    Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip Yu. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering, 2022. 1

    2022

  52. [60]

    Deep visual domain adapta- tion: A survey

    Mei Wang and Weihong Deng. Deep visual domain adapta- tion: A survey. Neurocomputing, 312:135–153, 2018. 3

    2018

  53. [61]

    Sharpness-aware gradient matching for domain generaliza- tion

    Pengfei Wang, Zhaoxiang Zhang, Zhen Lei, and Lei Zhang. Sharpness-aware gradient matching for domain generaliza- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 3769–3778,

  54. [62]

    Neural architecture search: Insights from 1000 pa- pers

    Colin White, Mahmoud Safari, Rhea Sukthanker, Binxin Ru, Thomas Elsken, Arber Zela, Debadeepta Dey, and Frank Hutter. Neural architecture search: Insights from 1000 pa- pers. arXiv preprint arXiv:2301.08727, 2023. 2

    2023 arXiv

  55. [63]

    Delving deep into the gener- alization of vision transformers under distribution shifts

    Chongzhi Zhang, Mingyuan Zhang, Shanghang Zhang, Daisheng Jin, Qiang Zhou, Zhongang Cai, Haiyu Zhao, Xi- anglong Liu, and Ziwei Liu. Delving deep into the gener- alization of vision transformers under distribution shifts. In Proceedings of the IEEE/CVF Conference on Computer V...

    2022

  56. [64]

    mixup: Beyond empirical risk minimiza- tion

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. arXiv preprint arXiv:1710.09412, 2017. 1, 7

    2017 arXiv

  57. [65]

    Adaptive risk minimization: A meta-learning approach for tackling group shift

    Marvin Zhang, Henrik Marklund, Abhishek Gupta, Sergey Levine, and Chelsea Finn. Adaptive risk minimization: A meta-learning approach for tackling group shift. arXiv preprint arXiv:2007.02931, 8:9, 2020. 2, 7

    2007 arXiv

  58. [66]

    Deep stable learning for out-of- distribution generalization

    Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjun Zhou, Yue He, and Zheyan Shen. Deep stable learning for out-of- distribution generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5372–5382, 2021. 1

    2021

  59. [67]

    Flatness-aware minimization for domain generalization

    Xingxuan Zhang, Renzhe Xu, Han Yu, Yancheng Dong, Pengfei Tian, and Peng Cui. Flatness-aware minimization for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5189– 5202, 2023. 2, 3, 8

    2023

  60. [68]

    Gradient norm aware minimization seeks first-order flatness and improves generalization

    Xingxuan Zhang, Renzhe Xu, Han Yu, Hao Zou, and Peng Cui. Gradient norm aware minimization seeks first-order flatness and improves generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20247–20257, 2023. 2, 3, 8

    2023

  61. [69]

    Deep learning for environmentally robust speech recognition: An overview of recent developments

    Zixing Zhang, J ¨urgen Geiger, Jouni Pohjalainen, Amr El- Desoky Mousa, Wenyu Jin, and Bj ¨orn Schuller. Deep learning for environmentally robust speech recognition: An overview of recent developments. ACM Transactions on In- telligent Systems and Technology (TIST) , 9(5):1–28...

    2018

  62. [70]

    Prompt vision transformer for domain generalization

    Zangwei Zheng, Xiangyu Yue, Kai Wang, and Yang You. Prompt vision transformer for domain generalization. arXiv preprint arXiv:2208.08914, 2022. 8

    2022 arXiv

  63. [71]

    Domain generalization with mixstyle

    Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xi- ang. Domain generalization with mixstyle. arXiv preprint arXiv:2104.02008, 2021. 7

    2021 arXiv

  64. [72]

    Surrogate gap minimiza- tion improves sharpness-aware training

    Juntang Zhuang, Boqing Gong, Liangzhe Yuan, Yin Cui, Hartwig Adam, Nicha Dvornek, Sekhar Tatikonda, James Duncan, and Ting Liu. Surrogate gap minimiza- tion improves sharpness-aware training. arXiv preprint arXiv:2203.08065, 2022. 2, 8

    2022 arXiv

  65. [73]

    Towards robust out-of- distribution generalization bounds via sharpness

    Yingtian Zou, Kenji Kawaguchi, Yingnan Liu, Jiashuo Liu, Mong-Li Lee, and Wynne Hsu. Towards robust out-of- distribution generalization bounds via sharpness. arXiv preprint arXiv:2403.06392, 2024. 3

    2024 arXiv

  66. [74]

    Pac-bayesian model averaging

    David A McAllester. Pac-bayesian model averaging. In Pro- ceedings of the twelfth annual conference on Computational learning theory, pages 164–170, 1999. 1 11 Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes Supplementary Material L...

    1999

  67. [75]

    For clarity and ease of understanding, we first pro- vide a detailed explanation of the relevant notations and concepts that will be used throughout the analysis

    Theoretical Analysis In this section, we will perform a theoretical analysis of the loss landscape consistency under the PAC-Bayesian frame- work. For clarity and ease of understanding, we first pro- vide a detailed explanation of the relevant notations and concepts that will ...

  68. [76]

    An efficient algorithm (Alogrithm 2) has been presented to address the associated KL divergence minimization problem there

    Projection Cross Entropy As mentioned in the main text, the projection cross entropy (PCE) can be used as a loss term to maintain the label cor- rectness during the refinement phase. An efficient algorithm (Alogrithm 2) has been presented to address the associated KL divergenc...

  69. [77]

    The stationarity conditions require that the partial derivatives of the Lagrangian with respect to each of the variables be zero, which corresponds to the opti- mality condition

    Stationarity. The stationarity conditions require that the partial derivatives of the Lagrangian with respect to each of the variables be zero, which corresponds to the opti- mality condition. For q1, we have the following equa- tion: ∂L ∂q1 = 1 + logq1 − log p1 − NX k=2 µk + ...

  70. [78]

    The primal feasibility condition en- sures that the original constraints are satisfied

    Primal feasibility. The primal feasibility condition en- sures that the original constraints are satisfied. There- fore, we have: αqj − q1 ≤ 0. (32)

  71. [79]

    The dual feasibility condition imposes non-negativity on the Lagrange multipliers associated with the inequality constraints: µj ≥ 0

    Dual feasibility. The dual feasibility condition imposes non-negativity on the Lagrange multipliers associated with the inequality constraints: µj ≥ 0. (33)

  72. [80]

    Finally, the complementary slackness condition relates the primal and dual variables

    Complementary slackness. Finally, the complementary slackness condition relates the primal and dual variables. In this case, the complementary slackness condition for the inequality constraint is: µj(αqj − q1) = 0. (34) This condition implies that eitherµj = 0 or αqj = q1 (or ...

  73. [81]

    The index j1, which corresponds to the largest element in B, must belong to A, i.e., j1 ∈ A. 4

  74. [82]

    , jt−1} ⊆A, if the inequality (pα 1 ( Y j∈C αpj)) 1 |C|+α < αpjt (44) holds, then jt ∈ A

    For any C = {j1, . . . , jt−1} ⊆A, if the inequality (pα 1 ( Y j∈C αpj)) 1 |C|+α < αpjt (44) holds, then jt ∈ A. Otherwise, for all s ∈ {t, t+ 1, . . . ,|B|}, we have js ̸∈ A. That is, A = C. Proof. Firstly, we can prove that j1 ∈ A. To do so, assume for the sake of contradict...

  75. [83]

    These experiments were conducted using ResNet-50, which was pre-trained on ImageNet

    Full Results with ResNet-50 In this section, we demonstrate the full results of our exper- iments that were carried out on five well-known benchmark datasets, including VLCS, PACS, OfficeHome, TerraIncog- nita and DomainNet. These experiments were conducted using ResNet-50, wh...

  76. [84]

    Full Results with ViT-B/16 and ViT-L/14 In this section, we show the full results of visual prompt tuning with the pre-trained large-scale vision transformers (including ViT-B/16 and ViT-L/14). 9.1. VLCS Table 11. Out-of-domain accuracies (%) on each domain of VLCS and their a...

  77. [85]

    Reproducibility To guarantee reproducibility, we will provide an explana- tion of the code and hyperparameters in this section. 10.1. Code Our work is built upon DomainBed, which is released under the MIT license. All experiments are conducted on a single NVIDIA Tesla V100 or ...

  78. [86]

    Broader Impacts This paper primarily focuses on developing an effective do- main generalization method to address the problem of do- main shifts. Given that domain shifts are ubiquitous in real- world applications, this work has the potential to make a positive impact by learn...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.