Pith. sign in

REVIEW 3 major objections 4 minor 62 references

DA-MergeLoRA adapts CLIP to unseen domains by merging per-domain LoRA modules with a meta-learned hypernetwork, achieving state-of-the-art few-shot test-time domain adaptation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:51 UTC pith:X5WNO3AU

load-bearing objection Solid, clearly-written FSTT-DA paper merging source LoRA modules via a meta-learned hypernetwork; works on several benchmarks, but its convex-combination design means generalization to novel shift types is untested. the 3 major comments →

arxiv 2607.17467 v1 pith:X5WNO3AU submitted 2026-07-20 cs.CV cs.LG

DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation

Siobhan Reid , Zhixiang Chi , Li Gu , Omid Reza Heidari , Ziqiang Wang , Yang Wang This is my paper
classification cs.CV cs.LG
keywords test-time domain adaptationfew-shot adaptationLoRAmodel merginghypernetworkmeta-learningCLIPparameter-efficient fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that few-shot test-time domain adaptation can be cast as a parameter-space merging problem: instead of retraining the backbone, you learn small per-source-domain LoRA adapters and then, given a handful of unlabeled target images, a meta-learned hypernetwork produces per-column merge weights that combine those adapters into a single target-specialized model. The authors argue that this approach preserves the base model's generalization while capturing domain-specific features, and they report consistent gains over prior methods on DomainNet and WILDS benchmarks. If correct, this reframes test-time adaptation from shallow updates (batch-norm statistics, prompts) or ensembles to a flexible, internal-weight merging mechanism that can selectively draw on the most relevant source domains.

Core claim

The central claim is that LoRA modules fine-tuned on CLIP's frozen vision encoder for each source domain can be merged at test time, guided by a hypernetwork that generates column-wise merge weights, to produce a single model adapted to an unseen target domain. The paper demonstrates this with a meta-learning scheme: at training time, one source domain is held out as a pseudo-target, its LoRA masked, and the hypernetwork must reconstruct a good merged model from the remaining source LoRAs using only a support batch of the pseudo-target's images; the merged model is then evaluated on a query batch to drive end-to-end optimization. At inference, the hypernetwork receives the actual (unlabeled)

What carries the argument

The core mechanism is a hypernetwork that takes (a) a mean embedding of the target batch (computed with the frozen CLIP encoder, then MLP- and LayerNorm-processed) and (b) the precomputed LoRA ΔW columns from all source domains (each enriched with layer-index, sub-layer-type, and domain tokens), and uses cross-attention to produce per-column, per-domain merge weights. These weights are used to compute a convex combination of the source LoRA columns, yielding a merged ΔW that is added to the frozen CLIP weights for the one-shot adaptation step. The hypernetwork is trained by meta-learning with a pseudo-target domain left out (its LoRA masked, weight fixed to zero) and a cross-entropy loss on

Load-bearing premise

The meta-learning framework implicitly assumes that the distribution shift from source domains to an unseen target domain is similar in kind to the shifts between the source domains used as pseudo-targets, so that a merging policy learned on those pseudo-targets will transfer; if the target domain differs qualitatively (new modality, new label distribution, or a shift not represented among the source shifts), the hypernetwork has no training signal for it.

What would settle it

One concrete test: train the hypernetwork on a set of source domains that are all photographic styles (e.g., different camera conditions), then evaluate on a target domain that is a fundamentally different modality such as sketches or clip-art (random line drawings). If the method fails to improve over zero-shot CLIP on that target, it would indicate that the learned merging policy does not generalize to qualitatively different shift types.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If this works as claimed, test-time adaptation no longer needs to assume the target domain is available during source training; a single small unlabeled batch is enough to synthesize a domain-specific model by recombining existing source knowledge.
  • The approach suggests that the difference between domains is largely encoded in low-rank, column-wise subspaces of the frozen backbone, and that those subspaces can be selectively amplified or suppressed based on a few target samples.
  • Because the merged model is applied once and then used without further updates, the method is compatible with privacy- and latency-constrained deployment where iterative adaptation on target data is forbidden.
  • The gains on FMoW worst-case accuracy and Camelyon17 indicate that the merging policy can be particularly helpful when the target domain is a temporal or hospital shift, where some source years/centers are more relevant than others.
  • The paper's ablation of removing target images shows that the conditional, target-driven aspect of the hypernetwork is essential; a generic trainable query vector performs significantly worse on complex shifts like iWildCam.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not pursue is using the same hypernetwork to merge more than just LoRA adapters, e.g., to merge adapters from different foundation models or to fuse domain-adaptation modules with task-specific modules, possibly enabling continual adaptation across both domain and task shifts.
  • The method's dependence on the pseudo-target scheme suggests that its generalization to a target domain depends on the target shift being 'in the convex hull' of source-domain shifts; one could test this explicitly by constructing synthetic targets from a mix of source shifts and checking whether the merge weights reflect the mix.
  • The per-column merging could be interpreted as a form of soft feature selection; one could probe this by analyzing the merge weights for interpretability (which columns correspond to texture vs. shape features) and verifying that the selected columns align with domain-discriminative directions.
  • The paper leaves open whether the hypernetwork's merging policy transfers to a completely different backbone or to a different task (e.g., semantic segmentation); testing on an out-of-distribution backbone would clarify whether the learned merging policy is CLIP-specific or a more general property of parameter-space adaptation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DA-MergeLoRA, a method for few-shot test-time domain adaptation (FSTT-DA). It trains separate LoRA adapters on a frozen CLIP vision encoder for each source domain, then uses a meta-learned hypernetwork to produce per-column merging weights that combine these LoRA updates into a single target-domain LoRA, conditioned on a small unlabeled batch of target images. The merged model is applied to the backbone and evaluated on the target domain. Experiments on DomainNet and WILDS (iWildCam, Camelyon17, FMoW, PovertyMap) report substantial gains over prior FSTT-DA methods on several benchmarks, with the largest improvements on FMoW worst-case accuracy and DomainNet average accuracy. Ablations analyze the hypernetwork components, merging strategies, and the effect of target conditioning.

Significance. If the results hold, the paper makes a useful contribution by reframing FSTT-DA as parameter-space LoRA merging and demonstrating that a lightweight hypernetwork can combine source-domain adapters effectively. The method is described in enough detail to reimplement, the code is publicly available, and the ablations isolate the importance of target conditioning, positional/domain tokens, and per-column merging. The main value is empirical: strong gains on DomainNet and FMoW. However, the method's generality is bounded by the convex-combination form of the merge (Eq. 7) and by meta-training that only uses source pseudo-targets, so its behavior on qualitatively different shifts (e.g., a new imaging modality or a label distribution shift) is untested. The iWildCam results also do not outperform prior methods on the official F1 metric, so the unqualified SOTA claim is too strong.

major comments (3)
  1. [Eq. (7), Algorithm 1, Sec. 3.3] The merged LoRA update in Eq. (7) is a convex combination of source LoRA columns because the weights are produced by a softmax in Eq. (6). Meta-training (Algorithm 1) samples pseudo-targets only from the set of source domains, so the hypernetwork is never required to approximate a target update lying outside the convex hull of source LoRA columns. All benchmarks involve shifts of a similar nature (style, time, hospital/scanner) and share the label space. The claim that the method generalizes to novel domains (Sec. 5) is therefore limited to shifts representable as mixtures of source updates. I recommend that the authors add an explicit limitations paragraph and temper the scope, or include an experiment with a qualitatively different shift (e.g., synthetic corruptions, new imaging modality, or label-shift-only) to probe this limitation.
  2. [Sec. 3.3, Algorithm 1] The text says 'its merge weight is fixed to zero' for the pseudo-target LoRA during meta-training, but Algorithm 1 only sets the pseudo-target LoRA columns to zero and does not show how the fixed-zero weight interacts with the softmax in Eq. (6) and the summation in Eq. (7). Clarify whether the softmax is over K-1 domains, the weight is masked and the remaining weights renormalized, or the zeroed LoRA simply contributes zero. This is important for reproducibility and for understanding the training distribution.
  3. [Sec. 4.1, Table 1b, Contribution (iii)] The iWildCam results do not support the unqualified SOTA statement. In Table 1b, DA-MergeLoRA's accuracy (73.70 ± 1.49) is statistically tied with L2C (73.4 ± 0.4), and its Macro F1 (33.62) is worse than both L2C (35.2) and MABN (38.3). The abstract and contribution (iii) claim 'state-of-the-art performance across various domain adaptation datasets' and 'achieves SOTA on the DomainNet and WILDS benchmarks'; these should be qualified to 'on several WILDS metrics' or otherwise specify the exceptions, to avoid overclaiming.
minor comments (4)
  1. [Eq. (5), Sec. 3.2] The notation for the domain embedding projection (P_dom q_d) is a bit terse; please specify the dimensions of P_dom and q_d explicitly.
  2. [Sec. 4, Training Details] 'Hypernetwork Models are trained for 1 to 5 epochs' seems to have a capitalization/typographical issue; should read 'are trained'. Consider adding per-dataset epoch counts or a pointer to the supplementary table.
  3. [Sec. 1, Abstract] The abstract's 'state-of-the-art performance across various domain adaptation datasets' is too broad given the iWildCam exception; suggest softening to 'state-of-the-art on several benchmarks'.
  4. [Algorithm 1] The line 'Set pseudo-target LoRA columns to zero: ΔW^(d) ← 0' could be misread as modifying the stored source LoRA; clarify it is a local masking for the episode.

Circularity Check

0 steps flagged

No circularity: the meta-learned hypernetwork is trained on pseudo-target source domains and evaluated on genuinely held-out target domains

full rationale

The paper's derivation chain is an empirical meta-learning pipeline, not a fit masquerading as a prediction. Source-domain LoRA modules are trained independently; the hypernetwork is meta-trained by leaving out one source domain as a pseudo-target (its LoRA columns are zeroed) and is evaluated on target domains excluded from training. Target labels are never used at test time, and the final evaluation is on the official held-out splits of DomainNet and WILDS, so the central claim is externally tested rather than forced by construction. Equation (7) does restrict the adapted model to convex combinations of source LoRA updates, but this is a representational limitation of the method's generalization, not a circular step; the paper itself acknowledges the pseudo-target diversity limitation in the future-work item on 'increasing the diversity of pseudo-target domains during meta-training'. Self-citations to VDPG and L2C are used as baselines and problem-setting references, not as load-bearing justifications for the merging mechanism; the per-column merging choice is attributed to the external LoRA.rar work. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no ansatz is smuggled in via a self-citation. The paper is therefore self-contained against external benchmarks with no significant circularity.

Axiom & Free-Parameter Ledger

7 free parameters · 3 axioms · 1 invented entities

The central claim rests primarily on source LoRA training and a meta-learned hypernetwork. Free parameters (rank, batch size, learning rates) are standard choices with limited sensitivity analysis. The key axioms are (1) mergeability of LoRA modules from a shared backbone, and (2) representativeness of pseudo-target shifts. No genuinely invented entities beyond the hypernetwork itself.

free parameters (7)
  • LoRA rank r = 16
    Chosen heuristically; controls capacity of each domain adapter.
  • LoRA scaling alpha = 32
    Standard LoRA scaling; not tuned per dataset.
  • Cross-attention temperature tau = not specified
    Temperature in Eq. (6) affects softmax sharpness; value not given in main text.
  • Support/query batch size = 16
    Used in meta-training; test-time support size not explicitly stated.
  • Hypernetwork MLP hidden dim = 256
    MLP hidden dimension; standard choice.
  • Learning rates = 3e-4 to 5e-4 (LoRA), 1e-5 to 5e-5 (hypernet)
    Chosen per dataset; not justified.
  • Training epochs = 1-5 (50 for PovertyMap)
    Stopped early; limited ablation on this choice.
axioms (3)
  • domain assumption Source LoRA modules trained from a shared frozen CLIP backbone are compatible for per-column linear merging.
    Relies on prior findings (Task Arithmetic, LoRA.rar) that weight-space interpolation works for models with a shared base; not explicitly validated for these specific adapters and datasets.
  • domain assumption The distribution shift between pseudo-target and other source domains in meta-training is representative of the shift to the real target domain.
    Core to the meta-learning approach; if real target shifts differ in kind, the learned merging policy may not transfer.
  • domain assumption A batch of 16 unlabeled target images contains enough signal to guide the merging policy.
    The method conditions on the mean embedding of the support batch; for class-imbalanced or highly diverse domains (iWildCam, QuickDraw) this signal is weak.
invented entities (1)
  • Hypernetwork for per-column LoRA merging independent evidence
    purpose: Generates per-column merge weights conditioned on target images and source LoRA columns.
    Hypernetworks are a known class of models; the specific architecture here is new but consists of standard components (MLP, cross-attention, tokens). No new physical or mathematical entity is postulated.

pith-pipeline@v1.3.0-alltime-deepseek · 16346 in / 17676 out tokens · 152534 ms · 2026-08-01T17:51:08.288474+00:00 · methodology

0 comments
read the original abstract

Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples. This setting is more realistic than typical domain adaptation setups, which assume access to target data during source training. However, prior FSTT-DA approaches fail to effectively leverage source domain-specific knowledge, relying on shallow batch normalization updates, prompt-based methods that treat the model as a black box, or ensembling strategies that do not capture cross-domain relationships. To address these limitations, we introduce a new FSTT-DA framework that integrates LoRA fine-tuning with model merging. In our approach, separate LoRA modules are fine-tuned on CLIP's vision encoder for each source domain. Since LoRA modifies only a small fraction of the model's parameters, it retains the base model's generalized knowledge while internally learning domain-specific features. To adapt the learned knowledge to a specific target domain, we propose a hypernetwork trained via meta-learning that generates per-column merging factors to combine LoRA modules. Given a small batch of target images, the hypernetwork produces merging weights that fuse source LoRA modules into a single adapted representation. Our results demonstrate state-of-the-art performance across various domain adaptation datasets. Our code is publicly available at https://github.com/nahbois4321/DA-MergeLoRA.

Figures

Figures reproduced from arXiv: 2607.17467 by Li Gu, Omid Reza Heidari, Siobhan Reid, Yang Wang, Zhixiang Chi, Ziqiang Wang.

Figure 1
Figure 1. Figure 1: Overview of our hypernetwork training procedure. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 1
Figure 1. Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Heatmaps of average merge weights. These plots illustrate the average merge weights per-attention-layer produced by the hypernetwork for each of the source domains. The x-axis shows the source domains and the y-axis shows the transformer layer. Plots (a) and (c) show the merge weights before training on FMoW ("Year 2016" target domain) and DomainNet ("Sketch" target domain). Plots (b) and (d) show results … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

62 extracted references · 15 linked inside Pith

  1. [1]

    Nature Mach

    Akiba, T., Shing, M., Tang, Y., Sun, Q., Ha, D.: Evolutionary optimization of model merging recipes. Nature Mach. Intell.7(2), 195–204 (2025)

    2025

  2. [2]

    arXiv preprint arXiv:1907.02893 (2019)

    Arjovsky, M., Bottou, L., Gulrajani, I., Lopez-Paz, D.: Invariant risk minimization. arXiv preprint arXiv:1907.02893 (2019)

    Pith/arXiv arXiv 1907

  3. [3]

    IEEE transactions on medical imaging38(2), 550–560 (2018)

    Bandi, P., Geessink, O., Manson, Q., Van Dijk, M., Balkenhol, M., Hermsen, M., Bejnordi, B.E., Lee, B., Paeng, K., Zhong, A., et al.: From detection of individual metastases to classification of lymph node status at the patient level: the came- lyon17 challenge. IEEE transactions on medical imaging38(2), 550–560 (2018)

    2018

  4. [4]

    arXiv preprint arXiv:2105.03494 (2021)

    Beery, S., Agarwal, A., Cole, E., Birodkar, V.: The iwildcam 2021 competition dataset. arXiv preprint arXiv:2105.03494 (2021)

    Pith/arXiv arXiv 2021

  5. [5]

    Blanchard, G., Lee, G., Scott, C.: Generalizing from several related classification tasks to a new unlabeled sample. Adv. Neural Inform. Process. Syst.24(2011)

    2011

  6. [6]

    Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., Erhan, D.: Domain separation networks. Adv. Neural Inform. Process. Syst.29(2016)

    2016

  7. [7]

    Cha, J., Lee, K., Park, S., Chun, S.: Domain generalization by mutual-information regularization with pre-trained models. In: Eur. Conf. Comput. Vis. pp. 440–457. Springer (2022)

    2022

  8. [8]

    arXiv preprint arXiv:2506.06105 (2025)

    Charakorn, R., Cetin, E., Tang, Y., Lange, R.T.: Text-to-lora: Instant transformer adaption. arXiv preprint arXiv:2506.06105 (2025)

    Pith/arXiv arXiv 2025

  9. [9]

    In: IEEE Conf

    Chen, D., Wang, D., Darrell, T., Ebrahimi, S.: Contrastive test-time adaptation. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 295–305 (2022)

    2022

  10. [10]

    Chi, Z., Gu, L., Liu, H., Wang, Z., Wu, Y., Wang, Y., Plataniotis, K.N.: Learning to adapt frozen clip for few-shot test-time domain adaptation. In: Int. Conf. Learn. Represent. (2025),https://openreview.net/forum?id=TD3SGJfBC7

    2025

  11. [11]

    Chi, Z., Gu, L., Zhong, T., Liu, H., Yu, Y., Plataniotis, K.N., Wang, Y.: Adapting to distribution shift by visual domain prompt generation. In: Int. Conf. Learn. Rep- resent. (2024),https://proceedings.iclr.cc/paper_files/paper/2024/file/ 440f269a4a6b9d51c51b4997963761ff-Paper-Conference.pdf

    2024

  12. [12]

    Chi, Z., Wu, Y., Gu, L., Liu, H., Wang, Z., Zhang, Y., Wang, Y., Plataniotis, K.: Plug-in feedback self-adaptive attention in clip for training-free open-vocabulary segmentation. In: Int. Conf. Comput. Vis. pp. 22815–22825 (2025)

    2025

  13. [13]

    arXiv preprint arXiv:2204.03044 (2022)

    Choshen, L., Venezian, E., Slonim, N., Katz, Y.: Fusing finetuned models for better pretraining. arXiv preprint arXiv:2204.03044 (2022)

    Pith/arXiv arXiv 2022

  14. [14]

    In: IEEE Conf

    Christie, G., Fendley, N., Wilson, J., Mukherjee, R.: Functional map of the world. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 6172–6180 (2018)

    2018

  15. [15]

    Csurka, G., et al.: Domain adaptation in computer vision applications, vol. 2. Springer (2017)

    2017

  16. [16]

    Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Int. Conf. Mach. Learn. pp. 1126–1135. PMLR (2017)

    2017

  17. [17]

    Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., March, M., Lempitsky, V.: Domain-adversarial training of neural networks. J. Mach. Learn. Res.17(59), 1–35 (2016)

    2016

  18. [18]

    In: IEEE Conf

    Gargiulo, A.A., Crisostomi, D., Bucarelli, M.S., Scardapane, S., Silvestri, F., Rodola, E.: Task singular vectors: Reducing task interference in model merging. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 18695–18705 (2025)

    2025

  19. [19]

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. Int. Conf. Learn. Represent.1(2), 3 (2022) DA-MergeLoRA: Hypernetwork-Based LoRA Merging for FSTT-DA 17

    2022

  20. [20]

    In: Conf

    Huang, C., Liu, Q., Lin, B.Y., Pang, T., Du, C., Lin, M.: Lorahub: Efficient cross- task generalization via dynamic lora composition. In: Conf. on Language Modeling (COLM) (2024),https://openreview.net/forum?id=TrloAXEJ2B

    2024

  21. [21]

    arXiv preprint arXiv:2212.04089 (2022)

    Ilharco, G., Ribeiro, M.T., Wortsman, M., Gururangan, S., Schmidt, L., Ha- jishirzi, H., Farhadi, A.: Editing models with task arithmetic. arXiv preprint arXiv:2212.04089 (2022)

    Pith/arXiv arXiv 2022

  22. [22]

    Jiang, L., Ma, R., Gu, L., Wang, Z., Zuo, X., Wang, Y.: Pointmac: Meta-learned adaptation for robust test-time point cloud completion. Adv. Neural Inform. Pro- cess. Syst.38, 44992–45017 (2026)

    2026

  23. [23]

    arXiv preprint arXiv:2212.09849 (2022)

    Jin, X., Ren, X., Preotiuc-Pietro, D., Cheng, P.: Dataless knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09849 (2022)

    Pith/arXiv arXiv 2022

  24. [24]

    Koh, P.W., Sagawa, S., Marklund, H., Xie, S.M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R.L., Gao, I., et al.: Wilds: A benchmark of in-the-wild distribution shifts. In: Int. Conf. Mach. Learn. pp. 5637–5664. PMLR (2021)

    2021

  25. [25]

    arXiv preprint arXiv:2310.11031 (2024),https://arxiv.org/abs/2310.11031

    Lee, G., Jang, W., Kim, J., Jung, J., Kim, S.: Domain generalization using large pretrained models with mixture-of-adapters. arXiv preprint arXiv:2310.11031 (2024),https://arxiv.org/abs/2310.11031

    Pith/arXiv arXiv 2024

  26. [26]

    Li, Y., Gao, V., Zhang, C., Torkamani, M.: Ensembles of low-rank expert adapters. In: Int. Conf. Learn. Represent. (2025),https://openreview.net/forum?id= l0gZS0sAlf

    2025

  27. [27]

    Liang, J., He, R., Tan, T.: A comprehensive survey on test-time adaptation under distribution shifts. Int. J. Comput. Vis.133(1), 31–64 (2025)

    2025

  28. [28]

    Liang, J., Hu, D., Feng, J.: Do we really need to access the source data? source hy- pothesis transfer for unsupervised domain adaptation. In: Int. Conf. Mach. Learn. pp. 6028–6039. PMLR (2020)

    2020

  29. [29]

    In: IEEE Conf

    Liu, H., Wu, Z., Li, L., Salehkalaibar, S., Chen, J., Wang, K.: Towards multi- domain single image dehazing via test-time training. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 5831–5840 (2022)

    2022

  30. [30]

    Liu, J., Yang, S., Jia, P., Zhang, R., Lu, M., Guo, Y., Xue, W., Zhang, S.: Vida: Homeostatic visual domain adapter for continual test time adaptation. In: Int. Conf. Learn. Represent. (2024)

    2024

  31. [31]

    Long, M., Cao, Y., Wang, J., Jordan, M.: Learning transferable features with deep adaptation networks. In: Int. Conf. Mach. Learn. pp. 97–105. PMLR (2015)

    2015

  32. [32]

    arXiv preprint arXiv:1711.05101 (2017)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

    Pith/arXiv arXiv 2017

  33. [33]

    arXiv preprint arXiv:2502.04959 (2025)

    Marczak, D., Magistri, S., Cygert, S., Twardowski, B., Bagdanov, A.D., van de Weijer, J.: No task left behind: Isotropic model merging with common and task- specific subspaces. arXiv preprint arXiv:2502.04959 (2025)

    Pith/arXiv arXiv 2025

  34. [34]

    Matena, M.S., Raffel, C.A.: Merging models with fisher-weighted averaging. Adv. Neural Inform. Process. Syst.35, 17703–17716 (2022)

    2022

  35. [35]

    In: IEEE Conf

    Nam, H., Lee, H., Park, J., Yoon, W., Yoo, D.: Reducing domain gap by reducing style bias. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 8690–8699 (2021)

    2021

  36. [36]

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high- performance deep learning library. Adv. Neural Inform. Process. Syst.32(2019)

    2019

  37. [37]

    Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., Wang, B.: Moment matching for multi-source domain adaptation. In: Int. Conf. Comput. Vis. pp. 1406–1415 (2019)

    2019

  38. [38]

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: Int. Conf. Mach. Learn. pp. 8748–8763. PmLR (2021) 18 S. Reid et al

    2021

  39. [39]

    Shah, V., Ruiz, N., Cole, F., Lu, E., Lazebnik, S., Li, Y., Jampani, V.: Ziplora: Any subject in any style by effectively merging loras. In: Eur. Conf. Comput. Vis. pp. 422–438. Springer (2024)

    2024

  40. [40]

    arXiv preprint arXiv:2508.04153 (2025)

    Shao, Y., Lin, X., Long, X., Chen, S., Yan, M., Liu, Y., Yan, Z., Ma, A., Tang, H., Guo, J.: Icm-fusion: In-context meta-optimized lora fusion for multi-task adapta- tion. arXiv preprint arXiv:2508.04153 (2025)

    Pith/arXiv arXiv 2025

  41. [41]

    rar: Learning to merge loras via hypernetworks for subject-style conditioned image generation

    Shenaj, D., Bohdal, O., Ozay, M., Zanuttigh, P., Michieli, U.: Lora. rar: Learning to merge loras via hypernetworks for subject-style conditioned image generation. In: Int. Conf. Comput. Vis. pp. 16132–16142 (2025)

    2025

  42. [42]

    arXiv preprint arXiv:2507.12297 (2025)

    Shu, Y.C., Lin, Z., Wang, Y.: Regcl: Continual adaptation of segment anything model via model merging. arXiv preprint arXiv:2507.12297 (2025)

    Pith/arXiv arXiv 2025

  43. [43]

    arXiv preprint arXiv:2410.19735 (2024)

    Stoica, G., Ramesh, P., Ecsedi, B., Choshen, L., Hoffman, J.: Model merging with svd to tie the knots. arXiv preprint arXiv:2410.19735 (2024)

    Pith/arXiv arXiv 2024

  44. [44]

    Sun,B.,Saenko,K.:Deepcoral:Correlationalignmentfordeepdomainadaptation. In: Eur. Conf. Comput. Vis. pp. 443–450. Springer (2016)

    2016

  45. [45]

    Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., Hardt, M.: Test-time training with self-supervision for generalization under distribution shifts. In: Int. Conf. Mach. Learn. pp. 9229–9248. PMLR (2020)

    2020

  46. [46]

    Wang, D., Shelhamer, E., Liu, S., Olshausen, B., Darrell, T.: Tent: Fully test-time adaptation by entropy minimization. In: Int. Conf. Learn. Represent. (2021)

    2021

  47. [47]

    IEEE Trans

    Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., Yu, P.: Generalizing to unseen domains: A survey on domain generalization. IEEE Trans. on Knowledge and Data Engineering (2022)

    2022

  48. [48]

    Wang, Z., Chi, Z., Wu, Y., Gu, L., Liu, Z., Plataniotis, K., Wang, Y.: Distribution alignment for fully test-time adaptation with dynamic online data streams. In: Eur. Conf. Comput. Vis. pp. 332–349. Springer (2024)

    2024

  49. [49]

    Wortsman, M., Ilharco, G., Gadre, S.Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A.S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., et al.: Model soups: averagingweightsofmultiplefine-tunedmodelsimprovesaccuracywithoutincreas- ing inference time. In: Int. Conf. Mach. Learn. pp. 23965–23998. PMLR (2022)

    2022

  50. [50]

    In: IEEE Conf

    Wortsman, M., Ilharco, G., Kim, J.W., Li, M., Kornblith, S., Roelofs, R., Lopes, R.G., Hajishirzi, H., Farhadi, A., Namkoong, H., Schmidt, L.: Robust fine-tuning of zero-shot models. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 7959–7971 (June 2022)

    2022

  51. [51]

    arXiv preprint arXiv:2404.13628 (2024)

    Wu, X., Huang, S., Wei, F.: Mixture of lora experts. arXiv preprint arXiv:2404.13628 (2024)

    Pith/arXiv arXiv 2024

  52. [52]

    In: AAAI

    Wu, Y., Chi, Z., Wang, Y., Plataniotis, K.N., Feng, S.: Test-time domain adapta- tion by learning domain-aware batch normalization. In: AAAI. vol. 38, pp. 15961– 15969 (2024)

    2024

  53. [53]

    In: IEEE Conf

    Wu, Y., Yan, Y., Chen, T., Chi, Z., Wu, Z., Jin, Y., Wang, Y., Li, Z.: Talon: Test- time adaptive learning for on-the-fly category discovery. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 22259–22269 (2026)

    2026

  54. [54]

    arXiv preprint arXiv:2501.13198 (2025)

    Wu, Y., Piao, H., Huang, L.K., Wang, R., Li, W., Pfister, H., Meng, D., Ma, K., Wei, Y.: Sd-lora: Scalable decoupled low-rank adaptation for class incremental learning. arXiv preprint arXiv:2501.13198 (2025)

    Pith/arXiv arXiv 2025

  55. [55]

    In: AAAI

    Xu, M., Zhang, J., Ni, B., Li, T., Wang, C., Tian, Q., Zhang, W.: Adversarial domain adaptation with domain mixup. In: AAAI. vol. 34, pp. 6502–6509 (2020)

    2020

  56. [56]

    Yadav, P., Tam, D., Choshen, L., Raffel, C.A., Bansal, M.: Ties-merging: Resolving interference when merging models. Adv. Neural Inform. Process. Syst.36, 7093– 7115 (2023) DA-MergeLoRA: Hypernetwork-Based LoRA Merging for FSTT-DA 19

    2023

  57. [57]

    Yang, E., Wang, Z., Shen, L., Liu, S., Guo, G., Wang, X., Tao, D.: Adamerging: Adaptive model merging for multi-task learning. In: Int. Conf. Learn. Represent. vol. 2024, pp. 22743–22763 (2024)

    2024

  58. [58]

    Nature communications11(1), 2583 (2020)

    Yeh, C., Perez, A., Driscoll, A., Azzari, G., Tang, Z., Lobell, D., Ermon, S., Burke, M.: Using publicly available satellite imagery and deep learning to understand economic well-being in africa. Nature communications11(1), 2583 (2020)

    2020

  59. [59]

    Zhang, M., Marklund, H., Dhawan, N., Gupta, A., Levine, S., Finn, C.: Adaptive risk minimization: Learning to adapt to domain shift. Adv. Neural Inform. Process. Syst.34, 23664–23678 (2021)

    2021

  60. [60]

    arXiv preprint arXiv:2208.08914 (2022)

    Zheng, Z., Yue, X., Wang, K., You, Y.: Prompt vision transformer for domain generalization. arXiv preprint arXiv:2208.08914 (2022)

    Pith/arXiv arXiv 2022

  61. [61]

    Zhong, T., Chi, Z., Gu, L., Wang, Y., Yu, Y., Tang, J.: Meta-dmoe: Adapting to domain shift by meta-distillation from mixture-of-experts. Adv. Neural Inform. Process. Syst.35, 22243–22257 (2022)

    2022

  62. [62]

    IEEE Trans

    Zhou, K., Liu, Z., Qiao, Y., Xiang, T., Loy, C.C.: Domain generalization: A survey. IEEE Trans. Pattern Anal. Mach. Intell.45(4), 4396–4415 (2022)

    2022