Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

FedPIA -- Permuting and Integrating Adapters leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FedPIA permutes each client's adapter neurons into a common coordinate system before federated averaging, and reports that this consistently beats prior PEFT-FL baselines and can match or exceed full fine-tuning.

desk verdict A real new entry in PEFT-FL with a broad experimental sweep, but the central Wasserstein-permutation mechanism is not isolated by any control and the full-fine-tuning claim is overreached. read the letter →

arxiv 2412.14424 v1 pith:L3UUFX4H submitted 2024-12-19 cs.CV cs.AIcs.LG

Pramit Saha , Divyanshu Mishra , Felix Wagner , Konstantinos Kamnitsas , J. Alison Noble This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords federatedlearningparameter-efficientfine-tuningadaptersWassersteinbarycenteroptimaltransportvision-languagemodelsmedicalvisualquestionansweringdataheterogeneity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FedPIA is a method for fine-tuning large vision-language models across hospitals and clinics without sharing raw data. The paper claims that separately trained lightweight adapters fail to blend under data, modality, and task heterogeneity because their neurons lack a one-to-one correspondence, so averaging them mixes unrelated features. Its fix is to permute each client adapter's neurons to align with a common reference before aggregating, using layerwise Wasserstein-barycenter matching in the server and again in each client. Across five medical federated settings built from 48 datasets, FedPIA is reported to outperform prior PEFT-FL baselines and, in a task-heterogeneous setting, to exceed full fine-tuning. If true, this makes decentralized fine-tuning of medical foundation models practical without centralized data collection.

What carries the argument

The load-bearing object is the layerwise permutation matrix $P_k^{(\ell)}$, obtained by minimizing the Wasserstein distance $W^{(\ell)}(\mu_k^{(\ell)}, \nu^{(\ell)}, C_k^{(\ell)})$ between probability measures whose supports are adapter neurons. In the server, each neuron's support is its incoming weight row; in the client, the support is the mean activation over a random batch. Equation (1) uses this matrix to align the client adapter weights before integration, and the weighted average of aligned adapters acts as a Wasserstein barycenter. Its role is to break the permutation-invariance barrier that makes naive averaging of independently trained adapters incoherent.

What would settle it

Measure the loss landscape between a client adapter and the global adapter produced by FedPIA: if the linear interpolation barrier is not lower than the barrier with naive averaging, the permutation is not actually landing in a shared basin. A second check is to replace the client's random batch with a fixed public batch; a sharp performance drop would show the alignment depends on incidental input samples rather than stable neuron semantics.

Watch

Extended reading notes

Core claim

The paper's central claim is that the failure of naive adapter averaging in heterogeneous federated learning is largely a permutation-alignment problem. Because each client trains its adapter on different data, modalities, and label pools, corresponding neurons sit at distant positions in weight space, and averaging without alignment mixes unrelated features. FedPIA applies a two-step alignment: in the server it permutes each client adapter toward a FedAvg-initialized global adapter, and in each client it permutes the global adapter toward the local adapter before combining them. The paper reports that this consistently outperforms the adaptive PEFT-FL baselines and can match or surpass full fine-tuning, suggesting that preserving client-specific knowledge through alignment is more effective than diluting updates across all parameters.

Load-bearing premise

The method depends on row-wise Euclidean distances between weight vectors (server) or mean activations over one random batch (client) being a faithful way to decide which neurons correspond across separately trained adapters; if that matching does not track functional correspondence, the permuted average is no better than naive averaging.

Editorial extensions

If this is right

  • Federated PEFT methods that currently average adapters without alignment can add the permutation step as a drop-in module, since the paper shows either server-side or client-side permutation alone beats the no-permutation baseline.
  • The method is orthogonal to the choice of aggregation scheme, so the reported gains should persist when FedAvg is replaced by proximal or control-variate federated optimizers.
  • Because alignment requires no retraining or mutual knowledge distillation, FedPIA adds no training overhead relative to naive adapter averaging, which matters for low-resource medical clients.
  • In the task-heterogeneous setting the reported gains over full fine-tuning imply that aligned adapter aggregation can replace full fine-tuning for federated medical vision-language applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: using a small shared public batch for the client-level activation matching might stabilize the permutation across rounds at no extra privacy cost; the paper's own ablation shows activation-based matching beats weight-based matching, so this is a natural next test.
  • Extension: because the method exploits permutation symmetry of adapter neurons, it should be tested on LoRA-style low-rank factors after expanding them to full rank; permuting unexpanded A/B factors may not share the same symmetry.
  • Extension: the convergence plots suggest FedPIA damps round-to-round loss spikes; quantifying oscillation amplitude would give a metric for comparing alignment methods beyond final accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FedPIA, a federated parameter-efficient fine-tuning framework for vision-language models in medical imaging. On the server, each client adapter is layerwise permuted via optimal transport to match a FedAvg-initialized global adapter before integration, and on each client the global adapter is permuted to match the local adapter (using activation-based costs) before being combined with it. The method is evaluated on five medical vision-language FL tasks -- VQA, image/text disease classification, and a mixed task -- using two VLM backbones and multiple PEFT adapter types, with claims of consistent improvements over PEFT-FL baselines and parity or gains over full fine-tuning.

Significance. If the reported results are robust, the contribution is practically valuable: FedPIA is a simple, communication-friendly add-on to existing PEFT-FL pipelines that requires no extra training overhead and is orthogonal to the aggregation scheme. The experimental scope is unusually broad (48 datasets, two backbones, five task settings), which strengthens the empirical case. However, the paper's central mechanistic claim -- that Wasserstein-based permutation aligns adapters into a common functional basin -- is not isolated by the provided ablations, and the absence of multi-seed results and of reported hyperparameters limits the reproducibility and strength of the headline claims.

major comments (4)
  1. [Methodology, Eq. (1)] Equation (1) defines the aligned weights for layer l in terms of the previous-layer permutation matrix P^(l-1)_k, but the paper never specifies the boundary condition for the first adapter layer. Without an explicit definition of P^(0)_k, the server-side alignment is not fully defined for l=1. Please state the boundary condition (e.g., P^(0)_k = I) and confirm that the normalization diag(1/beta^(l-1)) is applied consistently for that layer.
  2. [Table 5 and Fig. 2(c)] The central mechanistic claim is that the OT-based permutation brings separately trained adapters into the same functional basin so that averaging is coherent. Table 5 only ablates the presence or absence of the entire server-side and client-side PIA blocks; it does not test whether the specific matching matters. A control with independent random permutations per client and a loss-barrier or mode-connectivity measurement between the permuted adapters and the anchor is needed to support the basin-alignment interpretation in Fig. 2(c). Without such a control, the observed gains could also be explained by any permutation-induced smoothing or by the integration step itself.
  3. [Tables 1-5] All reported numbers appear to come from a single run per configuration. Several per-dataset differences are small (e.g., Table 1, VQA-Med 2019 overall: FedPIA (Houlsby) 60.26 vs full fine-tuning 60.43), and there are no error bars, multiple seeds, or significance tests. Given the paper's claim of consistent and robust gains over baselines, at least 3-5 seeds with mean/std, or paired significance tests over clients/tasks, are needed to support that claim.
  4. [Methodology, server-level and client-level PIA] Two hyperparameters are introduced but never reported: gamma in the server integration formula (the weighted average with exp(-gamma ||W~_k - W_G||_2)) and m, the batch size used for the activation supports in client-level PIA. Values or a sensitivity analysis for both are needed; without them the experiments are not fully reproducible.
minor comments (5)
  1. [Training and Implementation Details] In the training details, the ALBEF backbone is cited as '( ?)'; the reference is missing and should be added.
  2. [Methodology title] The method is described as exploiting Wasserstein barycenters, but the actual computation is pairwise optimal-transport matching to an anchor followed by averaging; the relationship to true barycenter computation should be clarified.
  3. [Figure 4] The caption states that 'peaks and troughs represent the losses at the start and end of each communicating round,' but the exact plotted quantity is not described in the text; please clarify what is shown.
  4. [Table 5] The ablation row labeled 'w/ weight-based PIA' is not defined in the main text; the reader must infer that it replaces activation-based costs with weight-based costs in client-level PIA. Please define it explicitly.
  5. [Abstract and Experiments] The abstract claims 'over 2000 client-level experiments,' but the main text does not provide a count or breakdown of how this number is obtained; please include a calculation in the experimental setup.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FedPIA's central claim is evaluated by external task accuracy, not by a self-defined objective.

full rationale

FedPIA constructs the global adapter by initializing it with FedAvg of client adapters, computing layerwise permutation matrices between each client adapter and this anchor using Euclidean ground costs over weight rows, and then averaging the aligned adapters with exponential weights. This is an algorithmic construction, not a fitted parameter renamed as a prediction. The client-side permutation uses mean activations over a randomly selected local batch, which is an unsupervised statistic and does not consume test labels. The reported results are held-out task accuracy and F1 scores on standard medical VQA and disease-detection benchmarks (Tables 1-5), which are external to the permutation objective. The Wasserstein-barycenter machinery is cited from external prior work (Singh and Jaggi 2020; Akash et al. 2022), and the paper's self-citations are background, dataset, or baseline references rather than load-bearing justification for the central permutation-integration step. Concerns about missing random-permutation ablations or loss-barrier measurements bear on experimental support for the mechanistic basin-alignment claim, but they do not make the derivation circular. The central evaluation is self-contained against external benchmarks, so the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method relies on standard OT theory and on several domain-specific matching assumptions. The main free parameters are gamma and the activation batch size m; learning rate and batch size are standard training choices and are not counted as free parameters of the method. No new physical entities are introduced.

free parameters (2)
  • gamma (server integration weight) = not reported
    Appears in the server integration formula fW_G = (1/K) sum fW_k exp(-gamma ||fW_k - W_G||^2). It controls how strongly distant adapters are downweighted. No value or sensitivity analysis is provided, so the method's gains cannot be separated from this choice.
  • m (activation batch size in client PIA) = not reported
    Client-level permutation uses mean activations over a randomly selected batch of m samples. The size m is unspecified and likely affects which neurons are deemed similar between the global and local adapters.
assumptions (4)
  • domain assumption Adapter neurons can be represented as support points using rows of the incoming weight matrix, and Euclidean distance between these rows is a valid ground cost for matching.
    Methodology, server-level PIA section. This is the basis of the permutation cost and is not proven for adapter networks.
  • domain assumption Sequential layerwise permutation of incoming weights, normalized by column marginals, gives a valid alignment between adapters of possibly different layer widths.
    Eq. (1) and surrounding text. Boundary conditions for the first layer are not specified, and the procedure is adopted from OT model fusion without re-derivation.
  • domain assumption Mean activations on a random local batch are a reliable signal for matching global and local adapter neurons.
    Client-level PIA section. This assumption lets the client permute the global adapter toward the local one, but activation statistics from a small batch may be noisy.
  • standard math Wasserstein barycenter theory as presented in Singh and Jaggi 2020 and Akash et al. 2022 applies to adapter weights in this federated setting.
    The paper invokes this theory but does not verify its conditions, such as equal mass or convergence of iterative matching, for adapter networks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedPIA -- Permuting and Integrating Adapters leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learning." pith.science (2026). https://pith.science/paper/L3UUFX4H

@misc{pith2026241214424,
  author       = {Pith},
  title        = {Pith review of: FedPIA -- Permuting and Integrating Adapters leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L3UUFX4H}},
  note         = {Machine review of arXiv:2412.14424}
}
read the original abstract

Large Vision-Language Models typically require large text and image datasets for effective fine-tuning. However, collecting data from various sites, especially in healthcare, is challenging due to strict privacy regulations. An alternative is to fine-tune these models on end-user devices, such as in medical clinics, without sending data to a server. These local clients typically have limited computing power and small datasets, which are not enough for fully fine-tuning large VLMs on their own. A naive solution to these scenarios is to leverage parameter-efficient fine-tuning (PEFT) strategies and apply federated learning (FL) algorithms to combine the learned adapter weights, thereby respecting the resource limitations and data privacy. However, this approach does not fully leverage the knowledge from multiple adapters trained on diverse data distributions and for diverse tasks. The adapters are adversely impacted by data heterogeneity and task heterogeneity across clients resulting in suboptimal convergence. To this end, we propose a novel framework called FedPIA that improves upon the naive combinations of FL and PEFT by introducing Permutation and Integration of the local Adapters in the server and global Adapters in the clients exploiting Wasserstein barycenters for improved blending of client-specific and client-agnostic knowledge. This layerwise permutation helps to bridge the gap in the parameter space of local and global adapters before integration. We conduct over 2000 client-level experiments utilizing 48 medical image datasets across five different medical vision-language FL task settings encompassing visual question answering as well as image and report-based multi-label disease detection. Our experiments involving diverse client settings, ten different modalities, and two VLM backbones demonstrate that FedPIA consistently outperforms the state-of-the-art PEFT-FL baselines.

Figures

Figures reproduced from arXiv: 2412.14424 by the authors.

Figure 1
Figure 1. Sample VQA triplets of 8 modality-specific medical clients [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of FedPIA: (a) shows the permutation of integrated client adapter weights to match the initialized global [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Convergence analysis for Tasks 1 & 3 . C, P, and H refer to Compacter, Parallel adapter, and Houlsby adapter [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Convergence analysis of adapter-based baseline [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. F$^3$OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristics

    cs.CV 2024-11 conditional novelty 6.0 of 10

    F3OCUS combines per-client LNTK layer importance scores with server-side meta-heuristic optimization of layer diversity to improve federated fine-tuning of vision-language models for medical tasks, and releases the 70...

Reference graph

Works this paper leans on

56 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Abacha, A. S. B.; Datla, V. V.; Hasan, S. A.; Demner-Fushman, D.; and M \"u ller, H. 2020. Overview of the VQA-Med Task at ImageCLEF 2020: Visual Question Answering and Generation in the Medical Domain. In CLEF 2020 Working Notes, volume 2696 of CEUR Workshop Proceedings

    work page 2020

  4. [4]

    Acar, D. A. E.; Zhao, Y.; Navarro, R. M.; Mattina, M.; Whatmough, P. N.; and Saligrama, V. 2021. Federated learning based on dynamic regularization. arXiv preprint arXiv:2111.04263

    arXiv 2021

  5. [5]

    K.; Li, S.; and Trillos, N

    Akash, A. K.; Li, S.; and Trillos, N. G. 2022. Wasserstein barycenter-based model fusion and linear mode connectivity of neural networks. arXiv preprint arXiv:2210.06671

    arXiv 2022

  6. [6]

    L.; and Parikh, D

    Antol, S.; Agrawal, A.; Lu, J.; Mitchell, M.; Batra, D.; Zitnick, C. L.; and Parikh, D. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision

    work page 2015

  7. [7]

    X.; and Feizi, S

    Basu, S.; Massiceti, D.; Hu, S. X.; and Feizi, S. 2023. Strong Baselines for Parameter Efficient Few-Shot Fine-tuning. arXiv preprint arXiv:2304.01917

    arXiv 2023

  8. [8]

    A.; Datla, V

    Ben Abacha, A.; Hasan, S. A.; Datla, V. V.; Demner-Fushman, D.; and M \"u ller, H. 2019. Vqa-med: Overview of the medical visual question answering task at imageclef 2019. In Proceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes

    work page 2019

Show all 56 references
  1. [9]

    A.; and M \"u ller, H

    Ben Abacha, A.; Sarrouti, M.; Demner-Fushman, D.; Hasan, S. A.; and M \"u ller, H. 2021. Overview of the vqa-med task at imageclef 2021: Visual question answering and generation in the medical domain. In Proceedings of the CLEF 2021 Conference and Labs of the Evaluation Forum-...

    2021

  2. [10]

    Ben Zaken, E.; Goldberg, Y.; and Ravfogel, S. 2022. B it F it: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models. In Proceedings of the Association for Computational Linguistics (Volume 2: Short Papers), 1--9. Dublin, Ireland

    2022

  3. [11]

    Cai, H.; Gan, C.; Zhu, L.; and Han, S. 2020. TinyTL: Reduce Memory, Not Parameters for Efficient On-Device Learning. In Advances in Neural Information Processing Systems, volume 33, 11285--11297

    2020

  4. [12]

    Chen, H.; Zhang, Y.; Krompass, D.; Gu, J.; and Tresp, V. 2024. Feddat: An approach for foundation model finetuning in multi-modal heterogeneous federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 11285--11293

    2024

  5. [13]

    Chen, J.; Xu, W.; Guo, S.; Wang, J.; Zhang, J.; and Wang, H. 2022. Fedtune: A deep dive into efficient federated fine-tuning with pre-trained transformers. arXiv preprint arXiv:2211.08025

    2022 arXiv

  6. [14]

    J.; and Morcos, A

    Frankle, J.; Schwab, D. J.; and Morcos, A. S. 2021. Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs. arXiv:2003.00152

    2021 arXiv

  7. [15]

    Guo, T.; Guo, S.; and Wang, J. 2023. Pfedprompt: Learning personalized prompt for vision-language models in federated learning. In Proceedings of the ACM Web Conference 2023, 1364--1374

    2023

  8. [16]

    Guo, T.; Guo, S.; Wang, J.; Tang, X.; and Xu, W. 2023. Promptfl: Let federated participants cooperatively learn prompts instead of models-federated learning in age of foundation model. IEEE Transactions on Mobile Computing

    2023

  9. [17]

    He, J.; Zhou, C.; Ma, X.; Berg-Kirkpatrick, T.; and Neubig, G. 2022. Towards a Unified View of Parameter-Efficient Transfer Learning. In International Conference on Learning Representations

    2022

  10. [18]

    Hernandez-Cruz, N.; Saha, P.; Sarker, M. M. K.; and Noble, J. A. 2024. Review of Federated Learning and Machine Learning-Based Methods for Medical Image Analysis. Big Data and Cognitive Computing, 8(9)

    2024

  11. [19]

    Houlsby, N.; Giurgiu, A.; Jastrzebski, S.; Morrone, B.; De Laroussilhe, Q.; Gesmundo, A.; Attariyan, M.; and Gelly, S. 2019. Parameter-efficient transfer learning for NLP. In International conference on machine learning, 2790--2799. PMLR

    2019

  12. [20]

    J.; yelong shen; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; yelong shen; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2022. Lo RA : Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations

    2022

  13. [21]

    Hu, Y.; Li, T.; Lu, Q.; Shao, W.; He, J.; Qiao, Y.; and Luo, P. 2024. OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 22170--22183

    2024

  14. [22]

    Jia, M.; Tang, L.; Chen, B.-C.; Cardie, C.; Belongie, S.; Hariharan, B.; and Lim, S.-N. 2022. Visual Prompt Tuning. In Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXIII, 709–727. Springer-Verlag

    2022

  15. [23]

    Karimi Mahabadi, R.; Henderson, J.; and Ruder, S. 2021. Compacter: Efficient low-rank hypercomplex adapter layers. Advances in Neural Information Processing Systems, 34: 1022--1035

    2021

  16. [24]

    P.; Kale, S.; Mohri, M.; Reddi, S.; Stich, S.; and Suresh, A

    Karimireddy, S. P.; Kale, S.; Mohri, M.; Reddi, S.; Stich, S.; and Suresh, A. T. 2020. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning. PMLR

    2020

  17. [25]

    Kim, W.; Son, B.; and Kim, I. 2021. Vilt: Vision-and-language transformer without convolution or region supervision. In International conference on machine learning, 5583--5594. PMLR

    2021

  18. [26]

    J.; Gayen, S.; Ben Abacha, A.; and Demner-Fushman, D

    Lau, J. J.; Gayen, S.; Ben Abacha, A.; and Demner-Fushman, D. 2018. A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1): 1--10

    2018

  19. [27]

    Lester, B.; Al-Rfou, R.; and Constant, N. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691

    2021 arXiv

  20. [28]

    Li, G.; Wu, W.; Sun, Y.; Shen, L.; Wu, B.; and Tao, D. 2023. Visual prompt based personalized federated learning. arXiv preprint arXiv:2303.08678

    2023 arXiv

  21. [29]

    Li, J.; Selvaraju, R.; Gotmare, A.; Joty, S.; Xiong, C.; and Hoi, S. C. H. 2021. Align before Fuse: Vision and Language Representation Learning with Momentum Distillation. In Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Inf...

    2021

  22. [30]

    Li, Q.; He, B.; and Song, D. 2021. Model-contrastive federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10713--10722

    2021

  23. [31]

    K.; Talwalkar, A.; and Smith, V

    Li, T.; Sahu, A. K.; Talwalkar, A.; and Smith, V. 2020. Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine, 37(3): 50--60

    2020

  24. [32]

    Li, W.-H.; Liu, X.; and Bilen, H. 2022. Cross-domain few-shot learning with task-specific adapters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7161--7170

    2022

  25. [33]

    L.; and Liang, P

    Li, X. L.; and Liang, P. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190

    2021 arXiv

  26. [34]

    Lian, D.; Zhou, D.; Feng, J.; and Wang, X. 2022. Scaling & shifting your features: A new baseline for efficient model tuning. arXiv preprint arXiv:2210.08823

    2022 arXiv

  27. [35]

    Liu, B.; Zhan, L.-M.; Xu, L.; Ma, L.; Yang, Y.; and Wu, X.-M. 2021. Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), 1650--1654. IEEE

    2021

  28. [36]

    Lu, W.; Hu, X.; Wang, J.; and Xie, X. 2023. Fedclip: Fast generalization and personalization for clip in federated learning. arXiv preprint arXiv:2302.13485

    2023 arXiv

  29. [37]

    McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; and y Arcas, B. A. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR

    2017

  30. [38]

    P.; Munoz, J

    Nguyen, D. P.; Munoz, J. P.; and Jannesari, A. 2024. Flora: Enhancing vision-language models with parameter-efficient federated learning. arXiv preprint arXiv:2404.15182

    2024 arXiv

  31. [39]

    Pfeiffer, J.; Kamath, A.; R \"u ckl \'e , A.; Cho, K.; and Gurevych, I. 2020. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247

    2020 arXiv

  32. [40]

    Rebuffi, S.-A.; Bilen, H.; and Vedaldi, A. 2018. Efficient parametrization of multi-domain deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    2018

  33. [41]

    Saha, P.; Mishra, D.; and Noble, J. A. 2023. Rethinking Semi-Supervised Federated Learning: How to co-train fully-labeled and fully-unlabeled client imaging data. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 414--424. Springer

    2023

  34. [42]

    Saha, P.; Mishra, D.; Wagner, F.; Kamnitsas, K.; and Noble, J. A. 2024 a . Examining Modality Incongruity in Multimodal Federated Learning for Medical Vision and Language-based Disease Detection. arXiv:2402.05294

    2024 arXiv

  35. [43]

    Saha, P.; Wagner, F.; Mishra, D.; Peng, C.; Thakur, A.; Clifton, D.; Kamnitsas, K.; and Noble, J. A. 2024 b . F ^3 OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristics. arXiv:2411.11912

    2024 arXiv

  36. [44]

    P.; and Jaggi, M

    Singh, S. P.; and Jaggi, M. 2020. Model fusion via optimal transport. Advances in Neural Information Processing Systems, 33: 22045--22055

    2020

  37. [45]

    Su, S.; Yang, M.; Li, B.; and Xue, X. 2022. Cross-domain federated adaptive prompt tuning for clip. arXiv preprint arXiv:2211.07864, 3

    2022 arXiv

  38. [46]

    Sun, G.; Mendieta, M.; Yang, T.; and Chen, C. 2022. Exploring parameter-efficient fine-tuning for improving communication efficiency in federated learning

    2022

  39. [47]

    Touvron, H.; Cord, M.; El-Nouby, A.; Verbeek, J.; and J \'e gou, H. 2022. Three things everyone should know about vision transformers. In Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXIV. Springer

    2022

  40. [48]

    Wagner, F.; Li, Z.; Saha, P.; and Kamnitsas, K. 2023. Post-Deployment Adaptation with Access to Source Data via Federated Learning and Source-Target Remote Gradient Alignment. In International Workshop on Machine Learning in Medical Imaging, 253--263. Springer

    2023

  41. [49]

    A.; and Kamnitsas, K

    Wagner, F.; Xu, W.; Saha, P.; Liang, Z.; Whitehouse, D.; Menon, D.; Newcombe, V.; Voets, N.; Noble, J. A.; and Kamnitsas, K. 2024. Feasibility of Federated Learning from Client Databases with Different Brain Diseases and MRI Modalities. arXiv:2406.11636

    2024 arXiv

  42. [50]

    Yang, M.; Su, S.; Li, B.; and Xue, X. 2024. Exploring One-Shot Semi-supervised Federated Learning with Pre-trained Diffusion Models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38

    2024

  43. [51]

    Yu, Q.; Liu, Y.; Wang, Y.; Xu, K.; and Liu, J. 2023. Multimodal federated learning via contrastive representation ensemble. arXiv preprint arXiv:2302.08888

    2023 arXiv

  44. [52]

    P.; and Jannesari, A

    Yu, S.; Mu \ n oz, J. P.; and Jannesari, A. 2023. Federated foundation models: Privacy-preserving and collaborative learning for large models. arXiv preprint arXiv:2305.11414

    2023 arXiv

  45. [53]

    Zellers, R.; Bisk, Y.; Farhadi, A.; and Choi, Y. 2019. From Recognition to Cognition: Visual Commonsense Reasoning. arXiv:1811.10830

    2019 arXiv

  46. [54]

    Zeng, H.; Yue, Z.; and Wang, D. 2024. Open-Vocabulary Federated Learning with Multimodal Prototyping. arXiv preprint arXiv:2404.01232

    2024 arXiv

  47. [55]

    Zhang, J.; Vahidian, S.; Kuo, M.; Li, C.; Zhang, R.; Yu, T.; Wang, G.; and Chen, Y. 2024. Towards building the federatedGPT: Federated instruction tuning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE

    2024

  48. [56]

    Zhuang, W.; Chen, C.; and Lyu, L. 2023. When foundation model meets federated learning: Motivations, challenges, and future directions. arXiv preprint arXiv:2306.15546

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.