Pith. sign in

REVIEW 3 major objections 7 minor 4 cited by

A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning

T0 review · 3 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A survey maps parameter-efficient fine-tuning inside federated learning onto three families—additive, selective, and reparameterized—and identifies a server-side aggregation bias that makes naive LoRA averaging deviate from the ideal…

desk verdict A useful but sloppy survey of PEFT in federated learning; the organization is sound, but the category assignments and citations need a careful pass before it can serve as a dependable reference. read the letter →

arxiv 2504.21099 v1 pith:VIZN2QN4 submitted 2025-04-29 cs.LG cs.AI

Jieming Bian , Yuanzhe Peng , Lei Wang , Yin Huang , Jie Xu This is my paper · ORCID
classification cs.LGcs.AI
keywords federatedlearningparameter-efficientfine-tuninglow-rankadaptationLoRAprompttuningadaptersfoundationmodelsdataheterogeneity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey tries to establish that the scattered literature on parameter-efficient fine-tuning inside federated learning can be organized into a single map. It groups every method it covers into three families—additive (new trainable modules such as adapters and soft prompts), selective (updates only a subset of existing parameters such as bias terms), and reparameterized (low-rank decompositions such as LoRA)—and attaches each method to the federated challenge it solves: data heterogeneity, communication cost, computational limits, or privacy. A sympathetic reader would care because the map turns a pile of method papers into guidance: which PEFT class to pick for which federated bottleneck, and which foundation model and dataset pairs have already been tested. The survey also singles out a concrete technical obstacle unique to federated LoRA: averaging local low-rank updates at the server differs from the ideal average of the full updates, which it calls server-side aggregation bias.

What carries the argument

The load-bearing object is the PEFT taxonomy of [42]—additive, selective, and reparameterized—transplanted into the federated setting. Additive methods insert trainable adapters or soft prompts into a frozen foundation model; selective methods freeze most weights and update only distinguished subsets such as bias terms; reparameterized methods express weight updates as products of low-rank matrices, LoRA being the canonical case. The survey's analytical device is the comparison between the ideal federated update, a weighted sum of local full updates, and what naive LoRA aggregation actually computes, a product of weighted averages of low-rank factors; the mismatch, called server-side aggregation bias, organizes the reparameterized section and motivates the correction methods reviewed there.

What would settle it

Compile the set of federated PEFT methods published before the newest reference in the survey and check whether every one appears in Tables 1–3 under the correct category; one missing or misclassified method that is demonstrably part of the literature would show the comprehensiveness claim to be overstated. A second check: identify a method whose update rule mixes categories (such as FedRA, which the survey itself notes spans selective and additive) and show the taxonomy cannot accommodate it without double-labeling.

Watch

Extended reading notes

Core claim

The paper's central claim is that every current approach to parameter-efficient fine-tuning in federated learning falls into one of three PEFT families and can be reviewed against a standard set of FL challenges. Within additive tuning, adapters and soft prompts are the workhorses, with methods like FedPrompt reducing communication to prompt parameters and adapter methods personalizing per client. Within selective tuning, bias-only updates and parameter-selection strategies provide cheap adaptation. Within reparameterized tuning, LoRA dominates, and the survey identifies a distinctive federated failure mode: because clients send low-rank factors $A_k$ and $B_k$ rather than full updates, the server's weighted average of products is not the product of averages, so naive FedIT aggregation deviates from the ideal global update. Several methods (FFA-LoRA, RoLoRA, FLoRA, FedEx-LoRA, LoRA-FAIR) are reviewed as attempts to remove that bias. On the application side, the survey maps these methods onto NLP tasks (text classification, text generation, machine translation, recommendation) and CV tasks (image classification, domain adaptation, multimodal learning), with recommended datasets and foundation models for each.

Load-bearing premise

The survey's practical value rests on its coverage being complete and representative, yet it provides no search protocol, database list, inclusion criteria, or cutoff date, so any important FL-PEFT method missing from the three categories would undermine the map it offers.

Editorial extensions

If this is right

  • A practitioner choosing a federated fine-tuning method can use the tables to shortlist by the bottleneck they face: adapters and prompts for communication savings, selective bias tuning for extreme resource limits, and LoRA variants for accuracy near full fine-tuning.
  • FedIT-style naive LoRA aggregation is expected to be biased, so any new federated LoRA method should compare against a correction (fixed A, alternating freeze, stacking, or residual term) rather than treating FedIT as an unbiased baseline.
  • The survey's application tables imply that BERT-family models and GLUE tasks are the default testbed for text classification, while CLIP and image-classification datasets are the default testbed for vision, giving new work a ready-made evaluation template.
  • Scaling to trillion-parameter models will require federated PEFT to solve communication and memory bottlenecks even for the small trainable parts, a direction the survey flags as open.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the three-way taxonomy is as exhaustive as the survey claims, then any future FL-PEFT method that cannot be filed as additive, selective, or reparameterized would be a genuinely new category; the survey does not predict what that would look like.
  • The server-side aggregation bias identified for LoRA should in principle afflict any reparameterized method whose local updates are averaged before being composed, so DoRA- or VeRA-style decompositions in federated settings would likely need the same correction machinery; the paper does not draw this extension.
  • A testable next step would be a standardized benchmark that reuses the exact model-dataset pairs in Tables 4 and 5, so that additive, selective, and reparameterized methods could be compared head-to-head under identical non-IID partitions.
  • Because the survey is a snapshot, its map will age quickly; the same taxonomy could be maintained as a living document that tracks new FL-PEFT methods as they appear.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This manuscript is a survey of parameter-efficient fine-tuning (PEFT) methods in federated learning (FL). It organizes the literature into three categories—additive, selective, and reparameterized PEFT—following the taxonomy of Han et al. [42], and it further groups works by application domain (NLP and vision). The survey also reviews challenges in FL (data heterogeneity, communication efficiency, computational constraints, privacy), discusses LoRA-specific issues such as server-side aggregation bias, and outlines future directions. The paper does not report a formal search protocol or inclusion criteria.

Significance. If the survey were accurate and comprehensive, it would be a useful reference map for the rapidly growing FL-PEFT area. The paper covers a broad set of recent methods, and the discussion of LoRA aggregation bias around Eq. (5)-(6) is a concrete technical contribution to the survey's conceptual framing. The application-oriented tables (Tables 4 and 5) also provide practical guidance. However, the manuscript's central claim is that it is a 'systematic and comprehensive review,' and that claim is weakened by internal taxonomy inconsistencies and citation/naming errors. The misclassification of FedRA in particular cuts against the survey's stated organizing principle, and the absence of a search protocol makes the comprehensiveness claim hard to verify.

major comments (3)
  1. [Section 3.2 and Table 2] FedRA [97] is listed under 'Selective PEFT' in Table 2, but the text at the end of Section 3.2 explicitly states that FedRA 'applies adapter-based fine-tuning,' and Section 4.2.1 / Table 5 describe FedRA as 'Selective and Additive.' Since the entire survey is organized around the Additive/Selective/Reparameterized taxonomy, this internal contradiction is load-bearing: a reader consulting Table 2 to identify selective-PEFT methods will be pointed to a method that the paper itself treats as hybrid. The taxonomy either needs a defined hybrid category or the tables and prose must be aligned.
  2. [Section 3.2 and Table 2] DepthFL [51] is included as a selective-PEFT method in Table 2, but the description in Section 3.2 (depth-wise pruning of a global model with mutual self-distillation) does not match the survey's definition of selective PEFT as fine-tuning a subset of the parameters of a frozen pre-trained backbone. The text correctly distinguishes 'traditional federated learning' methods such as CELL and HeteroFL from FL-PEFT methods, yet DepthFL is placed in the FL-PEFT portion and counted in Table 2. This blurs the boundary the survey needs to maintain to support its comprehensive-taxonomy claim. I do not find a comparable problem with CELL and HeteroFL, which the text frames only as background and which do not appear in Table 2.
  3. [Section 1 and Section 3] The abstract and introduction claim a 'systematic and comprehensive review,' but the manuscript does not provide a search protocol, database list, inclusion/exclusion criteria, or coverage cutoff date. Without such information, the reader cannot verify whether the reviewed set of papers is representative or whether omissions are intentional. Given that the central value of a survey is its reliability as a map of the literature, this lack of methodological transparency is a substantive weakness, not merely a presentation issue. The authors should either add a short methodology paragraph or soften the comprehensiveness claim.
minor comments (7)
  1. [Section 3.1.2 and Table 1] In Section 3.1.2, both PromptFL and PromptFolio are cited as reference [86], but Table 1 lists PromptFL as [41]. Please correct the in-text citation for PromptFL.
  2. [Section 3.2, Table 2, and Table 4] The method referenced as 'FedBF [127]' in Section 3.2 and Table 2 is called 'FedPETuning' in Table 4 and in the reference list entry itself. Please unify the method name and the table labels.
  3. [References] References [17] and [37] share the same title ('Prompt-enhanced federated content representation learning for cross-domain recommendation'); these appear to be the arXiv and published versions of the same work. The manuscript should cite one version consistently and distinguish them.
  4. [Table 4] The row for FedBiOT contains the typo 'AReparameterized'; it should read 'Reparameterized.'
  5. [Section 3.3 heading and elsewhere] The spelling 'Reparametrized' is used in the Section 3.3 heading while the rest of the paper uses 'Reparameterized.' Please make the spelling consistent.
  6. [Section 3.1.1 and Table 1] The text in Section 3.1.1 refers to 'ADAFEDSELECKD [32]' while Table 1 and the reference entry use 'ADAFED.' Please align the naming.
  7. [Table 5] The entry 'LLaV A 1.5' should be 'LLaVA 1.5'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey's taxonomy is imported from an external source, and the only author self-citations are descriptive entries that do not carry the derivation.

full rationale

This paper is a literature survey and does not derive any predictive quantity, so the input-equals-output circularity patterns do not apply. The organizing taxonomy is explicitly attributed to an external survey rather than the authors' own prior work: "Following [42], we categorize PEFT techniques into three main approaches: Additive PEFT, Selective PEFT, and Reparameterized PEFT," where [42] is Han et al., an independent survey. The only author self-citations, LoRA-FAIR [10] and FedALT [11], appear as surveyed methods in Section 3.3 and Table 3; their role is descriptive, and the survey's structure or conclusions do not rest on them. The absence of a search protocol and possible taxonomy inconsistencies (e.g., FedAdapter listed as additive, FedRA as selective, and CELL/HeteroFL as selective PEFT) are correctness and completeness concerns, not circularity: they do not show that some output is equivalent to an input by construction. No quoted reduction, fitted parameter renamed as prediction, or self-citation chain forcing the central claim can be exhibited, so the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

No free parameters or invented entities appear because the survey is an exposition of published work. The named concept 'Server-Side Aggregation Bias' restates the algebraic mismatch in FedIT's aggregation, not a new entity. The two assumptions listed are the adopted taxonomy and the unverified completeness of the reviewed set.

assumptions (2)
  • domain assumption The three-way PEFT taxonomy of Han et al. [42] (additive, selective, reparameterized) is the correct organizing scheme for FL-PEFT work.
    Section 2.2 states 'Following [42], we categorize...'; the survey's structure depends on this taxonomy being appropriate in federated settings.
  • ad hoc to paper The reviewed set of papers is a comprehensive and representative sample of FL-PEFT research.
    Section 1 claims 'a systematic and comprehensive review' without specifying search databases, inclusion or exclusion criteria, or date cutoff.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning." pith.science (2026). https://pith.science/paper/VIZN2QN4

@misc{pith2026250421099,
  author       = {Pith},
  title        = {Pith review of: A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VIZN2QN4}},
  note         = {Machine review of arXiv:2504.21099}
}
read the original abstract

Foundation models have revolutionized artificial intelligence by providing robust, versatile architectures pre-trained on large-scale datasets. However, adapting these massive models to specific downstream tasks requires fine-tuning, which can be prohibitively expensive in computational resources. Parameter-Efficient Fine-Tuning (PEFT) methods address this challenge by selectively updating only a small subset of parameters. Meanwhile, Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it ideal for privacy-sensitive applications. This survey provides a comprehensive review of the integration of PEFT techniques within federated learning environments. We systematically categorize existing approaches into three main groups: Additive PEFT (which introduces new trainable parameters), Selective PEFT (which fine-tunes only subsets of existing parameters), and Reparameterized PEFT (which transforms model architectures to enable efficient updates). For each category, we analyze how these methods address the unique challenges of federated settings, including data heterogeneity, communication efficiency, computational constraints, and privacy concerns. We further organize the literature based on application domains, covering both natural language processing and computer vision tasks. Finally, we discuss promising research directions, including scaling to larger foundation models, theoretical analysis of federated PEFT methods, and sustainable approaches for resource-constrained environments.

Figures

Figures reproduced from arXiv: 2504.21099 by the authors.

Figure 1
Figure 1. A high-level roadmap of the survey structure. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An illustration of three primary model fine-tuning paradigms: (1) Local Only, where data [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A schematic comparison of three major kinds of PEFT strategies. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CoCo-Fed: A Unified Framework for Memory- and Communication-Efficient Federated Learning at the Wireless Edge

    cs.IT 2026-01 conditional novelty 6.0 of 10

    CoCo-Fed compresses each client's federated updates into a single fixed-size matrix via double low-rank projection and orthogonal superposition, cutting memory and backhaul traffic while maintaining convergence on an ...

  2. FedCC: A Low-Resource Federated Adaptation of Foundation Models for Robust Corpus Callosum localization in Fetal Ultrasound Images

    cs.LG 2026-06 conditional novelty 5.0 of 10

    Frozen DINOv2 + LoRA + YOLO head, trained with FedAvg, localizes the fetal corpus callosum at mAP@50 0.857 with 8.5x lower communication than full fine-tuning.

  3. FedRS-Bench: Realistic Federated Learning Datasets and Benchmarks in Remote Sensing

    cs.LG 2025-05 conditional novelty 5.0 of 10

    A realistic 135-client federated learning benchmark for remote sensing, assembled from eight datasets, with baseline results for ten FL algorithms.

  4. DP-FedLoRA: Privacy-Enhanced Federated Fine-Tuning for On-Device Large Language Models

    cs.CR 2025-09 reject novelty 3.0 of 10

    DP-FedLoRA clips and adds Gaussian noise to per-client LoRA matrices in federated LLM fine-tuning, claiming unbiased updates and bounded variance, but the privacy calibration and experiments have significant gaps.

Reference graph

Works this paper leans on

132 extracted references · 32 canonical work pages · cited by 4 Pith papers

  1. [86]

    Advances in Neural Information Processing Systems 37, 30590–30623 (2024)

    Pan, B., Huang, W., Shi, Y .: Federated learning from vision-language foundation models: Theoretical analysis and method. Advances in Neural Information Processing Systems 37, 30590–30623 (2024)

    work page 2024

  2. [41]

    IEEE Transactions on Mobile Computing 23(5), 5179–5194 (2023)

    Guo, T., Guo, S., Wang, J., Tang, X., Xu, W.: Promptfl: Let federated participants coopera- tively learn prompts instead of models–federated learning in age of foundation model. IEEE Transactions on Mobile Computing 23(5), 5179–5194 (2023)

    2023

  3. [17]

    Prompt-enhanced Federated Content Representation Learning for Cross-domain Recommendation

    Chen, X., et al.: Prompt-enhanced federated content representation learning for cross-domain recommendation. arXiv preprint arXiv:2401.14678 (2024)

    work page Pith review arXiv 2024

  4. [37]

    In: Proceedings of the ACM Web Conference 2024

    Guo, L., Lu, Z., Yu, J., Nguyen, Q.V .H., Yin, H.: Prompt-enhanced federated content rep- resentation learning for cross-domain recommendation. In: Proceedings of the ACM Web Conference 2024. pp. 3139–3149 (2024)

    2024

  5. [42]

    arXiv preprint arXiv:2403.14608 (2024)

    Han, Z., Gao, C., Liu, J., Zhang, J., Zhang, S.Q.: Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608 (2024)

    arXiv 2024

  6. [97]

    In: European Conference on Computer Vision

    Su, S., Li, B., Xue, X.: Fedra: A random allocation strategy for federated tuning to unleash the power of heterogeneous clients. In: European Conference on Computer Vision. pp. 342–358. Springer (2024)

    work page 2024

  7. [51]

    In: The Eleventh International Conference on Learning Representations (2023)

    Kim, M., Yu, S., Kim, S., Moon, S.M.: Depthfl: Depthwise federated learning for hetero- geneous clients. In: The Eleventh International Conference on Learning Representations (2023)

    2023

  8. [1]

    arXiv preprint arXiv:2303.08774 (2023)

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

Show all 132 references
  1. [2]

    In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL)

    Adelani, D.I., Alabi, J., Dossou, B.F.P., et al.: Masakhanews: A multilingual news corpus for african languages. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL). pp. 10942–10959 (2023)

    2023

  2. [3]

    In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics

    Aharoni, R., Johnson, M., Firat, O.: Massively multilingual neural machine translation. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. pp. 3874–3884 (2019) 21

    2019

  3. [4]

    arXiv preprint arXiv:2305.10403 (2023), https://arxiv.org/abs/2305.10403

    Anil, R., Lee, S., et al.: Palm 2 technical report. arXiv preprint arXiv:2305.10403 (2023), https://arxiv.org/abs/2305.10403

    2023 arXiv

  4. [5]

    arXiv preprint arXiv:2308.06522 (2023)

    Babakniya, S., Elkordy, A.R., Ezzeldin, Y .H., Liu, Q., Song, K.B., El-Khamy, M., Avestimehr, S.: Slora: Federated parameter efficient fine-tuning of language models. arXiv preprint arXiv:2308.06522 (2023)

    2023 arXiv

  5. [6]

    arXiv preprint arXiv:2402.11505 (2024)

    Bai, J., Chen, D., Qian, B., Yao, L., Li, Y .: Federated fine-tuning of large language models under heterogeneous tasks and client resources. arXiv preprint arXiv:2402.11505 (2024)

    2024 arXiv

  6. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Bai, S., Zhang, J., Guo, S., Li, S., Guo, J., Hou, J., Han, T., Lu, X.: Diprompt: Disentangled prompt tuning for multiple latent domain generalization in federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 27284–27293 (2024)

    2024

  7. [8]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume 2: Short Papers)

    Ben Zaken, E., Ravfogel, S., Goldberg, Y .: Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume 2: Short Papers). pp. 1–9 (2022)

    2022

  8. [9]

    In: Proceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems

    Bian, J., Wang, L., Ren, S., Xu, J.: Cafe: Carbon-aware federated learning in geographically distributed data centers. In: Proceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems. pp. 347–360 (2024)

    2024

  9. [10]

    arXiv preprint arXiv:2411.14961 (2024)

    Bian, J., Wang, L., Zhang, L., Xu, J.: Lora-fair: Federated lora fine-tuning with aggregation and initialization refinement. arXiv preprint arXiv:2411.14961 (2024)

    2024

  10. [11]

    arXiv preprint arXiv:2503.11880 (2025)

    Bian, J., Wang, L., Zhang, L., Xu, J.: Fedalt: Federated fine-tuning through adaptive local training with rest-of-the-world lora. arXiv preprint arXiv:2503.11880 (2025)

    2025

  11. [12]

    In: European Conference on Computer Vision (2014)

    Bossard, L., Guillaumin, M., Van Gool, L.: Food-101 – mining discriminative components with random forests. In: European Conference on Computer Vision (2014)

    2014

  12. [13]

    arXiv preprint arXiv:2205.10162 (2022)

    Cai, D., Wu, Y ., Wang, S., Lin, F.X., Xu, M.: Fedadapter: Efficient federated learning for modern nlp. arXiv preprint arXiv:2205.10162 (2022)

    2022 arXiv

  13. [14]

    Patterns 5(12) (2024)

    Chen, C., Feng, X., Li, Y ., Lyu, L., Zhou, J., Zheng, X., Yin, J.: Integration of large language models and federated learning. Patterns 5(12) (2024)

    2024

  14. [15]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Chen, H., Zhang, Y ., Krompass, D., Gu, J., Tresp, V .: Feddat: An approach for foundation model finetuning in multi-modal heterogeneous federated learning. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 11285–11293 (2024)

    2024

  15. [16]

    arXiv preprint arXiv:2502.01755 (2025)

    Chen, S., Guo, Y ., Ju, Y ., Dalal, H., Khisti, A.: Robust federated finetuning of llms via alternating optimization of lora. arXiv preprint arXiv:2502.01755 (2025)

    2025

  16. [18]

    See https://vicuna

    Chiang, W.L., Li, Z., Lin, Z., Sheng, Y ., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y ., Gonzalez, J.E., et al.: Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023) 2(3), 6 (2023)

    2023

  17. [19]

    arXiv preprint arXiv:2401.06432 (2024)

    Cho, Y .J., Liu, L., Xu, Z., Fahrezi, A., Joshi, G.: Heterogeneous lora for federated fine-tuning of on-device foundation models. arXiv preprint arXiv:2401.06432 (2024)

    2024 arXiv

  18. [20]

    arXiv preprint arXiv:2208.09021 6, 17 (2022)

    Chochlakis, G., Srinivasan, T., Thomason, J., Narayanan, S.: Vault: Augmenting the vision- and-language transformer with the propagation of deep language representations. arXiv preprint arXiv:2208.09021 6, 17 (2022)

    2022 arXiv

  19. [21]

    arXiv preprint arXiv:2110.14168 (2021), https://arxiv.org/abs/2110.14168 22

    Cobbe, K., Kosaraju, V ., Kale, M., et al.: Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021), https://arxiv.org/abs/2110.14168 22

    2021 arXiv

  20. [22]

    In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL)

    Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V ., Wenzek, G., Guzmán, F., Grave, É., Ott, M., Zettlemoyer, L., Stoyanov, V .: Unsupervised cross-lingual representation learning at scale. In: Proceedings of the 58th Annual Meeting of the Association for Computational Ling...

    2020

  21. [23]

    In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    Conneau, A., Rinott, R., Lample, G., Williams, A., Bowman, S.R., Schwenk, H., Stoyanov, V .: Xnli: Evaluating cross-lingual sentence representations. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 2475–2485 (2018)

    2018

  22. [24]

    Proceedings of the 2nd International Workshop on Compiler Compilers (1992), https://dl

    Cordy, J., Hall, R., Carmichael, I., et al.: Rosetta: a generator of data language compilers. Proceedings of the 2nd International Workshop on Compiler Compilers (1992), https://dl. acm.org/doi/10.1145/258368.258412

    1992

  23. [25]

    In: Machine learning challenges workshop

    Dagan, I., Glickman, O., Magnini, B.: The pascal recognising textual entailment challenge. In: Machine learning challenges workshop. pp. 177–190. Springer (2005)

    2005

  24. [26]

    Databricks Blog (2023), https://www.databricks.com/blog/2023/04/12/ dolly-first-open-commercially-viable-instruction-tuned-llm

    Databricks: Dolly: Open instruction-tuned llm. Databricks Blog (2023), https://www.databricks.com/blog/2023/04/12/ dolly-first-open-commercially-viable-instruction-tuned-llm

    2023

  25. [27]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Deng, W., Thrampoulidis, C., Li, X.: Unlocking the potential of prompt-tuning in bridging generalized and personalized federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6087–6097 (2024)

    2024

  26. [28]

    In: Proceedings of NAACL-HLT

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT. pp. 4171–4186 (2019)

    2019

  27. [29]

    arXiv preprint arXiv:2010.01264 (2020)

    Diao, E., Ding, J., Tarokh, V .: Heterofl: Computation and communication efficient federated learning for heterogeneous clients. arXiv preprint arXiv:2010.01264 (2020)

    2020 arXiv

  28. [30]

    In: International Conference on Learning Representations (ICLR) (2021)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., De- hghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on...

    2021

  29. [31]

    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Tech. rep., California Institute of Technology (2004)

    2004

  30. [32]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing (2024)

    Feng, X., Feng, X., Du, X., Kan, M.Y ., Qin, B.: Adapter-based selective knowledge distilla- tion for federated multi-domain meeting summarization. IEEE/ACM Transactions on Audio, Speech, and Language Processing (2024)

    2024

  31. [33]

    AI 5(4), 2773–2800 (2024)

    Feretzakis, G., Verykios, V .S.: Trustworthy ai: Securing sensitive data in large language models. AI 5(4), 2773–2800 (2024)

    2024

  32. [34]

    for Research on Foundation Models, S.C.: Alpaca: A strong, replicable instruction-following model (2023), https://crfm.stanford.edu/2023/03/13/alpaca.html

    2023

  33. [35]

    CS224N project report, Stanford 1(12), 2009 (2009)

    Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. CS224N project report, Stanford 1(12), 2009 (2009)

    2009

  34. [36]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2012)

    Gong, B., Shi, Y ., Sha, F., Grauman, K.: Geodesic flow kernel for unsupervised domain adap- tation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2012)

    2012

  35. [38]

    arXiv preprint arXiv:2012.07463 (2021) 23

    Guo, M., Rush, A.M.: Parameter-efficient transfer learning with diff pruning. arXiv preprint arXiv:2012.07463 (2021) 23

    2021 arXiv

  36. [39]

    arXiv preprint arXiv:2410.01463 (2024)

    Guo, P., Zeng, S., Wang, Y ., Fan, H., Wang, F., Qu, L.: Selective aggregation for low-rank adaptation in federated learning. arXiv preprint arXiv:2410.01463 (2024)

    2024 arXiv

  37. [40]

    In: Proceedings of the ACM Web Conference 2023

    Guo, T., Guo, S., Wang, J.: Pfedprompt: Learning personalized prompt for vision-language models in federated learning. In: Proceedings of the ACM Web Conference 2023. pp. 1364– 1374 (2023)

    2023

  38. [43]

    arXiv preprint arXiv:2503.03920 (2025)

    Hao, J., Wu, Y ., Payani, A., Lee, M., Liu, M.: Personalized federated fine-tuning for het- erogeneous data: An automatic rank learning approach via two-level lora. arXiv preprint arXiv:2503.03920 (2025)

    2025 arXiv

  39. [44]

    In: Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

    2016

  40. [45]

    In: Proceedings of the 9th International Conference on Learning Representations (ICLR) (2021)

    He, P., Liu, X., Gao, J., Chen, W.: Deberta: Decoding-enhanced bert with disentangled attention. In: Proceedings of the 9th International Conference on Learning Representations (ICLR) (2021)

    2021

  41. [46]

    In: Proceedings of the 36th International Conference on Machine Learning

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, S., Gelly, S.: Parameter-efficient transfer learning for nlp. In: Proceedings of the 36th International Conference on Machine Learning. pp. 2790–2799. PMLR (2019)

    2019

  42. [47]

    ICLR 1(2), 3 (2022)

    Hu, E.J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. ICLR 1(2), 3 (2022)

    2022

  43. [48]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)

    Hudson, D.A., Manning, C.D.: Gqa: A new dataset for real-world visual reasoning and compositional question answering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)

    2019

  44. [49]

    arXiv preprint arXiv:2310.06825 (2023), https://arxiv.org/abs/2310.06825

    Jiang, A.Q., Sablayrolles, A., et al.: Mistral 7b. arXiv preprint arXiv:2310.06825 (2023), https://arxiv.org/abs/2310.06825

    2023 arXiv

  45. [50]

    In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (2014)

    Kazemzadeh, S., Ordonez, V ., Matten, M., Berg, T.L.: Referitgame: Referring to objects in photographs of natural scenes. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (2014)

    2014

  46. [52]

    In: International conference on machine learning

    Kim, W., Son, B., Kim, I.: Vilt: Vision-and-language transformer without convolution or region supervision. In: International conference on machine learning. pp. 5583–5594. PMLR (2021)

    2021

  47. [53]

    In: Findings of the Association for Computational Linguistics: ACL 2023

    Kim, Y ., Kim, J., Mok, W.L., Park, J.H., Lee, S.: Client-customized adaptation for parameter- efficient federated learning. In: Findings of the Association for Computational Linguistics: ACL 2023. pp. 1159–1172 (2023)

    2023

  48. [54]

    arXiv preprint arXiv:2410.22815 (2024)

    Koo, J., Jang, M., Ok, J.: Towards robust and efficient federated low-rank adaptation with heterogeneous clients. arXiv preprint arXiv:2410.22815 (2024)

    2024 arXiv

  49. [55]

    arXiv preprint arXiv:2310.11454 (2023)

    Kopiczko, D.J., Blankevoort, T., Asano, Y .M.: Vera: Vector-based random matrix adaptation. arXiv preprint arXiv:2310.11454 (2023)

    2023 arXiv

  50. [56]

    Krizhevsky, A.: Learning multiple layers of features from tiny images. Tech. rep., University of Toronto (2009) 24

    2009

  51. [57]

    In: Machine learning proceedings 1995, pp

    Lang, K.: Newsweeder: Learning to filter netnews. In: Machine learning proceedings 1995, pp. 331–339. Elsevier (1995)

    1995

  52. [58]

    arXiv preprint arXiv:2502.04387 (2025)

    Lee, R., Kim, M., Rezk, F., Li, R., Venieris, S.I., Hospedales, T.: Fedp 2eft: Federated learning to personalize parameter efficient fine-tuning for multilingual llms. arXiv preprint arXiv:2502.04387 (2025)

    2025

  53. [59]

    In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

    Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. pp. 3045–3059. Association for Computational Linguistics (2021)

    2021

  54. [60]

    arXiv preprint arXiv:1910.13461 (2019), https://arxiv.org/abs/1910.13461

    Lewis, M., Liu, Y ., Goyal, N., et al.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019), https://arxiv.org/abs/1910.13461

    2019 arXiv

  55. [61]

    In: Proceedings of the IEEE International Conference on Computer Vision (2017)

    Li, D., Yang, Y ., Song, Y .Z., Hospedales, T.M.: Deeper, broader and artier domain generaliza- tion. In: Proceedings of the IEEE International Conference on Computer Vision (2017)

    2017

  56. [62]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, H., Huang, W., Wang, J., Shi, Y .: Global and local prompts cooperation via optimal transport for federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12151–12161 (2024)

    2024

  57. [63]

    thirsty

    Li, P., Yang, J., Islam, M.A., Ren, S.: Making ai less" thirsty": Uncovering and addressing the secret water footprint of ai models. arXiv preprint arXiv:2304.03271 (2023)

    2023 arXiv

  58. [64]

    arXiv preprint arXiv:2406.12844 (2024)

    Li, S., Ye, F., Fang, M., Zhao, J., Chan, Y .H., Ngai, E.C.H., V oigt, T.: Synergizing foundation models and federated learning: A survey. arXiv preprint arXiv:2406.12844 (2024)

    2024

  59. [65]

    In: Advances in Neural Information Processing Systems

    Lian, D., Zhou, D., Feng, J., Wang, X.: Scaling & shifting your features: A new baseline for efficient model tuning. In: Advances in Neural Information Processing Systems. vol. 35, pp. 24829–24840 (2022)

    2022

  60. [66]

    In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    Liang, Y ., Duan, N., Gong, Y ., Wu, N., Guo, F., Qi, W., Gong, M., Shou, L., Jiang, D., Cao, G., Fan, X., Zhang, R., Agrawal, R., Cui, E., Wei, S., Bharti, T., Qiao, Y ., Chen, J.H., Wu, W., Liu, S., Yang, F., Majumder, R., Zhao, M., Yu, Q., Liu, Y ., Liu, W., Zhou, M.: Xglue...

    2020

  61. [67]

    In: European Conference on Computer Vision (2014)

    Lin, T.Y ., Maire, M., Belongie, S., et al.: Microsoft coco: Common objects in context. In: European Conference on Computer Vision (2014)

    2014

  62. [68]

    In: Advances in Neural Information Processing Systems

    Liu, H., Tam, D., Mohammed, M., Mohta, J., Huang, T., Bansal, M., Raffel, C.: Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. In: Advances in Neural Information Processing Systems. vol. 35, pp. 13299–13312 (2022)

    2022

  63. [69]

    Liu, H., Li, C., Zhang, Y ., Hu, Z., Shen, S., Li, B., Li, Y ., Lee, Y .J.: Visual instruction tuning (2023)

    2023

  64. [70]

    arXiv preprint arXiv:2412.20004 (2024)

    Liu, J., Liao, Y ., Xu, H., Xu, Y ., Liu, J., Qian, C.: Adaptive parameter-efficient federated fine-tuning on heterogeneous devices. arXiv preprint arXiv:2412.20004 (2024)

    2024 arXiv

  65. [71]

    In: Forty-first International Conference on Machine Learning (2024)

    Liu, S.Y ., Wang, C.Y ., Yin, H., Molchanov, P., Wang, Y .C.F., Cheng, K.T., Chen, M.H.: Dora: Weight-decomposed low-rank adaptation. In: Forty-first International Conference on Machine Learning (2024)

    2024

  66. [72]

    arXiv preprint arXiv:2305.12449 (2023)

    Liu, Y ., Bi, X., Li, L., Chen, S., Yang, W., Sun, X.: Communication efficient federated learning for multilingual neural machine translation with adapter. arXiv preprint arXiv:2305.12449 (2023)

    2023 arXiv

  67. [73]

    arXiv preprint arXiv:1907.11692 (2019) 25

    Liu, Y ., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V .: Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019) 25

    2019 arXiv

  68. [74]

    Advances in Neural Information Processing Systems 37, 39409–39433 (2024)

    Long, G., Shen, T., Jiang, J., Blumenstein, M., et al.: Dual-personalizing adapter for federated foundation models. Advances in Neural Information Processing Systems 37, 39409–39433 (2024)

    2024

  69. [75]

    Long, G., Zhou, T., Lu, Q., Ye, S., Jiang, J., et al.: Federated adapter on foundation models: An out-of-distribution approach

  70. [76]

    arXiv preprint arXiv:2204.08582 (2022)

    Lu, P., Wei, J., Chang, M.Y ., et al.: Learn to explain: Multimodal reasoning via thought chains for science question answering. arXiv preprint arXiv:2204.08582 (2022)

    2022 arXiv

  71. [77]

    arXiv preprint arXiv:2410.10114 (2024)

    Luo, J., Chen, C., Wu, S.: Mixture of experts made personalized: Federated prompt learning for vision-language models. arXiv preprint arXiv:2410.10114 (2024)

    2024 arXiv

  72. [78]

    arXiv preprint arXiv:2405.17267 (2024)

    Ma, Y ., Cheng, L., Wang, Y ., Zhong, Z., Xu, X., Wang, M.: Fedhpl: Efficient heterogeneous federated learning with prompt tuning and logit distillation. arXiv preprint arXiv:2405.17267 (2024)

    2024 arXiv

  73. [79]

    In: Artificial intelligence and statistics

    McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. pp. 1273–1282. PMLR (2017)

    2017

  74. [80]

    arXiv preprint arXiv:1609.07843 (2016)

    Merity, S., Xiong, C., Bradbury, J., Socher, R.: Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843 (2016)

    2016 arXiv

  75. [81]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

    Mishra, A., Alahari, K., Jawahar, C.V .: Ocr-vqa: Visual question answering by reading text in images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

    2019

  76. [82]

    mtgv: Mobilellama-1.4b-base (2023), https://huggingface.co/mtgv/MobileLLaMA-1. 4B-Base

    2023

  77. [83]

    NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2011)

    Netzer, Y ., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y .: Reading digits in natural im- ages with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2011)

    2011

  78. [84]

    In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    Ni, J., Li, J., McAuley, J.: Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 188–197 (2019)

    2019

  79. [85]

    In: Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing (2008)

    Nilsback, M.E., Zisserman, A.: Automated flower classification over a large number of classes. In: Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing (2008)

    2008

  80. [87]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2012)

    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Cats and dogs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2012)

    2012

  81. [88]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

    Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., Wang, B.: Moment matching for multi- source domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

    2019

  82. [89]

    In: Forty-first International Conference on Machine Learning (2024)

    Piao, H., Wu, Y ., Wu, D., Wei, Y .: Federated continual learning via prompt-based dual knowledge transfer. In: Forty-first International Conference on Machine Learning (2024)

    2024

  83. [90]

    In: Proceedings of the International Conference on Machine Learning (2021)

    Radford, A., Kim, J.W., Hallacy, C., et al.: Learning transferable visual models from natural language supervision. In: Proceedings of the International Conference on Machine Learning (2021)

    2021

  84. [91]

    arXiv preprint arXiv:1606.05250 (2016) 26

    Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016) 26

    2016 arXiv

  85. [92]

    RyokoAI: Sharegpt dataset (2023), https://huggingface.co/datasets/RyokoAI/ ShareGPT52K

    2023

  86. [93]

    arXiv preprint arXiv:1910.01108 (2019)

    Sanh, V ., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

    2019 arXiv

  87. [94]

    In: 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)

    Seo, S., Ko, S.W., Park, J., Kim, S.L., Bennis, M.: Communication-efficient and personalized federated lottery ticket learning. In: 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). pp. 581–585. IEEE (2021)

    2021

  88. [95]

    arXiv preprint arXiv:2410.09432 (2024)

    Singhal, R., Ponkshe, K., Vepakomma, P.: Exact aggregation for federated and efficient fine-tuning of foundation models. arXiv preprint arXiv:2410.09432 (2024)

    2024 arXiv

  89. [96]

    In: Proceedings of the 2013 conference on empirical methods in natural language processing

    Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C.D., Ng, A.Y ., Potts, C.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing. pp. 1631–1642 (2013)

    2013

  90. [98]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Su, S., Yang, M., Li, B., Xue, X.: Federated adaptive prompt tuning for multi-domain collaborative learning. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 15117–15125 (2024)

    2024

  91. [99]

    In: 2024 IEEE International Confer- ence on Big Data (BigData)

    Sun, G., Khalid, U., Mendieta, M., Wang, P., Chen, C.: Exploring parameter-efficient fine- tuning to enable foundation models in federated learning. In: 2024 IEEE International Confer- ence on Big Data (BigData). pp. 8015–8024. IEEE (2024)

    2024

  92. [100]

    arXiv preprint arXiv:2403.12313 (2024)

    Sun, Y ., Li, Z., Li, Y ., Ding, B.: Improving lora in privacy-preserving federated learning. arXiv preprint arXiv:2403.12313 (2024)

    2024 arXiv

  93. [101]

    arXiv preprint arXiv:2306.13264 (2023)

    Tamirisa, R., Won, J., Lu, C., Arel, R., Zhou, A.: Fedselect: Customized selection of parame- ters for fine-tuning during personalized federated learning. arXiv preprint arXiv:2306.13264 (2023)

    2023 arXiv

  94. [102]

    Advances in neural information processing systems 35, 19332–19344 (2022)

    Tan, Y ., Long, G., Ma, J., Liu, L., Zhou, T., Jiang, J.: Federated learning from pre-trained models: A contrastive learning approach. Advances in neural information processing systems 35, 19332–19344 (2022)

    2022

  95. [103]

    In: Advances in Neural Information Processing Systems (NeurIPS)

    Tolstikhin, I., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Keysers, D., Uszkoreit, J., Lucic, M., Dosovitskiy, A.: Mlp-mixer: An all-mlp architecture for vision. In: Advances in Neural Information Processing Systems (NeurIPS). vol. 34, pp. 242...

    2021

  96. [104]

    arXiv preprint arXiv:2302.13971 (2023), https://arxiv.org/abs/2302.13971

    Touvron, H., Lavril, T., et al.: Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023), https://arxiv.org/abs/2302.13971

    2023 arXiv

  97. [105]

    arXiv preprint arXiv:2307.09288 (2023), https://arxiv.org/abs/2307.09288

    Touvron, H., Martin, L., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023), https://arxiv.org/abs/2307.09288

    2023 arXiv

  98. [106]

    arXiv preprint arXiv:2210.07558 (2022)

    Valipour, M., Rezagholizadeh, M., Kobyzev, I., Ghodsi, A.: Dylora: Parameter efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. arXiv preprint arXiv:2210.07558 (2022)

    2022 arXiv

  99. [107]

    arXiv preprint arXiv:1804.07461 (2018)

    Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., Bowman, S.R.: Glue: A multi- task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461 (2018)

    2018 arXiv

  100. [108]

    In: Findings of the Association for Computational Linguistics: NAACL 2024

    Wang, R., Yu, T., Zhang, R., Kim, S., Rossi, R., Zhao, H., Wu, J., Mitra, S., Yao, L., Henao, R.: Personalized federated learning for text classification with gradient-free prompt tuning. In: Findings of the Association for Computational Linguistics: NAACL 2024. pp. 4597–4612 ...

    2024

  101. [109]

    arXiv preprint arXiv:2409.05976 (2024)

    Wang, Z., Shen, Z., He, Y ., Sun, G., Wang, H., Lyu, L., Li, A.: Flora: Federated fine- tuning large language models with heterogeneous low-rank adaptations. arXiv preprint arXiv:2409.05976 (2024)

    2024 arXiv

  102. [110]

    In: International Conference on Learning Representations (2022)

    Wei, J., Bosma, M., Zhao, V ., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., Le, Q.V .: Finetuned language models are zero-shot learners. In: International Conference on Learning Representations (2022)

    2022

  103. [111]

    arXiv preprint arXiv:1704.05426 (2017)

    Williams, A., Nangia, N., Bowman, S.R.: A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426 (2017)

    2017 arXiv

  104. [112]

    In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    Wu, F., Li, Z., Li, Y ., Ding, B., Gao, J.: Fedbiot: Llm local fine-tuning in federated learning without full model. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 3345–3355 (2024)

    2024

  105. [113]

    arXiv preprint arXiv:2501.13985 (2025)

    Xiong, B., Yang, X., Song, Y ., Wang, Y ., Xu, C.: Pilot: Building the federated multimodal instruction tuning framework. arXiv preprint arXiv:2501.13985 (2025)

    2025 arXiv

  106. [114]

    arXiv preprint arXiv:2304.12244 (2023), https://arxiv.org/abs/2304.12244

    Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., Lin, Q., Jiang, D.: Wizardlm: Empowering large pre-trained language models to follow complex instructions. arXiv preprint arXiv:2304.12244 (2023), https://arxiv.org/abs/2304.12244

    2023 arXiv

  107. [115]

    In: The Thirteenth International Conference on Learning Representations

    Yan, Y ., Feng, C.M., Zuo, W., Zhu, L., Goh, R.S.M., Liu, Y .: Federated residual low-rank adaption of large language models. In: The Thirteenth International Conference on Learning Representations

  108. [116]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Yang, F.E., Wang, C.Y ., Wang, Y .C.F.: Efficient model personalization in federated learning via client-specific prompt generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 19159–19168 (2023)

    2023

  109. [117]

    arXiv preprint arXiv:2409.15723 (2024)

    Yao, Y ., Zhang, J., Wu, J., Huang, C., Xia, Y ., Yu, T., Zhang, R., Kim, S., Rossi, R., Li, A., et al.: Federated large language models: Current progress and future directions. arXiv preprint arXiv:2409.15723 (2024)

    2024 arXiv

  110. [118]

    arXiv preprint arXiv:2310.13283 (2023)

    Yi, L., Yu, H., Wang, G., Liu, X., Li, X.: pfedlora: model-heterogeneous personalized federated learning with lora tuning. arXiv preprint arXiv:2310.13283 (2023)

    2023 arXiv

  111. [119]

    arXiv preprint arXiv:2305.11414 (2023)

    Yu, S., Muñoz, J.P., Jannesari, A.: Federated foundation models: Privacy-preserving and collaborative learning for large models. arXiv preprint arXiv:2305.11414 (2023)

    2023 arXiv

  112. [120]

    arXiv preprint arXiv:2106.10199 (2021)

    Zaken, E.B., Ravfogel, S., Goldberg, Y .: Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199 (2021)

    2021

  113. [121]

    In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Zhang, J., Vahidian, S., Kuo, M., Li, C., Zhang, R., Yu, T., Wang, G., Chen, Y .: Towards build- ing the federatedgpt: Federated instruction tuning. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 6915–6919. IEEE (2024)

    2024

  114. [122]

    arXiv preprint arXiv:2410.07738 (2024)

    Zhang, J., Duan, Y ., Niu, S., Cao, Y ., Lim, W.Y .B.: Enhancing federated domain adaptation with multi-domain prototype-based federated fine-tuning. arXiv preprint arXiv:2410.07738 (2024)

    2024 arXiv

  115. [123]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL)

    Zhang, N., Chen, M., Bi, Z., Liang, X., Li, L., Shang, X., Yin, K., Tan, C., Xu, J., Huang, F., et al.: Cblue: A chinese biomedical language understanding evaluation benchmark. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL). p...

    2022

  116. [124]

    arXiv preprint arXiv:2401.02385 (2023), https://arxiv.org/abs/2401.02385

    Zhang, P., et al.: Tinyllama: An open-source small language model. arXiv preprint arXiv:2401.02385 (2023), https://arxiv.org/abs/2401.02385

    2023 arXiv

  117. [125]

    arXiv preprint arXiv:2303.10512 (2023) 28

    Zhang, Q., Chen, M., Bukharin, A., Karampatziakis, N., He, P., Cheng, Y ., Chen, W., Zhao, T.: Adalora: Adaptive budget allocation for parameter-efficient fine-tuning. arXiv preprint arXiv:2303.10512 (2023) 28

    2023 arXiv

  118. [126]

    Advances in neural information processing systems 28 (2015)

    Zhang, X., Zhao, J., LeCun, Y .: Character-level convolutional networks for text classification. Advances in neural information processing systems 28 (2015)

    2015

  119. [127]

    In: Annual Meeting of the Association of Computational Linguistics 2023

    Zhang, Z., Yang, Y ., Dai, Y ., Wang, Q., Yu, Y ., Qu, L., Xu, Z.: Fedpetuning: When feder- ated learning meets the parameter-efficient tuning methods of pre-trained language models. In: Annual Meeting of the Association of Computational Linguistics 2023. pp. 9963–9977. Associ...

    2023

  120. [128]

    In: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Zhao, H., Du, W., Li, F., Li, P., Liu, G.: Fedprompt: Communication-efficient and privacy- preserving prompt tuning in federated learning. In: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1–5. IEEE (2023)

    2023

  121. [129]

    In: The Twelfth International Conference on Learning Representations (2024)

    Zhao, W., Chen, Y ., Lee, R., Qiu, X., Gao, Y ., Fan, H., Lane, N.D.: Breaking physical and linguistic borders: Multilingual federated prompt tuning for low-resource languages. In: The Twelfth International Conference on Learning Representations (2024)

    2024

  122. [130]

    In: 2023 IEEE 8th International Conference on Big Data Analytics (ICBDA)

    Zhao, Y ., Zhao, T., Xiang, P., Li, Q., Chen, Z.: Multi-task federated learning medical analysis algorithm integrated into adapter. In: 2023 IEEE 8th International Conference on Big Data Analytics (ICBDA). pp. 24–30. IEEE (2023)

    2023

  123. [131]

    In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)

    Zhong, M., Yin, D., Yu, T., Zaidi, A., Mutuma, M., Jha, R., Awadallah, A.H., Celikyilmaz, A., Liu, Y ., Qiu, X., Radev, D.: Qmsum: A new benchmark for query-based multi-domain meeting summarization. In: Proceedings of the 2021 Conference of the North American Chapter of the As...

    2021

  124. [132]

    arXiv preprint arXiv:2306.15546 (2023) 29

    Zhuang, W., Chen, C., Lyu, L.: When foundation model meets federated learning: Motivations, challenges, and future directions. arXiv preprint arXiv:2306.15546 (2023) 29

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.