Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

AFLoRA: Adaptive Federated Fine-Tuning of Large Language Models with Resource-Aware Low-Rank Adaption

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AFLoRA claims that dynamic per-client rank pruning plus decoupled LoRA training (clients update only B, server updates A) gives exact ideal aggregation of heterogeneous client updates at roughly half the client-side cost.

desk verdict A useful federated LoRA idea with a promising decoupled B/A design, but the written algorithm cannot learn as stated and the lossless-aggregation claim does not survive the server-side A update. read the letter →

arxiv 2505.24773 v2 pith:PWSGBMLI submitted 2025-05-30 cs.LG

Yajie Zhou , Xiaoyi Pang , Zhibo Wang This is my paper
classification cs.LG
keywords federatedlearninglow-rankadaptationLoRAlargelanguagemodelsheterogeneousclientsdynamicrankpruningnon-IIDdata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated fine-tuning of large language models usually assumes every client uses the same LoRA rank, which forces all devices down to the weakest client's budget and distorts aggregation when ranks differ. AFLoRA is built to remove that assumption: each client picks and dynamically prunes its own rank, fine-tunes only the B side of the LoRA update locally, and sends that plus a mask to the server. The server pads the heterogeneous B matrices to a common shape and aggregates them with rank-aware weights. The paper argues that because every client used the same shared A matrix, this aggregation equals the weighted sum of the exact local updates, achieving what it calls ideal aggregation, and that the scheme outperforms prior LoRA-federated methods while roughly halving client-side communication and computation. If correct, AFLoRA is a practical recipe for adapting LLMs with weak, heterogeneous edge clients.

What carries the argument

The load-bearing object is the three-factor LoRA form $\Delta W = B\Lambda A$, where $\Lambda$ is a learnable diagonal matrix. $\Lambda$ does two jobs: its entries serve as soft singular values that drive threshold-based rank pruning, and it is folded into the uploaded matrix ($B' = B\Lambda$ restricted to active columns), so the server aggregates a single matrix per client without losing per-dimension scaling. The decisive mechanism is the shared $A$: because $A$ is common across clients and maintained server-side, zero-padding the heterogeneous $B'$ matrices and multiplying by the same $A$ reproduces each client's exact local update, turning the aggregation into a lossless weighted average rather than the approximate separate averaging of Eq. (2).

What would settle it

Take a trained AFLoRA round and compute the relative Frobenius error $\|B_{\text{global}} A'_{\text{global}} - \sum_k p_k B'_k A_k\|_F / \|\sum_k p_k B'_k A_k\|_F$, where $A'_{\text{global}}$ is the broadcast matrix after server-side fine-tuning and $A_k$ is the truncated matrix clients actually used. With $\alpha < 1$ this error is nonzero; if it is large or tracks the accuracy gains the paper credits to ideal aggregation, the exact-lossless claim is falsified. A simpler variant: set $\alpha = 1$ to disable server $A$ updates and check whether AFLoRA still beats its baselines by the reported margins.

Watch

Extended reading notes

Core claim

AFLoRA's central claim is that federated LoRA fine-tuning can have both heterogeneous ranks and lossless aggregation. The paper reparameterizes the update as $\Delta W = B\Lambda A$, with a learnable diagonal matrix $\Lambda$ between the two low-rank factors; with row and column normalizations, each diagonal entry behaves like a singular value, so low-entry dimensions can be pruned and each client's effective rank drifts down as local adaptation saturates. Clients fine-tune only $B$ and $\Lambda$, while the shared $A$ is frozen locally and instead fine-tuned on the server on a small public dataset and then fused with the previous $A$. Because every client multiplies the same truncated $A$, the server can zero-pad the heterogeneous $B'$ matrices to a common shape and aggregate them with data- and rank-aware weights, and Eq. (14) concludes that the result equals $\sum_k p_k B'_k A_k$ exactly, which is ideal aggregation. Experiments on GPT-2, TinyLlama-1.1B, and Qwen2.5-3B then report consistently higher accuracy than FLoRA, FlexLoRA, and HETLoRA, with client-side communication and computation roughly halved.

Load-bearing premise

The lossless-aggregation result depends on the server broadcasting back the same $A$ matrix that clients used in the round, but the server fine-tunes $A$ on public data before broadcasting it; if that fine-tuning changes $A$ nontrivially, the claimed exact equality to the sum of client updates no longer holds.

Editorial extensions

If this is right

  • Clients with different computation and communication budgets can participate with different LoRA ranks without introducing aggregation error, so the global model is no longer bottlenecked by the weakest device.
  • Because clients upload only $B'$ and a mask, per-round client-side communication and training cost drop by roughly half relative to methods that also send $A$.
  • The diagonal-matrix threshold pruning means a client's rank shrinks as its local adaptation saturates, freeing local resources as training progresses.
  • Server-side fine-tuning of $A$ on a small public dataset counteracts non-IID overfitting and catastrophic forgetting, which the experiments tie to consistent gains on skewed data.
  • Under the reported settings, AFLoRA yields higher or comparable accuracy than FLoRA, FlexLoRA, and HETLoRA on GPT-2, TinyLlama-1.1B, and Qwen2.5-3B across IID and non-IID data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'ideal aggregation' equality is exact only within a single round when $A$ is unchanged; server-side fine-tuning of $A$ before the broadcast means the next round starts from a different shared subspace, so the claimed losslessness should be read as per-round conditional rather than global over training.
  • A direct test is to measure the relative Frobenius error between $B_{\text{global}} A'_{\text{global}}$ and $\sum_k p_k B'_k A_k$; if server-side $A$ changes are non-negligible, the residual will be nonzero and may grow with $\alpha$ and the size of the public dataset.
  • The same diagonal matrix could be repurposed as a communication-compression score: dimensions with small $\lambda$ could be dropped entirely from transmission, not just pruned locally, yielding further bandwidth savings under a fixed accuracy budget.
  • The log-rank weighting in Eq. (12) is a heuristic; extreme rank disparity across clients could over- or under-weight small clients, so a calibration study across skewed resource distributions would be a natural extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes AFLoRA, a federated fine-tuning framework for LLMs in which LoRA is reparameterized as ΔW = BΛA. Clients fix the shared A matrix, train B and a diagonal Λ locally, dynamically prune low-information dimensions using a βσ rule on the diagonal entries, and upload compact updates B′; the server zero-pads and rank-aware aggregates B′, fine-tunes A on a small public dataset, and fuses the two A matrices before broadcasting. The authors report accuracy gains over FLoRA, FlexLoRA, and HETLoRA on GPT-2, TinyLlama-1.1B, and Qwen2.5-3B across four datasets, with roughly halved client-side communication/computation cost, and claim that the zero-padding aggregation achieves 'ideal aggregation' by matching client updates exactly.

Significance. If the algorithm is corrected and the exactness claim is re-scoped, AFLoRA addresses a real problem: heterogeneous and resource-constrained clients in federated LLM fine-tuning. The decoupling of A (server) from B,Λ (clients) is a sensible way to reduce client load, the diagonal-matrix rank pruning is a concrete mechanism for adaptive rank, and the empirical comparison spans three model families and four datasets with consistent wins over recent baselines. The paper also reports client-side cost ratios, which is valuable. However, the two signature contributions—dynamic rank assignment and lossless aggregation—are currently undermined by the zero initialization of B and Λ and by the server-side A update that changes the broadcast adapter after the 'ideal aggregation' derivation. These are fixable but require the authors to state the implemented initialization precisely and to limit the exactness claim to the pre-fusion aggregation step.

major comments (3)
  1. [V-C (Client-Side Fine-Tuning)] The statement 'each client k initializes both B_t^k and Lambda_t^k with zeros' is incompatible with the rest of the algorithm. With ΔW = BΛA and the objective in Eq. (10), the gradients with respect to B and Λ are both zero at B=0, Λ=0, and the column-normalization regularizer also has zero derivative there because it contains a factor of b_j. Consequently, B and Λ never move during local training, and the aggregated results in Tables I-II cannot be produced by the algorithm as written. Please state the actual initialization (e.g., Λ=I, B=0) and verify that the local update equations are consistent with that initialization.
  2. [V-D, Eq. (14), and V-C, Eq. (11)] The 'ideal aggregation' derivation equates the aggregated update to sum_k p_k B'_k A_k using A_global, but the actual update broadcast to clients is B_global A'_global with A'_global = α A_global + (1-α) A_FT after server-side public-data fine-tuning. For α<1, B_global A'_global = α sum_k p_k B'_k A_k + (1-α) B_global A_FT, which is not equal to sum_k p_k B'_k A_k unless B_global A_FT equals that sum. The claim that 'the aggregated result matches the client updates exactly' should therefore be restricted to the intermediate step before server-side A fine-tuning, or the proof must account for the fusion of A.
  3. [V-B, Eq. (7), and V-C, Eq. (11)] The derivation of dimensional information content I_{k,j} = C^2 |λ_{k,j}|^2 assumes ‖a_j‖_2 = C and ‖b_j‖_2 = 1. The paper only states that A is initialized with normalized rows; after server-side fine-tuning and fusion, A'_global = α A_global + (1-α) A_FT will generally have row norms different from C, and no re-normalization is mentioned. This undermines the exact information-content proxy used for pruning in rounds after the first. Please specify whether/how A rows are re-normalized after fusion, or justify robustness of the βσ rule to non-uniform row norms.
minor comments (4)
  1. [V-C and VI-B] The text states both that clients 'upload only B′t_k and Ht_k' and that AFLoRA 'updates and transmits only the client-specific matrices B and Λ'; please clarify whether Λ is transmitted separately or is folded into B′, since this determines the reported client-side communication cost.
  2. [V-D, Eq. (14)] The notation 'B′t_k A_t^k' is dimensionally ambiguous because B′t_k is m×r_t^k while A_t^k has r1_k rows; the equality should explicitly restrict A to the active rows selected by the mask rather than the first r1_k rows.
  3. [VI-A] Hyperparameter values (β, γ, α, C) and training details (local epochs, batch size, learning rate, number of runs) are not reported, so the significance of the accuracy differences in Tables I-II cannot be fully assessed.
  4. [Figure 1] The 'classic aggregation' vs. 'ideal aggregation' comparison lacks experimental details (model, dataset, partitioning, number of rounds); adding these would make the motivation reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the empirical comparisons are externally anchored, and the flagged issues are correctness/consistency problems rather than circular reductions.

full rationale

The central performance claims of AFLoRA are empirical comparisons against external baselines (FLoRA, FlexLoRA, HETLoRA) on standard datasets and public models, so they do not reduce to fitted constants, renamed inputs, or self-citations. The 'ideal aggregation' identity in Section V-D is a direct linear-algebra consequence of the decoupled shared-A design: if all clients share the same A, then averaging the padded B matrices and multiplying by A equals the weighted average of the local B-A products. This is a design property rather than a circular prediction. The paper does, however, contain two internal inconsistencies that are outside the circularity taxonomy: (1) the local initialization of B and Lambda to zero would make all gradients vanish, so the reported training cannot proceed literally as stated; and (2) the exactness claim is overstated because the server fine-tunes and fuses A via Eq. (11) before broadcasting, so the actual global update B_global A'_global does not equal sum p_k B'_k A_k. These are reproducibility and correctness concerns, not cases where a result is equivalent to its input by construction. No load-bearing self-citation or author-derived uniqueness theorem is invoked, and the ablation results are empirical rather than fitted to the aggregation equations. The circularity score is therefore 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central derivation rests on assumptions about the information-theoretic meaning of the diagonal matrix, the validity of the pruning rule, and the knowledge split between A and B. Four hyperparameters (beta, gamma, alpha, C) are free and unreported. No new physical or conceptual entities are introduced; the diagonal matrix is a trainable parameter matrix, not an independent entity.

free parameters (4)
  • beta
    Pruning threshold multiplier in Eq. (8); chosen by hand; value not reported in the paper.
  • gamma
    Weight of the B-column norm penalty in Eq. (10); chosen by hand; value not reported.
  • alpha
    Fusion ratio for the server-side A update in Eq. (11); chosen by hand; value not reported.
  • C
    Target norm for A rows in the constraint ||a_j||_2 = C; chosen by hand; value not reported.
assumptions (5)
  • domain assumption A captures general knowledge and B captures client-specific knowledge.
    Used to justify decoupling client-side B training from server-side A training; attributed to prior work [11, 25] but not proven for the federated setting.
  • ad hoc to paper Squared diagonal values |lambda_j|^2 measure dimensional information content.
    Eq. (7) derives I = C^2 |lambda|^2 under unit-norm constraints; this SVD analogy is an assumption, not a theorem, and it fails if A rows drift from norm C after server-side fine-tuning.
  • ad hoc to paper Pruning dimensions with |lambda|^2 below beta times the standard deviation removes only redundant dimensions.
    The beta-sigma outlier rule in Eq. (8) is an arbitrary statistical heuristic; no evidence is given that it correctly identifies uninformative LoRA dimensions.
  • standard math Zero-padding a client's B matrix does not change the product with A.
    Padding zero columns leaves the matrix product unchanged, which is standard linear algebra.
  • domain assumption A small public dataset disjoint from client data improves global model generalization.
    The paper assumes such a public dataset exists and that server-side fine-tuning on it improves the global model; no analysis of size, domain match, or distribution requirements is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AFLoRA: Adaptive Federated Fine-Tuning of Large Language Models with Resource-Aware Low-Rank Adaption." pith.science (2026). https://pith.science/paper/PWSGBMLI

@misc{pith2026250524773,
  author       = {Pith},
  title        = {Pith review of: AFLoRA: Adaptive Federated Fine-Tuning of Large Language Models with Resource-Aware Low-Rank Adaption},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PWSGBMLI}},
  note         = {Machine review of arXiv:2505.24773}
}
read the original abstract

Federated fine-tuning has emerged as a promising approach to adapt foundation models to downstream tasks using decentralized data. However, real-world deployment remains challenging due to the high computational and communication demands of fine-tuning Large Language Models (LLMs) on clients with data and system resources that are heterogeneous and constrained. In such settings, the global model's performance is often bottlenecked by the weakest clients and further degraded by the non-IID nature of local data. Although existing methods leverage parameter-efficient techniques such as Low-Rank Adaptation (LoRA) to reduce communication and computation overhead, they often fail to simultaneously ensure accurate aggregation of low-rank updates and maintain low system costs, thereby hindering overall performance. To address these challenges, we propose AFLoRA, an adaptive and lightweight federated fine-tuning framework for LLMs. AFLoRA decouples shared and client-specific updates to reduce overhead and improve aggregation accuracy, incorporates diagonal matrix-based rank pruning to better utilize local resources, and employs rank-aware aggregation with public data refinement to strengthen generalization under data heterogeneity. Extensive experiments demonstrate that AFLoRA outperforms state-of-the-art methods in both accuracy and efficiency, providing a practical solution for efficient LLM adaptation in heterogeneous environments in the real world.

Figures

Figures reproduced from arXiv: 2505.24773 by the authors.

Figure 1
Figure 1. Performance comparison under different aggregation methods. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The overview of AFLoRA As a result, the aggregated update significantly deviates from the optimal direction that would have been obtained by directly averaging Bt kAt k across clients. This bias between the global aggregation results and local updates leads to significant parameter drift. As a result, model convergence is severely compromised, and the global model may ultimately fail to capture the true local data d… view at source ↗
Figure 3
Figure 3. Effect of the proposed mechanisms. TABLE I PERFORMANCE AND COST COMPARISON OF AFLORA AND BASELINE METHODS UNDER IID SCENARIOS. Foundation Model Dataset Model Performance Client-Side Costs (%) AFLoRA FLoRA FlexLoRA HETLoRA AFLoRA FLoRA FlexLoRA HETLoRA GPT-2 Wizard 0.3929 0.3752 0.2523 0.1967 0.8095 1.9027 1.9027 1.9027 FinGPT 0.6144 0.5746 0.2523 0.3975 0.8062 1.9027 1.9027 1.9027 TinyLlama-1.1B Wizard 0.4267 0.3882… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The performance comparison when the degree of non-IID [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Attention-Free and Lightweight Token Reduction for Efficient Vision-Language Models

    cs.CV 2026-07 conditional novelty 5.0 of 10

    ALTR keeps model accuracy nearly intact while pruning up to ~89% of visual tokens, using entropy ranking plus stride sampling on an MLP consistency signal, without attention maps or pairwise comparisons.

Reference graph

Works this paper leans on

40 extracted references · 9 canonical work pages · cited by 1 Pith paper

  1. [1]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , “Training language models to follow instructions with human feedback,” Advances in neural information processing systems , vol. 35, pp. 27 730–27 744, 2022

    2022

  2. [2]

    Finetuned language models are zero-shot learners,

    J. Wei, M. Bosma, V . Y . Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V . Le, “Finetuned language models are zero-shot learners,” arXiv preprint arXiv:2109.01652 , 2021

    arXiv 2021

  3. [3]

    Self-instruct: Aligning language models with self- generated instructions,

    Y . Wang, Y . Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi, “Self-instruct: Aligning language models with self- generated instructions,” arXiv preprint arXiv:2212.10560 , 2022

    arXiv 2022

  4. [4]

    Federated learning: Strategies for improving com- munication efficiency,

    J. Kone ˇcn`y, H. B. McMahan, F. X. Yu, P. Richt ´arik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving com- munication efficiency,” arXiv preprint arXiv:1610.05492 , 2016, doi: 10.48550/arXiv.1610.05492

    arXiv doi:10.48550/arxiv.1610.05492 2016

  5. [5]

    Federated learning: Challenges, methods, and future directions,

    T. Li, A. K. Sahu, A. Talwalkar, and V . Smith, “Federated learning: Challenges, methods, and future directions,” IEEE signal processing magazine, vol. 37, no. 3, pp. 50–60, 2020

    2020

  6. [6]

    Federated learning review: Fundamentals, enabling technologies, and future applications,

    S. Banabilah, M. Aloqaily, E. Alsayed, N. Malik, and Y . Jararweh, “Federated learning review: Fundamentals, enabling technologies, and future applications,” Information processing & management , vol. 59, no. 6, p. 103061, 2022

    2022

  7. [7]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International conference on machine learning . PMLR, 2019, pp. 2790–2799

    2019

  8. [8]

    Bitfit: Simple parameter- efficient fine-tuning for transformer-based masked language-models,

    E. B. Zaken, S. Ravfogel, and Y . Goldberg, “Bitfit: Simple parameter- efficient fine-tuning for transformer-based masked language-models,” arXiv preprint arXiv:2106.10199 , 2021

    arXiv 2021

Show all 40 references
  1. [9]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv preprint arXiv:2101.00190 , 2021

    2021 arXiv

  2. [10]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  3. [11]

    Lori: Reducing cross- task interference in multi-task low-rank adaptation,

    J. Zhang, J. You, A. Panda, and T. Goldstein, “Lori: Reducing cross- task interference in multi-task low-rank adaptation,” arXiv preprint arXiv:2504.07448, 2025

    2025 arXiv

  4. [12]

    Adalora: Adaptive budget allocation for parameter-efficient fine-tuning,

    Q. Zhang, M. Chen, A. Bukharin, N. Karampatziakis, P. He, Y . Cheng, W. Chen, and T. Zhao, “Adalora: Adaptive budget allocation for parameter-efficient fine-tuning,” arXiv preprint arXiv:2303.10512, 2023

    2023 arXiv

  5. [13]

    Initialization using update approximation is a silver bullet for extremely efficient low-rank fine-tuning,

    K. Ponkshe, R. Singhal, E. Gorbunov, A. Tumanov, S. Horvath, and P. Vepakomma, “Initialization using update approximation is a silver bullet for extremely efficient low-rank fine-tuning,” arXiv preprint arXiv:2411.19557, 2024

    2024

  6. [14]

    Slora: Federated parameter efficient fine- tuning of language models,

    S. Babakniya, A. R. Elkordy, Y . H. Ezzeldin, Q. Liu, K.-B. Song, M. El- Khamy, and S. Avestimehr, “Slora: Federated parameter efficient fine- tuning of language models,” arXiv preprint arXiv:2308.06522 , 2023

    2023 arXiv

  7. [15]

    Fedadapter: Efficient federated learning for modern nlp,

    D. Cai, Y . Wu, S. Wang, F. X. Lin, and M. Xu, “Fedadapter: Efficient federated learning for modern nlp,” arXiv preprint arXiv:2205.10162 , 2022

    2022 arXiv

  8. [16]

    Fedbiot: Llm local fine-tuning in federated learning without full model,

    F. Wu, Z. Li, Y . Li, B. Ding, and J. Gao, “Fedbiot: Llm local fine-tuning in federated learning without full model,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2024, pp. 3345–3355

    2024

  9. [17]

    Improving lora in privacy-preserving federated learning,

    Y . Sun, Z. Li, Y . Li, and B. Ding, “Improving lora in privacy-preserving federated learning,” arXiv preprint arXiv:2403.12313 , 2024

    2024 arXiv

  10. [18]

    Federated fine-tuning of large language models under heterogeneous language tasks and client resources,

    J. Bai, D. Chen, B. Qian, L. Yao, and Y . Li, “Federated fine-tuning of large language models under heterogeneous language tasks and client resources,” arXiv e-prints, pp. arXiv–2402, 2024

    2024

  11. [19]

    Flora: Federated fine-tuning large language models with heterogeneous low- rank adaptations,

    Z. Wang, Z. Shen, Y . He, G. Sun, H. Wang, L. Lyu, and A. Li, “Flora: Federated fine-tuning large language models with heterogeneous low- rank adaptations,” arXiv preprint arXiv:2409.05976 , 2024

    2024 arXiv

  12. [20]

    Towards building the federatedgpt: Federated instruction tun- ing,

    J. Zhang, S. Vahidian, M. Kuo, C. Li, R. Zhang, T. Yu, G. Wang, and Y . Chen, “Towards building the federatedgpt: Federated instruction tun- ing,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 6915–6919

    2024

  13. [21]

    pfedprompt: Learning personalized prompt for vision-language models in federated learning,

    T. Guo, S. Guo, and J. Wang, “pfedprompt: Learning personalized prompt for vision-language models in federated learning,” in Proceedings of the ACM Web Conference 2023 (WWW ’23) . Association for Computing Machinery, 2023, pp. 1364–1374. [Online]. Available: https://doi.org/10...

    2023

  14. [22]

    Fedperfix: To- wards partial model personalization of vision transformers in federated learning,

    G. Sun, M. Mendieta, J. Luo, S. Wu, and C. Chen, “Fedperfix: To- wards partial model personalization of vision transformers in federated learning,” in Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4988–4998

    2023

  15. [23]

    Openfedllm: Training large language models on decentralized private data via federated learning,

    R. Ye, W. Wang, J. Chai, D. Li, Z. Li, Y . Xu, Y . Du, Y . Wang, and S. Chen, “Openfedllm: Training large language models on decentralized private data via federated learning,” in Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining , 2024, pp. ...

    2024

  16. [24]

    Fed-sb: A silver bullet for extreme communication efficiency and performance in (private) federated lora fine-tuning,

    R. Singhal, K. Ponkshe, R. Vartak, L. R. Varshney, and P. Vepakomma, “Fed-sb: A silver bullet for extreme communication efficiency and performance in (private) federated lora fine-tuning,” arXiv preprint arXiv:2502.15436, 2025

    2025

  17. [25]

    Selective ag- gregation for low-rank adaptation in federated learning,

    P. Guo, S. Zeng, Y . Wang, H. Fan, F. Wang, and L. Qu, “Selective ag- gregation for low-rank adaptation in federated learning,” arXiv preprint arXiv:2410.01463, 2024

    2024 arXiv

  18. [26]

    Fedlfc: Towards efficient federated multilingual modeling with lora-based language family clustering,

    Z. Guo, Y . Zhang, Z. Zhang, Z. Xu, and I. King, “Fedlfc: Towards efficient federated multilingual modeling with lora-based language family clustering,” in Findings of the Association for Computational Linguistics: NAACL 2024 , 2024, pp. 1519–1528

    2024

  19. [27]

    Heterogeneous lora for federated fine-tuning of on-device foundation models,

    Y . J. Cho, L. Liu, Z. Xu, A. Fahrezi, and G. Joshi, “Heterogeneous lora for federated fine-tuning of on-device foundation models,” arXiv preprint arXiv:2401.06432, 2024

    2024 arXiv

  20. [28]

    Personalized federated fine-tuning for heterogeneous data: An automatic rank learning approach via two-level lora,

    J. Hao, Y . Wu, A. Payani, M. Lee, and M. Liu, “Personalized federated fine-tuning for heterogeneous data: An automatic rank learning approach via two-level lora,” arXiv preprint arXiv:2503.03920 , 2025

    2025 arXiv

  21. [29]

    Autorank: Mcda based rank personalization for lora-enabled distributed learning,

    S. Chen, O. Tavallaie, N. Nazemi, X. Chen, and A. Y . Zomaya, “Autorank: Mcda based rank personalization for lora-enabled distributed learning,” arXiv preprint arXiv:2412.15553 , 2024

    2024 arXiv

  22. [30]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics . PMLR, 2017, pp. 1273– 1282

    2017

  23. [31]

    Wizardlm: Empowering large language models to follow complex instructions,

    C. Xu, Q. Sun, K. Zheng, X. Geng, P. Zhao, J. Feng, C. Tao, and D. Jiang, “Wizardlm: Empowering large language models to follow complex instructions,” arXiv preprint arXiv:2304.12244 , 2023

    2023 arXiv

  24. [32]

    Fingpt: Democratizing internet-scale data for financial large language models,

    X.-Y . Liu, G. Wang, H. Yang, and D. Zha, “Fingpt: Democratizing internet-scale data for financial large language models,” arXiv preprint arXiv:2307.10485, 2023

    2023 arXiv

  25. [33]

    Conover, M

    M. Conover, M. Hayes, A. Mathur, J. Xie, J. Wan, S. Shah, A. Ghodsi, P. Wendell, M. Zaharia, and R. Xin. (2023) Free dolly: Introducing the world’s first truly open instruction-tuned llm. [Online]. Available: https://www.databricks.com/blog/2023/04/12/dolly- first-open-commerc...

    2023

  26. [34]

    Character-level convolutional net- works for text classification,

    X. Zhang, J. Zhao, and Y . LeCun, “Character-level convolutional net- works for text classification,” Advances in neural information processing systems, vol. 28, 2015

    2015

  27. [35]

    Stanford alpaca: An instruction-following llama model,

    R. Taori, I. Gulrajani, T. Zhang, Y . Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction-following llama model,” 2023

    2023

  28. [36]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  29. [37]

    Tinyllama: An open-source small language model,

    P. Zhang, G. Zeng, T. Wang, and W. Lu, “Tinyllama: An open-source small language model,” arXiv preprint arXiv:2401.02385 , 2024

    2024 arXiv

  30. [38]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv et al. , “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  31. [39]

    Measuring massive multitask language understanding,

    D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” arXiv preprint arXiv:2009.03300 , 2020

    2009 arXiv

  32. [40]

    Good debt or bad debt: Detecting semantic orientations in economic texts,

    P. Malo, A. Sinha, P. Korhonen, J. Wallenius, and P. Takala, “Good debt or bad debt: Detecting semantic orientations in economic texts,” Journal of the Association for Information Science and Technology , vol. 65, no. 4, pp. 782–796, 2014

    2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.