Pith. sign in

REVIEW 13 cited by

pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.13283 v2 pith:JXDB46G6 submitted 2023-10-20 cs.LG cs.DC

Liping Yi , Han Yu , Gang Wang , Xiaoguang Liu , Xiaoxiao Li This is my paper · ORCID
classification cs.LGcs.DC
keywords federatedlearningmodelpfedloraadapterloramodel-heterogeneouspersonalized
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Federated learning (FL) is an emerging machine learning paradigm in which a central server coordinates multiple participants (clients) collaboratively to train on decentralized data. In practice, FL often faces statistical, system, and model heterogeneities, which inspires the field of Model-Heterogeneous Personalized Federated Learning (MHPFL). With the increased interest in adopting large language models (LLMs) in FL, the existing MHPFL methods cannot achieve acceptable computational and communication costs, while maintaining satisfactory model performance. To bridge this gap, we propose a novel and efficient model-heterogeneous personalized Federated learning framework based on LoRA tuning (pFedLoRA). Inspired by the popular LoRA method for fine-tuning pre-trained LLMs with a low-rank model (a.k.a., an adapter), we design a homogeneous small adapter to facilitate federated client's heterogeneous local model training with our proposed iterative training for global-local knowledge exchange. The homogeneous small local adapters are aggregated on the FL server to generate a global adapter. We theoretically prove the convergence of pFedLoRA. Extensive experiments on two benchmark datasets demonstrate that pFedLoRA outperforms six state-of-the-art baselines, beating the best method by 1.35% in test accuracy, 11.81 times computation overhead reduction and 7.41 times communication cost saving.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning

    cs.LG 2026-08 conditional novelty 6.0 of 10

    A single federated reward model can personalize faster than group-specific models when preference groups are balanced, and group-debiased sampling restores this under imbalance.

  2. FedLSG: LLM-Enhanced Semantic Calibration for Federated Graph Backdoor Defense

    cs.CR 2026-07 reject novelty 6.0 of 10

    FedLSG combines LLM semantic scoring with rule-based structural signals to downweight backdoor triggers and malicious client updates in federated graph learning.

  3. Convergence Analysis of Aggregation-Broadcast in LoRA-enabled Distributed Fine-Tuning

    cs.LG 2025-08 conditional novelty 6.0 of 10

    A unified convergence analysis of LoRA aggregation in federated learning shows Product-Sum aggregation converges globally at the optimal rate, while Sum-Product aggregation suffers from broadcast error from SVD truncation.

  4. H2Tune: Federated Foundation Model Fine-Tuning with Hybrid Heterogeneity

    cs.LG 2025-07 conditional novelty 6.0 of 10

    H2Tune enables federated fine-tuning across heterogeneous foundation models by sharing sparsified rank-aligned middle matrices with learned layer mappings and alternating shared/private updates.

  5. FedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCA

    cs.LG 2025-06 conditional novelty 6.0 of 10

    FedRPCA decomposes federated LoRA client updates with Robust PCA into common and client-specific components, averaging the common part and scaled-averaging the sparse part, which improves accuracy and convergence over...

  6. DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models

    cs.LG 2025-05 reject novelty 6.0 of 10

    A truncated-SVD consensus step for decentralized LoRA is claimed to reach O(1/sqrt T) convergence, matching decentralized SGD, with supporting CLIP and LLAMA2-7B experiments.

  7. FedHL: Federated Learning for Heterogeneous Low-Rank Adaptation via Unbiased Aggregation

    cs.LG 2025-05 reject novelty 6.0 of 10

    FedHL aggregates heterogeneous LoRA updates against a full-rank global baseline and claims O(1/sqrt T) convergence, with small gains on three LLM fine-tuning datasets.

  8. Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Dysco reduces cross-client interference in federated LoRA fine-tuning by fixing each client's update subspace to directions insensitive to other clients' activations, yielding tighter bounds and better benchmark accuracy.

  9. LAARA: Layer-Aware Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

    cs.LG 2026-07 conditional novelty 5.0 of 10

    LAARA allocates LoRA ranks per layer from diagonal Fisher (gradient-based) estimates, reporting improved accuracy with fewer trainable parameters on GLUE and MathInstruct.

  10. FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models

    cs.LG 2025-06 reject novelty 5.0 of 10

    FedNano centralizes the frozen LLM on the server, trains lightweight NanoAdapters on clients, and reports higher federated VQA accuracy than FedAvg, FedProx, and FedDPA-F on ScienceQA and IconQA.

  11. Many-Task Federated Fine-Tuning via Unified Task Vectors

    cs.LG 2025-02 conditional novelty 5.0 of 10

    MaTU aggregates task vectors across clients using sign-conflict similarity and lightweight masks, reaching 77 to 84 percent normalized accuracy on 30 vision tasks with one transmitted vector per client.

  12. A New Pathway to Integrated Learning and Communication (ILAC): Large AI Model and Hyperdimensional Computing for Communication

    eess.SP 2025-06 conditional novelty 4.0 of 10

    This work proposes a unified ILAC framework enhanced by large AI models and hyperdimensional computing, with a cost-to-performance optimization case study solved by Dinkelbach and alternating optimization.

  13. AI-in-the-Loop Sensing and Communication Joint Design for Edge Intelligence

    cs.LG 2025-02 conditional novelty 4.0 of 10

    An AI-in-the-loop JSAC framework tunes sensing and transmission to lower both resource costs and validation loss in federated edge learning.

Pith tools