Pith. sign in

REVIEW 5 cited by

FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2108.06098 v3 pith:IKYHS4WS submitted 2021-08-13 cs.LG cs.CV

Nam Hyeon-Woo , Moon Ye-Bin , Tae-Hyun Oh This is my paper · ORCID
classification cs.LGcs.CV
keywords low-rankmethodfedparaparameterscommunication-efficientfederatedhadamardlayers
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this work, we propose a communication-efficient parameterization, FedPara, for federated learning (FL) to overcome the burdens on frequent model uploads and downloads. Our method re-parameterizes weight parameters of layers using low-rank weights followed by the Hadamard product. Compared to the conventional low-rank parameterization, our FedPara method is not restricted to low-rank constraints, and thereby it has a far larger capacity. This property enables to achieve comparable performance while requiring 3 to 10 times lower communication costs than the model with the original layers, which is not achievable by the traditional low-rank methods. The efficiency of our method can be further improved by combining with other efficient FL optimizers. In addition, we extend our method to a personalized FL application, pFedPara, which separates parameters into global and local ones. We show that pFedPara outperforms competing personalized FL methods with more than three times fewer parameters.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Assortment of Attention Heads: Accelerating Federated PEFT with Head Pruning and Strategic Client Selection

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A federated fine-tuning method prunes 90% of attention heads, weights updates by attention importance, and selects clients by loss gap, cutting communication 1.8x and training compute 3.9x with under 2% accuracy drop.

  2. Can Smaller LLMs do better? Unlocking Cross-Domain Potential through Parameter-Efficient Fine-Tuning for Text Summarization

    cs.CL 2025-09 reject novelty 5.0 of 10

    PEFT adapters trained on high-resource summarization domains can improve Llama-3-8B's summaries on unseen domains, but the reported gains are weakened by test-set selection and missing significance tests.

  3. LLM-driven Medical Report Generation via Communication-efficient Heterogeneous Federated Learning

    cs.CV 2025-06 conditional novelty 5.0 of 10

    FedMRG trains federated LLM-based report generators with low-rank adapters, diagnosis prompts, and dual-adapter mutual boosting, beating baselines on chest X-ray benchmarks.

  4. Parameter-Efficient Fine-Tuning of 3D DDPM for MRI Image Generation Using Tensor Networks

    eess.IV 2025-07 conditional novelty 4.0 of 10

    TenVOO represents 3D convolution weight updates as tensor networks, fine-tuning a brain MRI DDPM with only 0.3% of full trainable parameters while achieving competitive or better structural similarity on ADNI, PPMI, a...

  5. MAP: Revisiting Weight Decomposition for Low-Rank Adaptation

    cs.LG 2025-05 conditional novelty 4.0 of 10

    MAP decouples a weight matrix's direction and magnitude by normalizing the whole matrix and the low-rank update by their Frobenius norms and scaling each with a learnable scalar.

Pith tools