REVIEW 2 major objections 7 minor 54 references
One-shot Federated Learning Methods: A Practical Guide
T0 review · 2 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper surveys one-shot federated learning, groups the methods into four technique families, and draws practical design guidance from cross-method comparisons.
desk verdict Useful survey of one-shot FL with a sensible taxonomy; the 'findings' in Section 4.1 are informal observations, not validated comparisons, and the paper overclaims completeness. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the taxonomy itself: four main families — Parameter Learning, Knowledge Distillation, Generative Models, and Ensemble Methods — with hybrid methods that combine them. The taxonomy does the work of the argument: it gives each surveyed method a location, exposes which techniques address data heterogeneity versus model heterogeneity, and organizes the cross-method comparisons that support the paper's findings. Within the families, the paper also names recurring mechanisms such as prototype learning, empirical Fisher aggregation, data and model distillation, GAN/VAE/diffusion-based data generation, and static versus adaptive ensembling.
What would settle it
Run the surveyed methods on a single fixed benchmark with the same datasets, model architectures, compute budgets, and privacy settings; if generative models do not systematically beat knowledge distillation, or adaptive ensembles do not beat static ones, the paper's central comparative findings fail.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that one-shot federated learning can be systematically organized and that doing so reveals consistent patterns: the field faces data heterogeneity and model heterogeneity, methods fall into four technique families, and several design choices reliably help accuracy. The paper presents itself as the first survey focused specifically on OFL, with a taxonomy that separates methods by the technique they employ rather than by the problem they address. It reports findings such as generative models generally beating knowledge distillation, adaptive ensembles beating static ones, prototype learning being a strong building block, and performing multiple server-side updates after a single upload being beneficial.
Load-bearing premise
The findings in Section 4.1 compare accuracy numbers reported in different papers that use different datasets, model architectures, and experimental settings, and the paper provides no unified benchmark to guarantee those comparisons are fair; if the settings are not comparable, the rankings may not hold.
Editorial extensions
If this is right
- If the taxonomy is right, new OFL papers can be classified quickly, and the field can accumulate results by technique family rather than by individual method.
- If generative models generally outperform knowledge distillation in one-shot settings, then future OFL work should prioritize data-generation approaches and track the quality of the chosen generator.
- If adaptive ensembles reliably beat static ones, then server-side aggregation should be designed to weigh local models per data instance or per similarity, as in mixture-of-experts approaches.
- If multiple server-side updates after one upload help, then the one-shot constraint should be understood as restricting client-server communication, not the server's internal optimization.
- If prototype learning is a broadly compatible component, then new methods can graft prototype extraction onto distillation-, generative-, or ensemble-based pipelines.
Reading between the lines
- A natural next step the survey does not take is a unified benchmark that fixes datasets, architectures, and budgets across all four families, which would either confirm or overturn the ranking of generative over distillation and adaptive over static ensembles.
- The trend the paper documents, from GANs to VAEs to diffusion models, suggests that OFL accuracy will keep tracking the frontier of generative modeling; a practical extension would be a modular library where the data generator is swappable.
- For LLM-scale federated learning, the survey's 'upload once, update many times' finding points to a concrete research program: merging or distilling fine-tuned LLMs in a single round while running iterative server-side refinement without additional client round trips.
- The privacy discussion is mostly qualitative; a sharper extension would analyze each taxonomy family under a formal differential-privacy budget and characterize the accuracy-privacy-communication trade-off.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper is a survey of one-shot federated learning (OFL), a paradigm that restricts client-server communication to a single round. It proposes a taxonomy of OFL methods into four groups (parameter learning, knowledge distillation, generative models, and ensemble methods) plus hybrid methods, discusses the two main challenges (data and model heterogeneity), and derives a set of practical findings and future directions. The authors claim this is the first survey specifically focused on OFL and that it comprehensively covers all existing OFL literature.
Significance. If the taxonomy and coverage are accurate, the survey provides a useful structured entry point to a rapidly growing subfield, and the discussion of hybrid methods is a useful contribution. The paper's strengths include a coherent high-level categorization and a synoptic table that helps readers compare methods. However, the practical-guide findings in Section 4.1 rest on informal cross-paper comparisons rather than on any controlled evidence, which substantially limits the reliability of the guidance. The paper also makes strong completeness claims without providing a systematic selection protocol. These issues reduce the significance of the survey as a practical reference unless they are addressed.
major comments (2)
- [Section 4.1 and Section 3.4] The comparative findings in Section 4.1 (e.g., 'Generative models tend to perform better compared to knowledge distillation methods' and 'adaptive ensemble methods typically yield better results') are presented as general conclusions but are derived from informal reading of papers that differ in datasets, model architectures, communication constraints, and evaluation protocols. No quantitative synthesis or unified benchmark is provided. The comparisons are confounded: generative methods such as FedDISC, FedBiP, and FedDEO use pretrained foundation models (CLIP, latent diffusion), whereas distillation methods like FedKT and One-shot FL do not, so the observed advantage may stem from the foundation model rather than the OFL technique. Similarly, adaptive ensemble methods such as IntactOFL and Co-Boosting also introduce data generation or mixture-of-experts, so the isolated effect of adaptivity is unidentified. Because the paper's stated purpose is to 'provide guidance and insights for future research' (Section 1), these findings need to be either supported by a controlled comparison or qualified with explicit caveats about the confounds and the limited comparability of the underlying experiments.
- [Sections 1, 3.5, and 5] The paper repeatedly claims to cover 'all current methods of OFL' (Section 3.5), to be 'the first survey paper specifically focused on the OFL domain' (Section 1), and to 'comprehensively cover all existing OFL literature' (Section 5). However, the manuscript does not describe a systematic search protocol, inclusion criteria, or the period of coverage. Without such a methodology, the completeness claims are not verifiable. The authors should either add a methodology subsection describing how papers were identified and selected, or temper the claims to describe a representative overview rather than an exhaustive one.
minor comments (7)
- [Section 2.1, Eq. (1)] The problem formulation in Eq. (1) is not mathematically well-defined: as written, α_i multiplies the entire sum (and α_i is indexed by i but appears outside the sum), and the relationship between the global parameter w and the local parameters w_i is not specified. Please rewrite the objective so that α_i appropriately weights each local loss and state how w is obtained from {w_i}.
- [Section 2.2] The phrase 'This challenge is essentially an out-of-distribution (OOD) detection problem' is imprecise, since OOD detection usually refers to detecting samples outside the training distribution at inference time, whereas the issue here is about generalization under distribution shift. Consider rephrasing.
- [Section 3.1] The description of k-FED is ambiguous about what exactly is clustered (i.e., local model parameters) and what the server does with the cluster means. Please clarify.
- [Section 4.1] The text refers to 'FENSE' but the cited method is 'FENS' (Allouah et al., 2024). Please correct the typo.
- [Table 1] The row for FedDEO does not list 'image captions' under 'Additional Data', even though Section 3.3 states that the method transmits image captions. Please make the table consistent with the text.
- [References] The entries '[Liu and Zeng, 2024 ]' and '[Shah and Lau, 2021 ]' contain spaces before the closing bracket; please fix the formatting.
- [Section 1] The citation to Alemdar et al. (2021) to support the claim that OFL 'alleviates the requirements for transmission synchronization' seems out of place, as that reference concerns RF clock synchronization in wireless networks rather than OFL. Please either clarify the connection or replace the citation.
Circularity Check
No significant circularity: the survey summarizes prior work rather than deriving predictions from its own inputs, and the few self-citations are not load-bearing.
full rationale
This is a survey and taxonomy paper, not a derivation chain. It contains no fitted parameters, no equations that are solved to produce a 'prediction,' and no first-principles result that is later shown to be equivalent to its own assumptions. The Section 4.1 findings, such as 'Generative models tend to perform better compared to knowledge distillation methods' and 'adaptive ensemble methods typically yield better results,' are qualitative generalizations from the cited experimental literature. They are not forced by the survey's taxonomy by construction; they are empirical claims whose reliability depends on the comparability of heterogeneous source experiments. That is a correctness/evidence concern, not circularity, and the hard rules explicitly say that 'this is not standard consensus' or unsupported comparison is not a circularity argument. Some cited works are co-authored by the present survey's authors (e.g., FuseFL, HPFL, FedLPA, Mediator, and Co-Boosting shares an author with the survey). However, these citations are used as examples within a broader literature review, and the adaptive-ensemble finding, for instance, is also supported by external work such as FENS and IntactOFL. The survey does not invoke a uniqueness theorem, smuggle in an ansatz via self-citation, or rename a known result as a new derivation. Therefore, the central contribution—a dedicated OFL taxonomy and practical overview—is self-contained as a literature-organizing effort. The only caveat is that some comparative findings rest on informally aggregated cross-paper results, which lowers evidential strength but does not constitute circular dependence between the survey's inputs and outputs.
Assumptions & free parameters
assumptions (4)
- ad hoc to paper One-shot federated learning methods can be meaningfully divided into four technique groups: parameter learning, knowledge distillation, generative models, and ensemble methods.
- domain assumption Performance comparisons across methods from different papers are comparable enough to support the findings in Section 4.1.
- domain assumption The two central challenges of OFL are data heterogeneity and model heterogeneity.
- ad hoc to paper The set of surveyed methods constitutes all current OFL methods as claimed in Section 3.5.
Cite this review
Pith. "Pith review of One-shot Federated Learning Methods: A Practical Guide." pith.science (2026). https://pith.science/paper/XTHTQU3U
@misc{pith2026250209104,
author = {Pith},
title = {Pith review of: One-shot Federated Learning Methods: A Practical Guide},
year = {2026},
howpublished = {\url{https://pith.science/paper/XTHTQU3U}},
note = {Machine review of arXiv:2502.09104}
}
read the original abstract
One-shot Federated Learning (OFL) is a distributed machine learning paradigm that constrains client-server communication to a single round, addressing privacy and communication overhead issues associated with multiple rounds of data exchange in traditional Federated Learning (FL). OFL demonstrates the practical potential for integration with future approaches that require collaborative training models, such as large language models (LLMs). However, current OFL methods face two major challenges: data heterogeneity and model heterogeneity, which result in subpar performance compared to conventional FL methods. Worse still, despite numerous studies addressing these limitations, a comprehensive summary is still lacking. To address these gaps, this paper presents a systematic analysis of the challenges faced by OFL and thoroughly reviews the current methods. We also offer an innovative categorization method and analyze the trade-offs of various techniques. Additionally, we discuss the most promising future directions and the technologies that should be integrated into the OFL field. This work aims to provide guidance and insights for future research.
Figures
Reference graph
Works this paper leans on
-
[1]
Rfclock: Timing, phase and frequency synchroniza- tion for distributed wireless networks
[Alemdar et al., 2021] Kubra Alemdar, Divashree V arshney, Subhramoy Mohanti, Ufuk Muncuk, and Kaushik Chowd- hury. Rfclock: Timing, phase and frequency synchroniza- tion for distributed wireless networks. In MobiCom,
work page 2021
-
[4]
Practical secure aggregation for privacy-preservin g machine learning
[Bonawitz et al., 2017] Keith Bonawitz, Vladimir Ivanov, et al. Practical secure aggregation for privacy-preservin g machine learning. In CCS,
work page 2017
-
[6]
Homomorphic encryption for arithmetic of approximate numbers
[Cheon et al., 2017] Jung Hee Cheon, Andrey Kim, Miran Kim, and Y ongsoo Song. Homomorphic encryption for arithmetic of approximate numbers. In Asiacrypt,
work page 2017
-
[7]
Enhancing one- shot federated learning through data and ensemble co- boosting
[Dai et al., 2024] Rong Dai, Y onggang Zhang, Ang Li, Tongliang Liu, Xun Y ang, and Bo Han. Enhancing one- shot federated learning through data and ensemble co- boosting. ICLR,
work page 2024
-
[8]
Heterogeneity for the win: One-shot feder- ated clustering
[Dennis et al., 2021] Don Kurian Dennis, Tian Li, and Vir- ginia Smith. Heterogeneity for the win: One-shot feder- ated clustering. In ICML,
work page 2021
-
[9]
Heterofl: Computation and communication efficient feder- ated learning for heterogeneous clients
[Diao et al., 2021] Enmao Diao, Jie Ding, and V ahid Tarokh. Heterofl: Computation and communication efficient feder- ated learning for heterogeneous clients. In ICLR,
work page 2021
-
[10]
Towards addressing label skews in one-shot federated learning
[Diao et al., 2023] Yiqun Diao, Qinbin Li, and Bingsheng He. Towards addressing label skews in one-shot federated learning. In ICLR,
work page 2023
-
[14]
[Guha et al., 2019] Neel Guha, Ameet Talwalkar, and Vir- ginia Smith. One-shot federated learning. arXiv preprint arXiv:1902.11175,
arXiv 2019
Show all 54 references
-
[17]
Data-free one-shot federated learning un - der very high statistical heterogeneity
[Heinbaugh et al., 2023] Clare Elizabeth Heinbaugh, Emilio Luz-Ricca, et al. Data-free one-shot federated learning un - der very high statistical heterogeneity. In ICLR,
2023
-
[18]
Practical hybrid gradient compression for federated learn - ing systems
[Hu et al., 2024] Sixu Hu, Linshan Jiang, and Bingsheng He. Practical hybrid gradient compression for federated learn - ing systems. In IJCAI,
2024
-
[19]
One-shot federated conformal prediction
[Humbert et al., 2023] Pierre Humbert, Batiste Le Bars, Aur´ elien Bellet, and Sylvain Arlot. One-shot federated conformal prediction. In ICML,
2023
-
[20]
Adaptive mix- tures of local experts
[Jacobs et al., 1991] Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mix- tures of local experts. Neural computation,
1991
-
[22]
Ofl-w3: A one-shot federated learning system on web 3.0
[Jiang et al., 2024] Linshan Jiang, Moming Duan, Bing- sheng He, Y ulin Sun, Peishen Y an, Y ang Hua, and Tao Song. Ofl-w3: A one-shot federated learning system on web 3.0. VLDB,
2024
-
[23]
Advances and open problems in federated learning
[Kairouz et al., 2021] Peter Kairouz, H Brendan McMahan, et al. Advances and open problems in federated learning. F oundations and trends in machine learning,
2021
-
[24]
One-shot federated learning on medi- cal data using knowledge distillation with image synthesis and client model adaptation
[Kang et al., 2023] Myeongkyun Kang, Philip Chikontwe, Soopil Kim, et al. One-shot federated learning on medi- cal data using knowledge distillation with image synthesis and client model adaptation. In MICCAI,
2023
-
[25]
Scaffold: Stochastic controlled averaging for federated learning
[Karimireddy et al., 2020] Sai Praneeth Karimireddy, Satyen Kale, et al. Scaffold: Stochastic controlled averaging for federated learning. In ICML,
2020
-
[27]
Osgan: One-shot distributed learning using generative adversari al networks
[Kasturi, 2023] Chittaranjan Kasturi, Anirudh; Hota. Osgan: One-shot distributed learning using generative adversari al networks. Journal of Supercomputing,
2023
-
[28]
Mediator: Memory-efficient llm merging with less parameter conflicts and uncertainty based routing
[Lai et al., 2025] Kunfeng Lai, Zhenheng Tang, Xinglin Pan, Peijie Dong, and et al. Mediator: Memory-efficient llm merging with less parameter conflicts and uncertainty based routing. arXiv preprint,
2025
-
[29]
Federated optimization in heterogeneous networks
[Li et al., 2020] Tian Li, Anit Kumar Sahu, et al. Federated optimization in heterogeneous networks. MLSys,
2020
-
[30]
Federated learning on non-iid data silos: An experimental study
[Li et al., 2022] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In ICDE,
2022
-
[31]
Accel- erate one-shot federated learning through ensemble opti- mization
[Liu and Zeng, 2024 ] Zhen Liu and Qingmin Zeng. Accel- erate one-shot federated learning through ensemble opti- mization. In AINIT,
2024
-
[32]
Federated learning of deep networks using model averaging
[McMahan et al., 2016] H Brendan McMahan, Eider Moore, et al. Federated learning of deep networks using model averaging. arXiv:1602.05629,
2016 arXiv
-
[34]
FedTMOS: Efficient one-shot federated learning with tsetlin machine
[Qi et al., 2025] Shannon How Shi Qi, Jagmohan Chauhan, et al. FedTMOS: Efficient one-shot federated learning with tsetlin machine. In ICLR,
2025
-
[35]
Learning transferable visual models from natural language supervision
[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, et al. Learning transferable visual models from natural language supervision. In ICML,
2021
-
[36]
High-resolution image synthesis with latent diffusion models
[Rombach et al., 2022] Robin Rombach, Andreas Blattmann, et al. High-resolution image synthesis with latent diffusion models. In CVPR,
2022
-
[37]
[Shah and Lau, 2021 ] Suhail Mohmad Shah and Vincent K. N. Lau. Model compression for communication effi- cient federated learning. IEEE TNNLS,
2021
-
[38]
Bridging general and personalized federated learning through selec - tive model integration
[Shen et al., 2025] Lei Shen, Zhenheng Tang, et al. Bridging general and personalized federated learning through selec - tive model integration. In ICLR,
2025
-
[39]
Xor mixup: Privacy- preserving data augmentation for one-shot federated learning
[Shin et al., 2020] MyungJae Shin, Chihoon Hwang, Joongheon Kim, Jihong Park, et al. Xor mixup: Privacy- preserving data augmentation for one-shot federated learning. arXiv:2006.05148,
2020 arXiv
-
[40]
Ef- fective heterogeneous federated learning via efficient hypernetwork-based weight generation
[Shin et al., 2024] Y ujin Shin, Kichang Lee, et al. Ef- fective heterogeneous federated learning via efficient hypernetwork-based weight generation. In SenSys,
2024
-
[41]
Federated multi-task learning
[Smith et al., 2017] Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. Federated multi-task learning. In NeurIPS,
2017
-
[42]
Proto- typical networks for few-shot learning
[Snell et al., 2017] Jake Snell, Kevin Swersky, et al. Proto- typical networks for few-shot learning. NeurIPS,
2017
-
[43]
Federated learning via decentralized dataset distillation in resour ce- constrained edge environments
[Song et al., 2023] Rui Song, Dai Liu, et al. Federated learning via decentralized dataset distillation in resour ce- constrained edge environments. In IJCNN,
2023
-
[44]
One-shot federated learning without server-side training
[Su et al., 2023] Shangchao Su, Bin Li, and Xiangyang Xue. One-shot federated learning without server-side training . Neural Networks, 164:203–215,
2023
-
[45]
Virtual homogeneity learning: Defend- ing against data heterogeneity in federated learning
[Tang et al., 2022] Zhenheng Tang, Y onggang Zhang, Shao- huai Shi, and et al. Virtual homogeneity learning: Defend- ing against data heterogeneity in federated learning. In ICML,
2022
-
[46]
Federated learning with matched averaging
[Wang et al., 2020] Hongyi Wang, Mikhail Y urochkin, Y uekai Sun, et al. Federated learning with matched averaging. ICLR,
2020
-
[47]
Data-free diversity-based ensemble selection for one-shot federate d learning
[Wang et al., 2023] Naibo Wang, Wenjie Feng, Moming Duan, Fusheng Liu, See-Kiong Ng, et al. Data-free diversity-based ensemble selection for one-shot federate d learning. TMLR,
2023
-
[48]
Mixture-of-agents enhances large language model capabilities
[Wang et al., 2024a] Junlin Wang, Jue Wang, Ben Athi- waratkun, Ce Zhang, and James Zou. Mixture-of-agents enhances large language model capabilities. arXiv preprint arXiv:2406.04692,
-
[49]
A survey on efficient federated learning methods for foundation model training
[Woisetschlager and others, 2024] Herbert Woisetschlager et al. A survey on efficient federated learning methods for foundation model training. IJCAI,
2024
-
[50]
Open- fedllm: Training large language models on decentralized private data via federated learning
[Y eet al., 2024] Rui Y e, Wenhao Wang, and et al. Open- fedllm: Training large language models on decentralized private data via federated learning. In SIGKDD,
2024
-
[51]
Bayesian nonparametric federated learning of neural networks
[Y urochkinet al., 2019] Mikhail Y urochkin, Mayank Agar- wal, et al. Bayesian nonparametric federated learning of neural networks. In ICML,
2019
-
[52]
One-shot- but-not-degraded federated learning
[Zeng et al., 2024] Hui Zeng, Minrui Xu, et al. One-shot- but-not-degraded federated learning. MM,
2024
-
[53]
Dense: Data-free one-shot federated learning
[Zhang et al., 2022] Jie Zhang, Chen Chen, et al. Dense: Data-free one-shot federated learning. NeurIPS,
2022
-
[54]
Distilled one-shot federated learning
[Zhou et al., 2020] Y anlin Zhou, George Pu, et al. Distilled one-shot federated learning. arXiv:2009.07999, 2020
2020 arXiv
-
[1991]
Towards a theoretical and practical understanding of one-shot federated learnin g with fisher information
[Jhunjhunwala et al., 2023] Divyansh Jhunjhunwala, Shiqiang Wang, and Gauri Joshi. Towards a theoretical and practical understanding of one-shot federated learnin g with fisher information. In ICML W orkshop FL,
2023
-
[2011]
Preserving privacy in federated learning with ensemble cross-domain knowledge distillation
[Gong et al., 2022] Xuan Gong, Abhishek Sharma, et al. Preserving privacy in federated learning with ensemble cross-domain knowledge distillation. In AAAI,
2022
-
[2016]
Knowledge distillation for fed - erated learning: a practical guide
[Mora et al., 2024] Alessio Mora, Irene Tenison, Paolo Bellavista, and Irina Rish. Knowledge distillation for fed - erated learning: a practical guide. IJCAI,
2024
-
[2017]
Fedbip: Heterogeneous one-shot federated learning with personal- ized latent diffusion models
[Chen et al., 2024] Haokun Chen, Hang Li, et al. Fedbip: Heterogeneous one-shot federated learning with personal- ized latent diffusion models. arXiv:2410.04810,
2024 arXiv
-
[2019]
Exploring one-shot federated learning by model inversion and token relabel with vision transformers,
[Hao et al., 2025] Xiaolei Hao, Li Shen, et al. Exploring one-shot federated learning by model inversion and token relabel with vision transformers,
2025
-
[2020]
Fusion learning: A one shot feder- ated learning
[Kasturi et al., 2020] Anirudh Kasturi, Anish Reddy Ellore, and Chittaranjan Hota. Fusion learning: A one shot feder- ated learning. In ICCS,
2020
-
[2021]
Revisiting ensembling in one-shot federated learn- ing
[Allouah et al., 2024] Y oussef Allouah, Akash Dhasade, et al. Revisiting ensembling in one-shot federated learn- ing. NeurIPS,
2024
-
[2022]
MiniLLM: Knowledge distillation of large language models
[Gu et al., 2024] Y uxian Gu, Li Dong, and et al. MiniLLM: Knowledge distillation of large language models. In ICLR,
2024
-
[2023]
Differential privacy
[Dwork, 2011] Cynthia Dwork. Differential privacy. Ency- clopedia of Cryptography and Security ,
2011
-
[2024]
Parametric feature transfer: One-shot federated learning with foundation models
[Beitollahi et al., 2024] Mahdi Beitollahi, Alex Bie, et al. Parametric feature transfer: One-shot federated learning with foundation models. arXiv:2402.01862,
2024 arXiv
-
[2025]
Calibrated one round federated learning with bayesian inference in the predic- tive space
[Hasan et al., 2024] Mohsin Hasan, Guojun Zhang, Kaiyang Guo, Xi Chen, and Pascal Poupart. Calibrated one round federated learning with bayesian inference in the predic- tive space. In AAAI,
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.