Pith. sign in

REVIEW 4 major objections 3 minor 36 references

Tackling Data Heterogeneity in Federated Learning through Knowledge Distillation with Inequitable Aggregation

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read KDIA claims that, when only a small fraction of many clients are sampled, aggregating a teacher model from all clients—weighted by participation interval, participation count, and data volume—lets that teacher guide the participating…

desk verdict Plausible empirical method for large-N/small-C FL, but the O(1/T) proof is borrowed from FedProx without meeting its conditions, and the evaluation is a bit generous. read the letter →

arxiv 2506.20431 v1 pith:L6VPSHZC submitted 2025-06-25 cs.LG

Xing Ma This is my paper
classification cs.LG
keywords FederatedLearningDataHeterogeneityKnowledgeDistillationConditionalGeneratorTeacher-StudentAggregationClientSamplingCommunicationEfficiencyLabelSkew
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes KDIA, a federated learning method aimed at the realistic setting where many clients exist but only a small fraction are sampled each round. It claims that in this setting, building a teacher model from all clients—weighted by how recently and how often each client participated and by its data volume—lets the teacher guide the few participating student models through knowledge distillation. The author argues this leverages knowledge from non-participating clients, and experiments on CIFAR-10/100 and CINIC-10 with Dirichlet label skew show better accuracy in fewer communication rounds than FedAvg, FedProx, MOON, FedGKD, FedDisco, and FedCOG. A convergence analysis claims an O(1/T) rate. If right, communication-efficient federated learning under realistic sampling can improve without requiring more clients to participate.

What carries the argument

The central mechanism is the triFreqs weighting scheme: for each client, three frequencies are computed—participation interval (an exponentially decaying score based on how many rounds since the client last participated), participation count (the share of total participation events), and data volume proportion—and their geometric mean is normalized to form the teacher aggregation weight. This teacher model, aggregated over all clients, is then used in self-knowledge distillation with a temperature-scaled KL divergence loss, while a conditional generator trained on the server produces approximately class-balanced features for auxiliary local training. The geometric mean is chosen because it is less influenced by extreme values than arithmetic or harmonic means and avoids large oscillations in the teacher updates.

What would settle it

Run KDIA and FedAvg under N=100, C=0.1, beta=0.1 with the same seeds, recording per-round test accuracy and the squared gradient norm of the global objective; if KDIA's teacher does not reach FedAvg's best accuracy in fewer rounds, or if the gradient norm does not decay at roughly a 1/T rate, the central claim fails. Alternatively, remove the participation-interval and participation-count terms from triFreqs, keeping only data-volume weighting; if accuracy then drops to the FedKF level, the triFreqs mechanism is the source of the gain.

Watch

Extended reading notes

Core claim

On the paper's own terms, KDIA establishes that inequitable aggregation—deliberately weighting clients unequally in the teacher model by three frequencies—is better than equitable aggregation or data-volume-only aggregation. The student model remains a FedAvg-style average of the sampled clients, while the teacher model is a weighted aggregation of all clients using the geometric mean of participation intervals, participation counts, and data volume proportions. During local training, the student performs self-knowledge distillation against the teacher and also trains on approximately IID features produced by a server-trained conditional generator. The empirical claim is that under large client counts and low sampling ratios, the teacher model consistently outperforms the student and the baseline methods, reaching given accuracy targets in fewer rounds—for example, 2.5x acceleration on CIFAR-10 with beta = 0.1—and the paper further claims the final convergence rate is O(1/T).

Load-bearing premise

The convergence proof assumes that FedProx's assumptions—bounded dissimilarity and gamma-inexact local solutions—carry over to KDIA's three-term objective, but the paper does not show that these conditions hold for that objective, so the O(1/T) guarantee and the claimed stability rest on that unverified transfer.

Editorial extensions

If this is right

  • Under large-N, small-C conditions, model accuracy can improve without raising the sampling ratio, so communication overhead can be cut while keeping or improving performance.
  • The teacher model's stable convergence suggests that distillation from a slowly but consistently updated all-client ensemble reduces the oscillation that FedAvg and MOON show at low sampling ratios.
  • The adjusted conditional generator, which pre-samples labels and shuffles them, produces features closer to real intermediate features and helps clients with very small data volumes.
  • If the transferred convergence argument holds, the O(1/T) rate supports the observed training stability and gives a theoretical basis for the method's communication efficiency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The triFreqs weighting recipe could be applied to other federated learning components, such as client selection or fairness-aware aggregation, although the paper does not test those uses.
  • The O(1/T) guarantee is transferred from FedProx rather than derived for KDIA's three-term objective; a direct proof or empirical gradient-norm tracking would be needed to fully trust the rate.
  • The method's gains are largest under extreme label skew, and on near-IID CINIC-10 with beta = 5.0 the teacher can underperform MOON, suggesting the benefit is specifically heterogeneity compensation rather than a universal accuracy boost.
  • A standalone test of the label-shuffled generator without distillation would isolate its contribution, since the ablation shows generator loss alone can hurt accuracy on two datasets while helping when combined with distillation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The manuscript proposes KDIA, a federated learning method aimed at the large-client, low-participation regime (large N, small C). A teacher model is aggregated on the server from all clients using a geometric-mean combination of participation-interval, participation-count, and data-volume frequencies, while a student model is formed by the standard weighted average of the sampled clients. During local training, clients optimize cross-entropy plus temperature-scaled KL distillation toward the teacher plus a generator-augmented loss, where the generator is trained on the server and distributed to clients. Experiments on CIFAR-10, CIFAR-100, and CINIC-10 under Dirichlet heterogeneity compare KDIA against seven baselines and report accuracy gains, faster convergence in communication rounds, and ablations. The paper also claims an O(1/T) convergence rate for KDIA.

Significance. The empirical setting addressed by the paper, large N with low participation C, is realistic and comparatively under-studied, and the proposed triFreqs teacher aggregation is a concrete mechanism for using knowledge from non-participating clients. The authors release code, report standard deviations over three seeds, include ablations for the distillation and generator components, and document computational and communication costs. If the empirical claims hold, KDIA would be a useful contribution to the large-client, low-participation federated learning regime; however, the formal convergence guarantee is not established for the actual objective, and the main experimental comparisons are weakened by test-set-based hyperparameter and model selection. The empirical content is stronger than the formal claims as written.

major comments (4)
  1. [5.4, Eq. (14), Theorem 1] The formal convergence claim is not supported. Theorem 1 states that the proof follows from Theorem 4 of FedProx, but the KDIA local objective in Eq. (14) is not the FedProx proximal objective: it contains a KL-divergence term whose gradient with respect to θ_k is not the asserted μ L_h/δ (θ_k − θ_T), so the γ-inexactness condition in Assumption 2.1 is asserted without derivation. Assumption 1.2 bounds the dissimilarity of L_CE only, not of L_CE + L_KL, and the teacher θ_T is a moving, stale aggregation target rather than the fixed global model in FedProx. Consequently, the inequality E[L(θ_{t+1})] ≤ L(θ_t) − ρ‖∇L(θ_t)‖² and the final O(1/T) rate do not follow from the cited proof. The authors should either prove the rate for the actual objective minimized in Eq. (13) and Algorithm 2, or remove the convergence theorem and frame the efficiency claims as empirical.
  2. [6.4, Fig. 8; 5.1, Algorithm 1] Key hyperparameters are selected on the test set, and the final reported model is also selected by test performance. Section 6.4 chooses λ_gen separately for each β (0.01/1.0/0.01 for β = 0.1/0.5/5.0) from accuracy curves on the test data, while the baselines use fixed defaults; Algorithm 1 outputs 'θ_g or θ_T (selected by performance)' and Section 5.1 says the final target model is chosen based on superior performance. This is oracle selection on the evaluation metric. The main accuracy comparisons in Table 2 should be reported under fixed hyperparameters and a model-selection rule based on validation data only, or the reported gains should be framed accordingly.
  3. [Table 4] The communication-efficiency table appears internally inconsistent with Table 2. Table 4 reports a target accuracy of 64.07% for CIFAR-10 in all three β columns, but Table 2 reports FedAvg reaching only 50.41% at β = 0.1 and 64.36% at β = 0.5 on CIFAR-10; the value 64.07 matches the N = 100, C = 0.1, β = 0.5 FedAvg entry in Table 5. If the target is meant to be FedAvg's own final accuracy per heterogeneity level, the β columns should have different targets, and at β = 0.1 no baseline in Table 2 reaches 64.07%. Please correct the table and recompute the speedups, or clarify which setting the target applies to.
  4. [Eq. (13) vs Eq. (14)] The theoretical objective used in the convergence analysis is not the actual training loss. Eq. (13) consists of l_CE + λ_kd l_kd + λ_gen l_gen, but Eq. (14) drops λ_kd and λ_gen and instead introduces a μ/(2N_k)-weighted KL average without defining μ in terms of λ_kd, λ_gen, or N_k; the generator term is then folded into L_CE without comment. The convergence analysis should be stated for the exact objective minimized in Algorithm 2, with all coefficients defined and with the same loss terms appearing in the assumptions and in the theorem.
minor comments (3)
  1. [Throughout] There are numerous typographical and reference inconsistencies: 'tirFreqs' is used for 'triFreqs' in several places; Eq. (6) writes Fintν_k with a Greek ν instead of a superscript; Fig. 6(c) is labeled CINIC-100 instead of CINIC-10; Tables 2 and 4 cite MOON as [6] and FedGKD as [3], while the text cites them as [7] and [24] respectively.
  2. [Algorithm 2, line 2] When N_k < N_b, Algorithm 2 sets N_k = E × N_k, overwriting the client's true data size with the total number of generated samples across epochs. This is confusing because the same symbol N_k is used for data-volume weighting in Eq. (8); please use a separate variable for the number of generated auxiliary samples so that the weighting in Eq. (8) remains clearly based on original data counts.
  3. [6.1, 6.4] The hyperparameter selection procedure is described vaguely: Section 6.1 fixes λ_kd = 0.5 and gives per-β values of λ_gen, while Section 6.4 says λ_gen is selected to be 'stable with changes in λ_kd' without stating the selection criterion or using a validation split. Please specify the exact protocol.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: KDIA's accuracy and speedup results are empirical measurements, and the FedProx-based convergence proof is an external-citation support gap, not a circular reduction.

full rationale

I walked the derivation chain and found no step in which a claimed result is equivalent to its inputs by construction. The central performance claims are experimental: Table 2 reports three-seed test accuracies, Table 4 reports measured communication rounds to reach a fixed target accuracy, and Table 7 reports ablation measurements; none of these quantities is generated from the method's own fitted parameters or from the triFreqs weights. The triFreqs teacher aggregation in Eq. (10) is a weighted average defined from participation interval, participation count, and data-volume frequencies; the accuracy improvements attributed to it are independently measured, not entailed by that definition. The only formal derivation, Theorem 1 in Section 5.4, delegates its proof to Theorem 4 of FedProx [13]. This is a non-overlapping external citation, not a self-citation, and the issue is a support gap rather than circularity: the local objective in Eq. (14) includes a KL distillation term whose gradient is asserted as (μL_h/δ)(θ_k^t - θ_T^t) without derivation, and the γ-inexactness condition is stated for the proximal form and then transferred to the KDIA objective. These are unverified assumptions about whether FedProx's proof applies, not a case where the conclusion is presupposed by the method's definitions. I therefore find no self-definitional reduction, no fitted input renamed as a prediction, and no load-bearing self-citation; the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The method relies on standard FedProx/FedGKD smoothness and bounded-dissimilarity assumptions, and on the unverified premise that FedProx's proof transfers to the three-term KDIA objective. The hyperparameters lambda_kd, lambda_gen, and tau are tuned on the test set for each heterogeneity level, adding a fitting burden.

free parameters (3)
  • lambda_kd = 0.5 (tuned from {0.1, 0.5, 1.0})
    Weight of the self-knowledge distillation loss in Eq. (13); selected based on CIFAR-10 experiments in Fig. 8(a-c) and fixed to 0.5 for all settings.
  • lambda_gen = 0.01 (beta=0.1), 1.0 (beta=0.5), 0.01 (beta=5.0)
    Weight of the generator auxiliary loss in Eq. (12); chosen per heterogeneity level to be 'stable with changes in lambda_kd' (Section 6.4), effectively tuned on the test set.
  • tau = 2.0 (tuned from {1.0, 2.0, 5.0, 7.0, 9.0})
    Distillation temperature in Eq. (11); selected to maximize performance across heterogeneity levels as shown in Fig. 8(d).
assumptions (6)
  • standard math L-smoothness and lower-bounded Hessian eigenvalue of client loss (Assumption 1.1)
    Borrowed from FedProx to enable a descent-type convergence argument; stated but not proved for the KDIA loss.
  • domain assumption Bounded dissimilarity of client gradients (Assumption 1.2)
    Standard in heterogeneous FL analysis; assumes client gradient norms are controlled by the global gradient norm.
  • domain assumption Softmax outputs bounded below by delta and Lipschitz continuous (Assumption 1.3)
    Imported from FedGKD to bound the KL distillation term in Eq. (14).
  • domain assumption Gamma-inexact solution condition for the local update (Assumption 2.1)
    Assumes each client's local solve is a gamma-inexact minimizer of the proximal objective, but Algorithm 2 runs SGD on a three-term loss and no such guarantee is shown.
  • ad hoc to paper FedProx Theorem 4 proof carries over to the KDIA objective (Section 5.4)
    The paper explicitly says 'For the proof of Theorem 1, please refer to the proof of Theorem 4 in FedProx [13]' without deriving the specific conditions for the modified loss.
  • domain assumption Conditional generator trained on the server produces features that help local training (Section 5.3)
    The method assumes the server-trained generator approximates the real feature distribution well enough for the auxiliary loss in Eq. (12) to improve generalization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tackling Data Heterogeneity in Federated Learning through Knowledge Distillation with Inequitable Aggregation." pith.science (2026). https://pith.science/paper/L6VPSHZC

@misc{pith2026250620431,
  author       = {Pith},
  title        = {Pith review of: Tackling Data Heterogeneity in Federated Learning through Knowledge Distillation with Inequitable Aggregation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L6VPSHZC}},
  note         = {Machine review of arXiv:2506.20431}
}
read the original abstract

Federated learning aims to train a global model in a distributed environment that is close to the performance of centralized training. However, issues such as client label skew, data quantity skew, and other heterogeneity problems severely degrade the model's performance. Most existing methods overlook the scenario where only a small portion of clients participate in training within a large-scale client setting, whereas our experiments show that this scenario presents a more challenging federated learning task. Therefore, we propose a Knowledge Distillation with teacher-student Inequitable Aggregation (KDIA) strategy tailored to address the federated learning setting mentioned above, which can effectively leverage knowledge from all clients. In KDIA, the student model is the average aggregation of the participating clients, while the teacher model is formed by a weighted aggregation of all clients based on three frequencies: participation intervals, participation counts, and data volume proportions. During local training, self-knowledge distillation is performed. Additionally, we utilize a generator trained on the server to generate approximately independent and identically distributed (IID) data features locally for auxiliary training. We conduct extensive experiments on the CIFAR-10/100/CINIC-10 datasets and various heterogeneous settings to evaluate KDIA. The results show that KDIA can achieve better accuracy with fewer rounds of training, and the improvement is more significant under severe heterogeneity.

Figures

Figures reproduced from arXiv: 2506.20431 by the authors.

Figure 1
Figure 1. On the CIFAR-10 dataset, (a) shows the test accuracy curves of FedAvg and MOON with different numbers of clients N ; (b), (c), (d) show the test accuracy curves of FedAvg and MOON with a fixed N under different levels of heterogeneity, while varying the client sampling ratio C. C ” configuration sets a small N and a sampling ratio C = 1.0, which seems inconsistent with the characteristics of federated learning—parti… view at source ↗
Figure 2
Figure 2. The figure shows the overall architecture of KDIA, where the white section represents local updates and the blue section represents server-side aggregation. The generator θgen is trained on the server and then distributed to the clients, with the noise and label sampling methods being the same [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. The figure illustrates the different ways in which the teacher model is acquired in knowledge distillation. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Distribution of data under different Dir(β) partitions on the CIFAR-10 dataset. The x-axis represents the class index, the y-axis represents the client index, and the color bar indicates the sample size. training epochs, a batch size of 64, and 200 batches generated. I…
Figure 5
Figure 5. Figure 5: (a) shows the distribution of clients for sinFreqs (left) / triFreqs (right) at different levels of heterogeneity. (b) illustrates the training variation of client participation interval frequency variance, participation count frequency variance, and data proportion va…
Figure 6
Figure 6. Figure 6: (a)/(b)/(c) show the test accuracy curves for the CIFAR-10, CIFAR-100, and CINIC-10 datasets with β = 0.5, respectively. when the heterogeneity is lower, the participation interval frequency becomes the main factor, so the distribution range is larger than that of sinF…
Figure 7
Figure 7. Figure 7: Generated vs. real feature similarity under data heterogeneity in CIFAR-10. the feature extractor’s output space, and the model size, with computational expenses scaling proportionally to these factors. Thus, the server’s compu￾tational cost mainly stems from condition…
Figure 8
Figure 8. Figure 8: On the CIFAR-10 dataset, (a)/(b)/(c) show the accuracy improvements of KDIA relative to FedAvg under different λkd/λgen settings across three levels of data hetero￾geneity. (d) shows the performance improvements for different distillation temperatures τ across the thre…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 24 canonical work pages

  1. [1]

    Zhang, Y

    C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, Y. Gao, A survey on federated learning, Knowledge-Based Systems 216 (2021) 106775

    work page 2021

  2. [2]

    McMahan, E

    B. McMahan, E. Moore, D. Ramage, S. Hampson, B. A. y Arcas, Communication-efficient learning of deep networks from decentralized data, in: Artificial intelligence and statistics, PMLR, 2017, pp. 1273– 1282

    work page 2017

  3. [3]

    J. Wang, Q. Liu, H. Liang, G. Joshi, H. V. Poor, Tackling the ob- jective inconsistency problem in heterogeneous federated optimization, Advances in neural information processing systems 33 (2020) 7611–7623

    work page 2020

  4. [4]

    Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, V. Chandra, Federated learning with non-iid data, arXiv preprint arXiv:1806.00582 (2018)

    arXiv 2018

  5. [5]

    H. Zhu, J. Xu, S. Liu, Y. Jin, Federated learning on non-iid data: A survey, Neurocomputing 465 (2021) 371–390

    work page 2021

  6. [6]

    Q. Li, Y. Diao, Q. Chen, B. He, Federated learning on non-iid data silos: An experimental study, in: 2022 IEEE 38th international conference on data engineering (ICDE), IEEE, 2022, pp. 965–978. 30

    work page 2022

  7. [7]

    Q. Li, B. He, D. Song, Model-contrastive federated learning, in: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 10713–10722

    work page 2021

  8. [8]

    R. Ye, M. Xu, J. Wang, C. Xu, S. Chen, Y. Wang, Feddisco: Feder- ated learning with discrepancy-aware collaboration, in: International Conference on Machine Learning, PMLR, 2023, pp. 39879–39902

    work page 2023

Show all 36 references
  1. [9]

    H. M. Son, M.-H. Kim, T.-M. Chung, C. Huang, X. Liu, Feduv: Unifor- mity and variance for heterogeneous federated learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, 2024, pp. 5863–5872

    2024

  2. [10]

    Ribero, H

    M. Ribero, H. Vikalo, Communication-efficient federated learning via optimal client sampling. arxiv 2020, arXiv preprint arXiv:2007.15197 (2020)

    2020 arXiv

  3. [11]

    L. Fu, H. Zhang, G. Gao, M. Zhang, X. Liu, Client selection in federated learning: Principles, challenges, and opportunities, IEEE Internet of Things Journal (2023)

    2023

  4. [12]

    Z. Zhu, J. Hong, J. Zhou, Data-free knowledge distillation for heteroge- neous federated learning, in: International conference on machine learn- ing, PMLR, 2021, pp. 12878–12889

    2021

  5. [13]

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, V. Smith, Fed- erated optimization in heterogeneous networks, Proceedings of Machine learning and systems 2 (2020) 429–450

    2020

  6. [14]

    Li, D.-C

    X.-C. Li, D.-C. Zhan, Fedrs: Federated learning with restricted softmax for label distribution non-iid data, in: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, 2021, pp. 995–1005

    2021

  7. [15]

    Zhang, Y

    L. Zhang, Y. Luo, Y. Bai, B. Du, L.-Y. Duan, Federated learning for non- iid data via unified feature learning and optimization objective align- ment, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 4420–4428

    2021

  8. [16]

    L. Gao, H. Fu, L. Li, Y. Chen, M. Xu, C.-Z. Xu, Feddc: Federated learning with non-iid data via local drift decoupling and correction, in: 31 Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, 2022, pp. 10112–10121

    2022

  9. [17]

    Y. Shi, J. Liang, W. Zhang, V. Y. Tan, S. Bai, Towards understanding and mitigating dimensional collapse in heterogeneous federated learning, arXiv preprint arXiv:2210.00226 (2022)

    2022 arXiv

  10. [18]

    T.-M. H. Hsu, H. Qi, M. Brown, Measuring the effects of non-identical data distribution for federated visual classification. arxiv 2019, arXiv preprint arXiv:1909.06335 (2019)

    2019 arXiv

  11. [19]

    Zhang, L

    L. Zhang, L. Shen, L. Ding, D. Tao, L.-Y. Duan, Fine-tuning global model via data-free knowledge distillation for non-iid federated learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10174–10183

    2022

  12. [20]

    Hinton, Distilling the knowledge in a neural network, arXiv preprint arXiv:1503.02531 (2015)

    G. Hinton, Distilling the knowledge in a neural network, arXiv preprint arXiv:1503.02531 (2015)

    2015 arXiv

  13. [21]

    L. Qin, T. Zhu, W. Zhou, P. S. Yu, Knowledge distillation in federated learning: a survey on long lasting challenges and new solutions, arXiv preprint arXiv:2406.10861 (2024)

    2024 arXiv

  14. [22]

    J. Gou, B. Yu, S. J. Maybank, D. Tao, Knowledge distillation: A survey, International Journal of Computer Vision 129 (6) (2021) 1789–1819

    2021

  15. [23]

    H. Seo, J. Park, S. Oh, M. Bennis, S. Kim, Federated knowledge distil- lation, arXiv preprint arXiv:2011.02367 (2020)

    2020 arXiv

  16. [24]

    D. Yao, W. Pan, Y. Dai, Y. Wan, X. Ding, H. Jin, Z. Xu, L. Sun, Local- global knowledge distillation in heterogeneous federated learning with non-iid data, arXiv preprint arXiv:2107.00051 (2021)

    2021 arXiv

  17. [25]

    Xiong, R

    Y. Xiong, R. Wang, M. Cheng, F. Yu, C.-J. Hsieh, Feddm: Iterative distribution matching for communication-efficient federated learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, 2023, pp. 16323–16332

    2023

  18. [26]

    Z. Yang, Y. Zhang, Y. Zheng, X. Tian, H. Peng, T. Liu, B. Han, Fedfed: Feature distillation against data heterogeneity in federated learning, Ad- vances in Neural Information Processing Systems 36 (2024). 32

    2024

  19. [27]

    X. Zhou, X. Lei, C. Yang, Y. Shi, X. Zhang, J. Shi, Handling data heterogeneity in federated learning via knowledge fusion, arXiv preprint arXiv:2207.11447 (2022)

    2022 arXiv

  20. [28]

    Mirza, Conditional generative adversarial nets, arXiv preprint arXiv:1411.1784 (2014)

    M. Mirza, Conditional generative adversarial nets, arXiv preprint arXiv:1411.1784 (2014)

    2014 arXiv

  21. [29]

    Y. Wu, Y. Kang, J. Luo, Y. He, Q. Yang, Fedcg: Leverage conditional gan for protecting privacy and maintaining competitive performance in federated learning, arXiv preprint arXiv:2111.08211 (2021)

    2021 arXiv

  22. [30]

    Furlanello, Z

    T. Furlanello, Z. Lipton, M. Tschannen, L. Itti, A. Anandkumar, Born again neural networks, in: International conference on machine learning, PMLR, 2018, pp. 1607–1616

    2018

  23. [31]

    C. Yang, L. Xie, C. Su, A. L. Yuille, Snapshot distillation: Teacher- student optimization in one generation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2859–2868

    2019

  24. [32]

    L. Yuan, F. E. H. Tay, G. Li, T. Wang, J. Feng, Revisiting knowl- edge distillation via label smoothing regularization, arXiv preprint arXiv:1909.11723 (2021)

    2021 arXiv

  25. [33]

    H. Wang, M. Yurochkin, Y. Sun, D. S. Papailiopoulos, Y. Khaz- aeni, Federated learning with matched averaging, arXiv preprint arXiv:2002.06440 (2020)

    2020 arXiv

  26. [34]

    K. Kim, B. Ji, D. Yoon, S. Hwang, Self-knowledge distillation: A simple way for better generalization, arXiv preprint arXiv:2006.12000 3 (1) (2020)

    2020 arXiv

  27. [35]

    R. Ye, Y. Du, Z. Ni, Y. Wang, S. Chen, Fake it till make it: Federated learning with consensus-oriented generation, in: The Twelfth Interna- tional Conference on Learning Representations, 2024

    2024

  28. [36]

    Kairouz, H

    P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et al., Advances and open problems in federated learning, Foundations and trends® in machine learning 14 (1–2) (2021) 1–210. 33

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.