Pith. sign in

REVIEW 17 cited by

FedMD: Heterogenous Federated Learning via Model Distillation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1910.03581 v1 pith:MK64VU4S submitted 2019-10-08 cs.LG stat.ML

Daliang Li , Junpu Wang This is my paper
classification cs.LGstat.ML
keywords learningmodelfederateddataparticipantsdatasetdistillationenables
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Federated learning enables the creation of a powerful centralized model without compromising data privacy of multiple participants. While successful, it does not incorporate the case where each participant independently designs its own model. Due to intellectual property concerns and heterogeneous nature of tasks and data, this is a widespread requirement in applications of federated learning to areas such as health care and AI as a service. In this work, we use transfer learning and knowledge distillation to develop a universal framework that enables federated learning when each agent owns not only their private data, but also uniquely designed models. We test our framework on the MNIST/FEMNIST dataset and the CIFAR10/CIFAR100 dataset and observe fast improvement across all participating models. With 10 distinct participants, the final test accuracy of each model on average receives a 20% gain on top of what's possible without collaboration and is only a few percent lower than the performance each model would have obtained if all private datasets were pooled and made directly available for all participants.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 480 citations worldwide. Full citation record

  1. AS-FedBridge: Pseudo-Spike Bridge Distillation for Heterogeneous ANN-SNN Federated Learning

    cs.LG 2026-08 conditional novelty 6.0 of 10

    AS-FedBridge trains a shared pseudo-spike bridge so ANN and SNN clients in federated learning can align representations and improve collaborative accuracy.

  2. FedJigsaw: Multi-Agent Collaborative Model Reassembly for Decentralized Heterogeneous Federated Learning

    cs.DC 2026-08 reject novelty 6.0 of 10

    FedJigsaw replaces fixed subnet extraction in heterogeneous federated learning with decentralized, reinforcement-learned module assembly, reporting up to 13.87% relative accuracy gains over prior MHFL baselines.

  3. FedTopo: Relation-Level Topology Sharing for Model-Heterogeneous Federated Learning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Sharing class-relation topology with reliability weighting beats parameter, distillation, and prototype sharing under heterogeneous federated backbones on CIFAR and Tiny-ImageNet.

  4. Autonomous Collaborative Learning Among an Ensemble of Tsetlin Machines with Consensus-Based Inference

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A two-layer Tsetlin Machine ensemble with gossip-based vote sharing matches centralized accuracy on several benchmarks without exchanging raw data.

  5. Federated Lightweight Fine-Tuning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A federated fine-tuning method transmits only 1,280 latent floats per round and reaches near-FedAvg accuracy by exploiting the exact averaging identity of affine mapping networks.

  6. Flexible Personalized Split Federated Learning for On-Device Fine-Tuning of Foundation Models

    cs.DC 2025-08 conditional novelty 6.0 of 10

    FlexP-SFL fine-tunes foundation models on resource-constrained devices through personalized split learning without parameter aggregation, improving accuracy and cutting wall-clock time and communication versus federat...

  7. Collate: Collaborative Neural Network Learning for Latency-Critical Edge Systems

    cs.LG 2026-07 conditional novelty 5.5 of 10

    Collate jointly trains heterogeneous models under per-device latency constraints via dynamic zeroizing-recovering and proto-corrected aggregation, gaining ~2–3% accuracy over prior heterogeneous FL.

  8. Communication-Aware Knowledge Distillation for Federated LLM Fine-Tuning over Wireless Networks

    cs.LG 2025-09 conditional novelty 5.0 of 10

    A federated distillation scheme combining channel-aware Top-k logit sparsification, sparsity-aware aggregation, and LoRA projection alignment cuts communication by about 50% while improving fine-tuning accuracy in a G...

  9. Heterogeneity-Oblivious Robust Federated Learning

    cs.LG 2025-08 conditional novelty 5.0 of 10

    Horus makes federated learning robust to poisoning under extreme client heterogeneity by aggregating only LoRA adapters and detecting attackers from the spectral structure of the LoRA-A component.

  10. Measuring and Predicting Where and When Pathologists Focus their Visual Attention while Grading Whole Slide Images of Cancer

    eess.IV 2025-08 unverdicted novelty 5.0 of 10

    The submission combines an attention prediction abstract with a federated learning body, so the claimed result cannot be evaluated from the provided material.

  11. Hypernetworks for Model-Heterogeneous Personalized Federated Learning

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A server-side multi-head hypernetwork generates personalized parameters for clients with heterogeneous model architectures, plus an optional global-model distillation variant, and beats several pFL baselines on four b...

  12. TinyProto: Communication-Efficient Federated Learning with Sparse Prototypes in Resource-Constrained Environments

    cs.LG 2025-07 conditional novelty 5.0 of 10

    Compressing class prototypes with per-class masks and a sample-count scaling trick cuts communication cost in prototype-based federated learning by up to several times without hurting accuracy.

  13. FedBKD: Distilled Federated Learning to Embrace Gerneralization and Personalization on Non-IID Data

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A data-free GAN plus bidirectional knowledge distillation between global and local models improves both personalization and generalization in non-IID federated classification.

  14. Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocation

    cs.LG 2025-06 conditional novelty 5.0 of 10

    Fed-HeLLo allocates different LoRA layers to clients of different resource levels using importance scores and geometric patterns, improving federated fine-tuning accuracy over random allocation baselines.

  15. Generalizable Federated Learning using Client Adaptive Focal Modulation

    cs.CV 2025-08 reject novelty 4.0 of 10

    The abstract describes AdaptFED, a claimed federated learning method, but the full text is an unrelated graph theory paper, so the claimed results are absent from the submission.

  16. Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization

    cs.LG 2025-09 conditional novelty 3.0 of 10

    A PhD dissertation showing unified compression theory, personalized accelerated local training, and pruning methods that reduce communication costs in federated learning and maintain accuracy in LLM pruning.

  17. Heterogeneous Federated Learning with Prototype Alignment and Upscaling

    cs.LG 2025-07 conditional novelty 3.0 of 10

    ProtoNorm adds server-side prototype alignment and a per-dataset scaling factor to FedProto-style federated learning, improving accuracy but with the gain largely driven by the tuned scaling factor.

Pith tools