Pith. sign in

REVIEW 12 cited by

Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1811.11479 v2 pith:R45UJYHU submitted 2018-11-28 cs.LG cs.NIstat.ML

Eunjeong Jeong , Seungeun Oh , Hyesung Kim , Jihong Park , Mehdi Bennis , Seong-Lyun Kim This is my paper
classification cs.LGcs.NIstat.ML
keywords datafederatedcommunicationlearningmodelaugmentationcompareddataset
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

On-device machine learning (ML) enables the training process to exploit a massive amount of user-generated private data samples. To enjoy this benefit, inter-device communication overhead should be minimized. With this end, we propose federated distillation (FD), a distributed model training algorithm whose communication payload size is much smaller than a benchmark scheme, federated learning (FL), particularly when the model size is large. Moreover, user-generated data samples are likely to become non-IID across devices, which commonly degrades the performance compared to the case with an IID dataset. To cope with this, we propose federated augmentation (FAug), where each device collectively trains a generative model, and thereby augments its local data towards yielding an IID dataset. Empirical studies demonstrate that FD with FAug yields around 26x less communication overhead while achieving 95-98% test accuracy compared to FL.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FedTopo: Relation-Level Topology Sharing for Model-Heterogeneous Federated Learning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Sharing class-relation topology with reliability weighting beats parameter, distillation, and prototype sharing under heterogeneous federated backbones on CIFAR and Tiny-ImageNet.

  2. FedWCM: Unleashing the Potential of Momentum-based Federated Learning in Long-Tailed Scenarios

    cs.LG 2025-07 reject novelty 6.0 of 10

    FedWCM uses per-client data-distribution scores to adapt momentum and aggregation weights in federated learning, showing empirical gains over FedAvg and FedCM on long-tailed non-IID datasets, but its convergence proof...

  3. HtFLlib: A Comprehensive Heterogeneous Federated Learning Library and Benchmark

    cs.LG 2025-06 conditional novelty 6.0 of 10

    HtFLlib is a unified benchmark and library with 12 datasets, 40 heterogeneous model architectures, and systematic accuracy, convergence, and cost evaluations of 10 HtFL methods.

  4. Collate: Collaborative Neural Network Learning for Latency-Critical Edge Systems

    cs.LG 2026-07 conditional novelty 5.5 of 10

    Collate jointly trains heterogeneous models under per-device latency constraints via dynamic zeroizing-recovering and proto-corrected aggregation, gaining ~2–3% accuracy over prior heterogeneous FL.

  5. Enhancing Model Privacy in Federated Learning with Random Masking and Quantization

    cs.LG 2025-08 reject novelty 5.0 of 10

    FedQSN hides part of the server model with random masks and quantizes the remainder to give clients a degraded proxy, reporting a large global-vs-proxy performance gap with modest loss in the final global model.

  6. Hypernetworks for Model-Heterogeneous Personalized Federated Learning

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A server-side multi-head hypernetwork generates personalized parameters for clients with heterogeneous model architectures, plus an optional global-model distillation variant, and beats several pFL baselines on four b...

  7. Federated Learning for Commercial Image Sources

    cs.CV 2025-07 conditional novelty 5.0 of 10

    The authors present a new 31-class, 8-source image classification dataset for federated learning and show that Fed-Cyclic and Fed-Star beat FedAvg and RingFed on it.

  8. TinyProto: Communication-Efficient Federated Learning with Sparse Prototypes in Resource-Constrained Environments

    cs.LG 2025-07 conditional novelty 5.0 of 10

    Compressing class prototypes with per-class masks and a sample-count scaling trick cuts communication cost in prototype-based federated learning by up to several times without hurting accuracy.

  9. Multimodal Federated Learning: A Survey through the Lens of Different FL Paradigms

    cs.LG 2025-05 conditional novelty 5.0 of 10

    A paradigm-based taxonomy of multimodal federated learning that assigns each branch a headline challenge: modality heterogeneity (horizontal), privacy leakage (vertical), and efficiency (hybrid).

  10. Federated Learning-Distillation Alternation for Resource-Constrained IoT

    eess.SP 2025-05 conditional novelty 5.0 of 10

    FLDA alternates between cheap federated distillation exchanges and full federated learning updates, achieving higher accuracy than either method alone with large energy savings in simulated EH-IoT networks.

  11. Mixture-of-Translators: Translating KV Caches Across Heterogeneous Large Language Models

    cs.CL 2026-07 conditional novelty 4.0 of 10

    MoT translates KV caches between LLMs using token-level translator mixtures and a context-correction loss; QA results are strong for same-family pairs but do not demonstrate true cross-architecture reuse.

  12. Heterogeneous Federated Learning with Prototype Alignment and Upscaling

    cs.LG 2025-07 conditional novelty 3.0 of 10

    ProtoNorm adds server-side prototype alignment and a per-dataset scaling factor to FedProto-style federated learning, improving accuracy but with the gain largely driven by the tuned scaling factor.

Pith tools