LEAF: A Benchmark for Federated Settings
pith:4YPAYJISopen to challenge →
read the original abstract
Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day. This wealth of data can help to learn models that can improve the user experience on each device. However, the scale and heterogeneity of federated data presents new challenges in research areas such as federated learning, meta-learning, and multi-task learning. As the machine learning community begins to tackle these challenges, we are at a critical time to ensure that developments made in these areas are grounded with realistic benchmarks. To this end, we propose LEAF, a modular benchmarking framework for learning in federated settings. LEAF includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.
This paper has not been read by Pith yet.
Forward citations
Cited by 28 Pith papers
-
FedXDS: Leveraging Model Attribution Methods to counteract Data Heterogeneity in Federated Learning
FedXDS uses propagation-based attribution to identify task-relevant features for selective data sharing in federated learning, yielding higher accuracy and faster convergence under heterogeneity with formal privacy gu...
-
Totoro$^+$: An Adaptive and Scalable Edge Federated Learning System
Totoro+ is a DHT-based fully decentralized FL system with locality-aware multi-ring P2P structure, pub/sub forest, and game-theoretic path planning that claims O(log N) hops and 1.2-14x speedup for many concurrent app...
-
Your Neighbors Know: Leveraging Local Neighborhoods for Backdoor Detection in Decentralized Learning
Argus detects backdoors in decentralized learning by local trigger analysis and neighbor similarity checks on consistency, with theoretical convergence guarantees and empirical reductions in attack success up to 90 points.
-
Your Neighbors Know: Leveraging Local Neighborhoods for Backdoor Detection in Decentralized Learning
Argus enables backdoor detection in decentralized ML by collaborative neighbor-based validation of triggers, backed by convergence theory and reducing attack success by up to 90% on tested datasets.
-
From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learning
FedSAF shifts prototype alignment in heterogeneous federated learning from coordinate matching to inter-class structural relations and reports up to 3.52% gains over prior methods.
-
Data-Free Contribution Estimation in Federated Learning using Gradient von Neumann Entropy
Matrix von Neumann entropy of final-layer gradients acts as a data-free proxy for client contribution in federated learning, showing high correlation with standalone accuracy on non-IID benchmarks.
-
SMART: A Spectral Transfer Approach to Multi-Task Learning
SMART transfers knowledge in multi-task linear regression via spectral subspace similarity assumptions, achieving near-minimax Frobenius error rates while requiring only a fitted source model.
-
SketchGuard: Scaling Byzantine-Robust Decentralized Federated Learning via Sketch-Based Screening
SketchGuard decouples Byzantine filtering from aggregation in decentralized federated learning by exchanging k-dimensional Count Sketches for screening and full models only from accepted neighbors, achieving up to 50-...
-
FeDa4Fair: Client-Level Federated Datasets for Fairness Evaluation
FeDa4Fair is a new library and benchmark for creating federated datasets with heterogeneous client-level biases to standardize evaluation of fairness methods in federated learning.
-
Closing the Alignment-Maturity Gap in Federated Prototype Learning
FedSAP stabilizes federated prototype learning via a deterministic alignment curriculum and proxy separation loss, reporting up to 4 percentage point gains under high heterogeneity across three benchmarks.
-
Silent Failures in Federated Personalization of Foundation Models
Federated personalization of foundation models creates hard-to-detect trustworthiness failures due to privacy constraints, and existing benchmarks cannot adequately evaluate them.
-
Efficient Multi-objective Prompt Optimization via Pure-exploration Bandits
Adapting multi-objective pure-exploration bandits enables efficient Pareto prompt set recovery and best feasible prompt identification for LLMs, with linear-case guarantees and empirical gains over baselines.
-
Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
Rescaled ASGD recovers convergence to the true global objective by rescaling worker stepsizes proportional to computation times, matching the known time lower bound in the leading term under non-convex smoothness and ...
-
FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
Fed-FSTQ uses Fisher information to guide token selection and mixed-precision quantization in federated PEFT, cutting uplink traffic 46x and wall-clock time-to-accuracy by 52% on multilingual and medical QA tasks.
-
PrivEraserVerify: Efficient, Private, and Verifiable Federated Unlearning
PrivEraserVerify unifies efficiency via adaptive checkpointing, privacy via layer-adaptive DP, and verifiability via fingerprints in federated unlearning, claiming 2-3x faster performance than retraining with formal g...
-
Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
FedGMR progressively restores sub-model capacity for bandwidth-constrained clients via gradual density increases and mask-aware aggregation, narrowing the gap to full-model federated learning.
-
Incentivizing Honesty among Competitors in Collaborative Learning and Optimization
Formulates a game where competitors in collaborative learning are incentivized to manipulate updates, then proposes mechanisms that restore honest participation for mean estimation, convex SGD, and non-convex federate...
-
Adaptive Federated Optimization
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
-
Exploring CKKS Parameter Trade-offs for Privacy-Preserving Personalized Federated Learning
pFedCKKS derives CKKS parameter constraints for PFL under 128-bit security reducing choices to inner and outer ciphertext primes and evaluates precision-cost trade-offs on FEMNIST, CelebA and Sentiment140.
-
CRAFT: Conflict-Resolved Aggregation for Federated Training
CRAFT derives a closed-form solution for conflict-resolved aggregation in federated learning via geometric constraints and projection, with theoretical support for common descent and empirical gains on heterogeneous data.
-
FedFrozen: Two-Stage Federated Optimization via Attention Kernel Freezing
FedFrozen improves stability in heterogeneous federated Transformer training by warming up the full model then freezing the attention kernel (query/key) while optimizing the value block under a fixed kernel.
-
FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
Fed-FSTQ uses Fisher-guided token quantization to cut uplink traffic 46-fold and improve straggler-limited time-to-accuracy by 52% versus Fed-LoRA in non-IID multilingual and medical QA tasks.
-
FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
Fed-FSTQ reduces uplink traffic by 46x and improves time-to-accuracy by 52% in federated LLM fine-tuning using Fisher-guided token quantization and selection.
-
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
Non-identical data distributions degrade federated averaging accuracy on visual classification, but server momentum raises CIFAR-10 accuracy from 30.1% to 76.9% in the most skewed regimes.
-
FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
FedMTFI clusters heterogeneous clients, trains cluster prototypes, and applies multi-teacher distillation with SHAP to improve accuracy over standard FL in non-IID settings.
-
Task2vec Readiness: Diagnostics for Federated Learning from Pre-Training Embeddings
Task2Vec-based unsupervised metrics of client embedding cohesion, dispersion, and density correlate strongly with final federated learning performance across multiple datasets and heterogeneity levels.
-
Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems
A survey of architectures, threats, defenses, and future directions for security and privacy in RAG systems.
-
A Survey on Foundation Models for Personalized Federated Intelligence
The survey introduces personalized federated intelligence (PFI) as a framework integrating federated learning and foundation models to support privacy-aware personalization of AI models.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.