SEMFED: Semantic-Aware Resource-Efficient Federated Learning for Heterogeneous NLP Tasks

Muhammad Sohail; Nauman Ali Khan; Sajid Hussain

REVIEW 7 major objections 6 minor 23 references

SEMFED: Semantic-Aware Resource-Efficient Federated Learning for Heterogeneous NLP Tasks

T0 review · 7 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SEMFED combines semantic-aware client selection, device-tailored models, and feature compression to cut federated NLP communication by 80.5% while keeping accuracy above 98%.

desk verdict Coherent integration of known FL ideas, but the reported compression ratio and evaluation protocol contradict the setup, so the headline numbers are uncheckable. read the letter →

arxiv 2505.23801 v1 pith:7HGEODOF submitted 2025-05-26 cs.CL cs.AIcs.LG

Sajid Hussain , Muhammad Sohail , Nauman Ali Khan This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords federatedlearningnaturallanguageprocessingsemanticheterogeneityclientselectionfeaturecompressionresourceefficiencyheterogeneousdevicescommunication

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SEMFED is a proposed framework for federated learning on natural-language tasks where clients differ in both their data (vocabulary and class distributions) and their hardware (memory, compute, battery, network). The paper's central claim is that accounting for semantic heterogeneity—not just resource constraints—is what lets federated NLP stay accurate after compression; its three mechanisms are semantic-aware client selection, per-device model architectures, and semantic feature compression. On a synthetic 10-client text classification benchmark with a skewed label distribution and simulated device types, SEMFED is reported to reach 98.5% accuracy with a compression ratio of 0.195, an 80.5% reduction in communication cost, and a final battery level near 99.6%. The intended takeaway is that a federation of heterogeneous edge devices can train one effective text classifier cheaply if the server selects clients by semantic utility and exchanges compressed semantic features instead of raw model parameters.

What carries the argument

The load-bearing mechanism is the semantic-preserving pipeline. A utility score that mixes three terms—semantic diversity, resource efficiency, and participation fairness—selects which clients join each round. Each selected client runs an architecture matched to its device (Small CNN, Tiny LSTM, or MobileBERT) over a shared embedding layer that adds soft semantic-cluster embeddings to token embeddings. Instead of parameters, the client sends features compressed by sparse coding or semantic PCA followed by quantization; the server decompresses and aligns these features through semantic cluster centers before classification. This pipeline is what the paper credits for both the 80.5% communication saving and the maintained accuracy.

What would settle it

Re-run SEMFED on a real text-classification corpus partitioned into non-IID client splits (for example, by author or topic) with actual mobile, laptop, and desktop hardware and real network conditions, and check whether accuracy stays above 98%, communication savings stay near 80.5%, and battery and compute times stay at the reported levels.

Watch

Extended reading notes

Core claim

The central discovery is the specific combination: a client-selection utility that balances semantic diversity, resource efficiency, and participation fairness; heterogeneous local models that share a semantic-preserving embedding layer; and a compression pipeline of sparse coding, semantic PCA, and 8-bit quantization that lets clients send compact features. On the paper's benchmark this combination reaches 98.5% accuracy at compression ratio 0.195, beating FedAvg (91.2%), FedProx (92.5%), FedNLP (94.7%), HeteroFL (95.3%), and a resource-only variant (96.1%) while also improving energy and battery metrics. The paper presents this as evidence that feature-level semantic distillation outperforms model averaging in heterogeneous NLP settings.

Load-bearing premise

All headline results rest on a single synthetic 10-client text dataset with a skewed label distribution and simulated device types; if that dataset does not capture real vocabulary mismatch, class skew, or edge-device constraints, the reported gains over baselines may not survive deployment.

Editorial extensions

If this is right

On the paper's evidence, federated NLP can run on edge devices by exchanging compressed semantic features instead of model parameters, cutting bandwidth by roughly four-fifths.
Semantic-aware client selection should converge faster than random or resource-only selection (about 10 rounds versus 14–18 rounds in the reported experiments).
Heterogeneous fleets can each run an architecture sized to the device—small CNN on phones, tiny LSTM on laptops, MobileBERT on desktops—while still contributing to one global classifier.
Battery and per-round compute can stay low enough for long deployments, with battery stabilizing above 99% and average compute near 0.67 seconds after the first round in the reported runs.
The approach implies that semantic overlap between clients is an asset to exploit rather than noise to ignore, because selection can use that overlap to maximize information gain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the synthetic benchmark transfers to real corpora, the same semantic-feature compression could be applied to larger transformer-based federated systems, where communication cost scales with model size; the cluster-attention alignment would likely need retuning for richer vocabularies.
A direct test on real devices with actual bandwidth limits would be the natural next experiment; the paper itself flags real-world deployment as an open limitation.
The utility-scoring idea could generalize to other non-IID modalities (for example, speech or sensor data) by replacing the semantic-similarity term with a domain-appropriate divergence, but that generalization is not demonstrated here.
Because all headline numbers come from one synthetic 10-client dataset, the stated 80.5% saving and 98%+ accuracy should be read as a proof of concept until replicated on real NLP data.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is a textbook example of a coherent framework undermined by an unverifiable evaluation. The three pillars—semantic-aware client selection, heterogeneous architectures, and feature compression—are each established in the literature, and the integration is clean. But the central claim of an 80.5% communication reduction at 98.5% accuracy cannot be checked: the reported compression ratio contradicts the stated setup, and the accuracy evaluation set is never defined.

What the paper does well: the writing is accessible, the equations are laid out readably, and the ablation tables give some sense of what each component contributes. The limitation section is honest about the lack of real-world deployment. The semantic-preserving embedding layer (Eq. 7-8) is a modest extension of existing ideas, though not the breakthrough the abstract implies.

The problems are load-bearing. Section 5.1.3 says you used 8-bit quantization and a compression ratio of 0.4. Table 3 reports a compression ratio of 0.195, and Figure 6(a) claims average savings of 80.5%—which is exactly ratio 0.195. These cannot both be right. Either the setup is misstated or the ratio is miscalculated.

'Total Comm.' is never defined. For SEMFED it appears to count only the compressed features, while FedAvg and FedProx transmit full model parameters. The FedAvg total of 24.78 MB over 20 rounds is hard to reconcile with the architecture sizes in Table 1 (~0.5M to 15M parameters). Under float32, even a single round of full-precision updates across ten clients would use far more, unless the models are effectively much smaller or the reported number uses a different convention. The comparison is apples-to-oranges.

Accuracy is likewise underspecified. The text and Figure 10 say client models approach accuracy near 1.0 within five rounds, but never state whether this is measured on each client's local test split or on a shared held-out set. If it is local, that number reflects specialization, not generalization, and cannot be compared with global-model accuracy from baselines. The client-server agreement of 0.967 in Table 2 suggests the server model is quite different, yet no server accuracy is tabulated.

There are also signs of a hasty manuscript: a duplicated 'Experimental Results' heading, a citation for 'FedMatch' that points to an unrelated paper, and a claimed 'theoretical foundation' that never materializes. These are not fatal on their own, but they add up.

As submitted, this paper does not establish its own central claim. The single synthetic dataset, the contradictory compression numbers, and the undefined evaluation protocol mean the 80.5% / 98.5% numbers are not checkable. The stress-test note holds up on reading.

Who is this for? A reader wanting a conceptual starting point for combining these techniques, not evidence that the combination works. I would not send it to review as is. If the authors provided code, real NLP benchmarks, error bars, and fixed the accounting, the work might deserve a second look. For now, reject.

Referee Report

7 major / 6 minor

Summary. SEMFED proposes a federated learning framework for heterogeneous NLP tasks built on three components: semantic-aware client selection (Eq. 4), heterogeneous model architectures with a semantic-preserving embedding layer (Eqs. 7-8), and communication-efficient semantic feature compression via sparse coding, PCA, and quantization (Eqs. 9-11). The paper claims an 80.5% reduction in communication costs while maintaining accuracy above 98%, based on experiments on a synthetic text classification dataset with 10 clients and 20 rounds, compared against FedAvg, FedProx, FedNLP, HeteroFL, and two in-house baselines.

Significance. If the reported results were reproducible, SEMFED would be a useful contribution to communication-efficient federated NLP, particularly the idea of transmitting compressed semantic features rather than model parameters in a heterogeneous-device setting. The paper also formulates a sensible three-part architecture (selection, heterogeneous models, compression). However, the central quantitative claim is not currently supported: the compression-ratio numbers are internally inconsistent, the communication accounting is opaque, the accuracy evaluation set is unspecified, and several reported metrics are undefined. The paper provides no code, no data, no repeated runs, and no external benchmark, so the claimed advantages over baselines cannot be verified. The significance of the work as presented is therefore mostly potential rather than demonstrated.

major comments (7)

[Section 5.1.3 vs. Table 3 and Figure 6(a)] The text states 'For feature compression, we use 8-bit quantization and a compression ratio of 0.4,' but Table 3 reports SEMFED's compression ratio as 0.195 and Figure 6(a) reports average savings of 80.5%, i.e., a ratio of 0.195. These numbers are contradictory unless 'compression ratio' is defined differently across these locations. The paper never defines compression ratio, so the headline 80.5% reduction in communication costs is not tied to a concrete, verifiable quantity.
[Section 5.1.3 and Table 3] 'Total Comm.' is never defined. The FedAvg total of 24.78 MB over 20 rounds is inconsistent with the model sizes in Table 1: a 0.5M-parameter Small CNN transmitted in float32 would require 2 MB per client per round, which across 10 clients and 20 rounds already exceeds 400 MB, and the larger models would yield tens of GB. The reported 24.78 MB implies either a much smaller model or a different accounting rule (e.g., counting only compressed features for SEMFED but something else for baselines). The paper must provide a precise definition of Total Comm. and apply identical accounting to all methods before the 80.5% savings claim can be assessed.
[Section 5.3.7 and Figure 10] The accuracy evaluation is underspecified. The text reports that client models 'reach near-perfect accuracy (approaching 1.0) within 5 rounds,' but it does not state whether this accuracy is measured on each client's local test split, a global held-out set, or some other partition. If client models are evaluated on their own non-IID local splits, high accuracy reflects local specialization rather than generalization and is not comparable to the global-model accuracy of baselines. The accuracy figures in Table 2 (e.g., 98.5% for SEMFED) require a clear statement of the evaluation set and metric definition.
[Table 5 and Section 5.4] The three 'Semantic Preservation Metrics' in Table 5 (Semantic Similarity, Vocab Overlap, Feature Coherence) are not defined anywhere in the text, nor are the measurement procedures given. Without these definitions the values cannot be interpreted or reproduced. Similarly, 'Convergence (rounds)' in Table 6 and 'Compression Ratio' in Table 8 lack formal definitions in the experimental setup. In addition, all results are reported as single values with no error bars or number of independent runs, so the 2.4 percentage-point gap between SEMFED (98.5%) and Resource-Only FL (96.1%) in Table 2 may be within noise.
[Section 5.3.5 and Figures 7-8] The evaluation of client selection is partly circular. The selection rule (Eq. 4) optimizes semantic diversity and participation fairness, and the paper then reports balanced selection frequencies and semantic diversity in the selected set as evidence of success (e.g., Section 5.3.2 and Figure 3). To validate the selection mechanism, the authors should show that these choices improve a downstream metric such as global test accuracy relative to counterfactual selection rules on a fixed, pre-specified test set, rather than reporting the optimized objective itself as an outcome.
[Section 5.1.2 and Tables 2-5] The baseline 'FedNLP' is a benchmark suite (Lin et al., NAACL 2021), not a federated learning algorithm. Comparing SEMFED against 'FedNLP' as a method is therefore not meaningful unless the authors specify the concrete algorithm they implemented under that name. The paper should either replace this baseline with an actual FL algorithm from the FedNLP benchmark or clearly describe the implementation used.
[Section 1 and Section 4] The paper claims in the contributions list that it develops 'a theoretical foundation for balancing semantic diversity and resource efficiency,' but Section 4 contains only heuristic formulations (Eqs. 1-13) and no theorems, proofs, convergence analysis, or formal guarantees. This claim should either be removed or substantiated with theoretical results.

minor comments (6)

[Abstract and Section 2] There are typographical errors such as 'ExperimentalresultsonvariousNLP...' in the abstract and missing spaces throughout the text; these should be corrected.
[Section 5.2-5.3] Section 5.2 and Section 5.3 are both titled 'Experimental Results,' which creates duplicate section headings; the numbering and titles should be made consistent.
[Algorithm 1 and Section 5.1.3] The parameter m in Algorithm 1 (the number of selected clients per round) is never specified in the experimental setup, yet it directly affects the communication totals and the client selection analysis.
[Section 3.3 and Eq. (2)] The resource efficiency definition in Eq. (2) includes network reliability Nk without specifying a normalization, and the weights ωM, ωP, ωB, and ωN are not stated to sum to 1; no sensitivity analysis is provided for these weights or for the hyperparameters α, λ1, λ2, and λ3.
[Section 2.2 and Reference [4]] Reference [4] is a paper on federated learning for speaker recognition, but the text states that 'Zhu et al. [4] propose federated pre-training of language models'; the citation does not support the claim and should be corrected.
[Table 4] Table 4 reports 'Avg. Energy (units)' and 'Final Battery (%)' without defining the energy unit or the battery model; the statement that battery levels remain above 99% after 20 rounds while energy consumption is non-negligible is not explained.

Circularity Check

3 steps flagged · score 6.0 of 10

Central results partly restate the framework's own inputs: the balanced selection pattern is the Eq. (4) fairness objective, the 80.5% communication saving is the complement of the configured compression ratio, and the 'semantic preservation' metrics include input vocabulary and class statistics.

self definitional [Section 4.1 Eq. (4); Section 5.3.5]
"The selection is based on a utility score that combines semantic diversity, resource efficiency, and fairness in participation: utility(k, t) = λ1 · divsemantic(k, St) + λ2 · effresource(k) + λ3 · fairparticipation(k, t) (4) ... SEMFED achieves a balanced client selection pattern as shown in Figure 8(a), with selection frequencies ranging from 35% to 70% for different clients. ... This balance is crucial for ensuring fair participation while optimizing for semantic diversity and resource constraints."

The balance reported in Section 5.3.5 is the fairness objective itself: Eq. (4) explicitly maximizes fairparticipation(k,t) with weight λ3=0.3, so the observed 'balanced selection pattern' is a direct consequence of the utility function, not an independent empirical finding. The paper presents this by-construction outcome as evidence that SEMFED's client selection is effective, but the qualitative balance is baked into the objective being optimized, making the validation circular.
fitted input called prediction [Section 5.1.3 vs. Section 5.3.4 and Table 3]
"For feature compression, we use 8-bit quantization and a compression ratio of 0.4. ... the red dotted line shows the average savings percentage (80.5%). ... SEMFED (Ours) 4.82 0.195 98.5"

The headline 80.5% reduction is the complement of the 0.195 compression ratio reported in Table 3. The compression ratio is set as a configuration choice in Section 5.1.3 (given as 0.4, with Fig. 6(b) reporting ratios 0.1–0.25), so the 'result' is a restatement of the chosen compression level, not a quantity predicted by the SEMFED framework. Reporting this number as an experimental outcome conflates the input parameter with the measured saving; the claimed communication advantage is forced by the way the compression ratio is defined and configured, rather than derived from the semantic-preservation machinery.

1 more flagged steps

renaming known result [Section 3.1–3.2 Eq. (1); Section 5.3.1; Table 5]
"Each client has a semantic profile Sk = {Vk, Ck, Tk}, where Vk represents the vocabulary statistics, Ck represents the class distribution... and simsemantic(k, j) = α · (1 − JS(Ck, Cj)) + (1 − α) · Jaccard(Vk, Vj) (1) ... Table 5: Semantic Preservation Metrics ... SEMFED (Ours) 0.835 0.500 0.812"

Table 5 reports SEMFED's 'Semantic Similarity' and 'Vocab Overlap' as if they were outcomes of the method, but these are the input statistics used to define semantic profiles and to compute Eq. (1) for client selection. Vocab overlap is exactly Jaccard(Vk,Vj) from the pre-defined synthetic split; semantic similarity is the same combination of class-distribution JS and vocabulary Jaccard that feeds the utility score. Thus the semantic-preservation metrics rename the data-generation and selection inputs as model achievements; they do not measure something SEMFED produced beyond the fit of the learned clusters to the data they were trained on.

full rationale

The circularity is internal to the evaluation design rather than a self-citation chain. Three load-bearing 'results' are equal to the framework's own inputs by construction. First, the balanced client-selection pattern is the fairparticipation term optimized in Eq. (4); reporting it as evidence validates the objective rather than the method. Second, the 80.5% communication saving is 1 − 0.195, where the compression ratio is a configured parameter in Section 5.1.3 (stated as 0.4, with Fig. 6(b) showing 0.1–0.25); the headline number is therefore a restatement of the compression setting, not a derived prediction. Third, the 'Semantic Preservation Metrics' in Table 5 include Vocab Overlap and Semantic Similarity, which are the input vocabulary and class statistics from Eq. (1) and the client profiles defined in Section 3.1; presenting them as SEMFED's achievements renames the synthetic data's properties as model outputs. The accuracy comparison against FedAvg and FedProx contains some independent empirical content, so the paper is not wholly circular, but the central communication-efficiency and semantic-preservation claims partially reduce to the paper's own definitions and configuration choices. No load-bearing self-citation was found.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The paper's central claims rely on hand-set weighting parameters, learned semantic clusters, and a specific compression ratio, all evaluated on a private synthetic dataset. No independent external benchmark or falsifiable prediction is provided, so the contribution is a proposed system whose numbers are tied to the authors' configuration choices.

free parameters (7)

alpha in Eq. 1 = not stated
Hand-set weight between class-distribution JS divergence and vocabulary Jaccard similarity; no sensitivity analysis provided.
omega_M, omega_P, omega_B, omega_N in Eq. 2 = not stated
Resource efficiency weights for memory, compute, battery, network; chosen without sensitivity analysis.
lambda_1, lambda_2, lambda_3 in Eq. 4 = 0.4, 0.3, 0.3 (Section 5.1.3)
Client selection utility weights, selected by hand and not varied.
sparsity regularization lambda in Eq. 9 = not stated
Controls the sparse coding trade-off; no ablation or value reported.
number of semantic clusters = 5, 8, 10 for Small CNN, Tiny LSTM, MobileBERT
Hyperparameters per device class, chosen without sensitivity analysis.
compression ratio = target 0.4, reported average 0.195
Directly controls the claimed 80.5% communication saving; the ratio is a design choice, not an emergent property.
quantization bits b in Eq. 11 = 8
Chosen without sensitivity analysis; affects the accuracy/compression trade-off.

assumptions (5)

domain assumption Semantic heterogeneity is adequately captured by class-distribution JS divergence and vocabulary Jaccard overlap (Eq. 1).
The paper assumes these two statistics are sufficient to represent the semantic differences that matter for FL convergence; no empirical or theoretical justification is given.
domain assumption Resource efficiency can be modeled as a weighted sum of normalized memory, compute, battery, and network reliability (Eq. 2).
The additive linear form is chosen for convenience; real device performance may have nonlinear interactions.
ad hoc to paper The semantic-preserving embedding layer (Eq. 7-8) with learned soft cluster assignments improves FL accuracy on heterogeneous text.
No theorem or analysis supports this; the clusters are learned on the same data used for evaluation, so any benefit is empirical and possibly task-specific.
ad hoc to paper The server's semantic alignment layers (Eq. 13) can map heterogeneous client feature spaces to a common space without information loss.
This is a core design choice, but the paper provides no formal guarantee or analysis of the alignment error.
standard math Standard math background: JS divergence, Jaccard similarity, PCA, sparse coding, and quantization behave as expected in high-dimensional text feature spaces.
These are standard tools, but their applicability to the specific learned feature distributions is not verified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SEMFED: Semantic-Aware Resource-Efficient Federated Learning for Heterogeneous NLP Tasks." pith.science (2026). https://pith.science/paper/7HGEODOF

@misc{pith2026250523801,
  author       = {Pith},
  title        = {Pith review of: SEMFED: Semantic-Aware Resource-Efficient Federated Learning for Heterogeneous NLP Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7HGEODOF}},
  note         = {Machine review of arXiv:2505.23801}
}

read the original abstract

Background: Federated Learning (FL) has emerged as a promising paradigm for training machine learning models while preserving data privacy. However, applying FL to Natural Language Processing (NLP) tasks presents unique challenges due to semantic heterogeneity across clients, vocabulary mismatches, and varying resource constraints on edge devices. Objectives: This paper introduces SEMFED, a novel semantic-aware resource-efficient federated learning framework specifically designed for heterogeneous NLP tasks. Methods: SEMFED incorporates three key innovations: (1) a semantic-aware client selection mechanism that balances semantic diversity with resource constraints, (2) adaptive NLP-specific model architectures tailored to device capabilities while preserving semantic information, and (3) a communication-efficient semantic feature compression technique that significantly reduces bandwidth requirements. Results: Experimental results on various NLP classification tasks demonstrate that SEMFED achieves an 80.5% reduction in communication costs while maintaining model accuracy above 98%, outperforming state-of-the-art FL approaches. Conclusion: SEMFED effectively manages heterogeneous client environments with varying computational resources, network reliability, and semantic data distributions, making it particularly suitable for real-world federated NLP deployments.

Figures

Figures reproduced from arXiv: 2505.23801 by the authors.

**Figure 2.** Semantic diversity and preservation in SEMFED. (a) Client Vocabulary Diversity. The [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

**Figure 3.** Client Semantic Similarity Network. This visualization shows the semantic relationships [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Feature Space Visualization colored by class. This visualization shows a 2D projection [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

**Figure 5.** Figure 5: Resource efficiency metrics in SEMFED. (a) Energy Usage by Client. The bar chart [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

**Figure 6.** Figure 6: Communication efficiency of SEMFED. (a) Communication Savings by Round. The bars [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

**Figure 7.** Figure 7: Client Selection Pattern. The heatmap shows which clients were selected in each commu [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Client selection and model distribution in SEMFED. (a) Client Selection Frequency. The [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

**Figure 9.** Figure 9: Client Battery Levels. The graph shows battery levels across communication rounds for [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

**Figure 10.** Figure 10: Model Accuracy Comparison. The graph compares accuracy across communication [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 14 canonical work pages

[1]

Communication- efficient learning of deep networks from decentralized data,

H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication- efficient learning of deep networks from decentralized data,” inProceedings of the 20th Inter- national Conference on Artificial Intelligence and Statistics (AISTATS), pp. 1273–1282, 2017

work page 2017
[2]

Advances and open problems in federated learning,

P. Kairouz et al., “Advances and open problems in federated learning,”Foundations and Trends in Machine Learning, vol. 14, no. 1-2, pp. 1–210, 2021

work page 2021
[3]

FedNLP: Benchmarking federated learning methods for natural language pro- cessing tasks,

B. Lin et al., “FedNLP: Benchmarking federated learning methods for natural language pro- cessing tasks,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 5749–5757, 2021

work page 2021
[4]

Federated learning for speaker recognition based on self-attention mechanism,

F. Zhu, F. Ye, Y. Fu, Q. Liu, and B. Shen, “Federated learning for speaker recognition based on self-attention mechanism,”IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 28, pp. 2578–2591, 2020

work page 2020
[5]

Federated learning: Strategies for improving communication efficiency,

J. Konečn` y, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,”arXiv preprint arXiv:1610.05492, 2016

arXiv 2016
[6]

Federated learning with non-iid data,

Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” arXiv preprint arXiv:1806.00582, 2018

arXiv 2018
[7]

A joint learning and communica- tions framework for federated learning over wireless networks,

M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communica- tions framework for federated learning over wireless networks,”IEEE Transactions on Wireless Communications, vol. 20, no. 1, pp. 269–283, 2021

work page 2021
[8]

Federated optimization in heterogeneous networks,

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,”Proceedings of Machine Learning and Systems, vol. 2, pp. 429–450, 2020

work page 2020

Show all 23 references

[9]

Adaptive federated learning in resource constrained edge computing systems,

S. Wang et al., “Adaptive federated learning in resource constrained edge computing systems,” IEEE Journal on Selected Areas in Communications, vol. 37, no. 6, pp. 1205–1221, 2019

2019
[10]

HeteroFL: Computation and communication efficient feder- ated learning for heterogeneous clients,

E. Diao, J. Ding, and V. Tarokh, “HeteroFL: Computation and communication efficient feder- ated learning for heterogeneous clients,”arXiv preprint arXiv:2010.01264, 2020

2010 arXiv
[11]

On the convergence of federated optimization in heterogeneous networks,

A. K. Sahu, T. Li, M. Sanjabi, M. Zaheer, A. Talwalkar, and V. Smith, “On the convergence of federated optimization in heterogeneous networks,”arXiv preprint arXiv:1812.06127, 2018

2018 arXiv
[12]

Tackling the objective inconsistency problem in heterogeneous federated opti- mization,

J. Wang et al., “Tackling the objective inconsistency problem in heterogeneous federated opti- mization,” Advances in Neural Information Processing Systems, vol. 33, pp. 7611–7623, 2020

2020
[13]

Deep gradient compression: Reducing the communication bandwidth for distributed training,

Y. Lin, S. Han, H. Mao, Y. Wang, and W. J. Dally, “Deep gradient compression: Reducing the communication bandwidth for distributed training,”arXiv preprint arXiv:1712.01887, 2017

2017 arXiv
[14]

Model pruning enables efficient federated learning on edge devices,

Y. Jiang, S. Wang, B. J. Ko, W.-H. Lee, and L. Tassiulas, “Model pruning enables efficient federated learning on edge devices,”arXiv preprint arXiv:1909.12326, 2019

1909 arXiv
[15]

QSGD: Communication-efficient SGD via gradient quantization and encoding,

D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic, “QSGD: Communication-efficient SGD via gradient quantization and encoding,” Advances in Neural Information Processing Systems, vol. 30, pp. 1709–1720, 2017. 20

2017
[16]

Deep neural networks with massive learned knowledge,

J. Chen and X. Ran, “Deep neural networks with massive learned knowledge,” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2118–2133, 2021

2021
[17]

A secure federated learning framework for 5G networks,

L. Liu et al., “A secure federated learning framework for 5G networks,”IEEE Wireless Com- munications, vol. 27, no. 4, pp. 24–31, 2020

2020
[18]

MobileBERT: a compact task- agnostic BERT for resource-limited devices,

Z. Sun, H. Yu, X. Song, R. Liu, Y. Yang, and D. Zhou, “MobileBERT: a compact task- agnostic BERT for resource-limited devices,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2158–2170, 2020

2020
[19]

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,

V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,”arXiv preprint arXiv:1910.01108, 2019

1910 arXiv
[20]

TinyBERT:DistillingBERTfornaturallanguageunderstanding,

X.Jiaoetal., “TinyBERT:DistillingBERTfornaturallanguageunderstanding,” arXiv preprint arXiv:1909.10351, 2019

1909 arXiv
[21]

FedMD: Heterogeneous federated learning via model distillation,

D. Li and J. Wang, “FedMD: Heterogeneous federated learning via model distillation,”arXiv preprint arXiv:1910.03581, 2019

1910 arXiv
[22]

Ensemble distillation for robust model fusion in federated learning,

T. Lin, L. Kong, S. U. Stich, and M. Jaggi, “Ensemble distillation for robust model fusion in federated learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 2351– 2363, 2020

2020
[23]

Client selection for federated learning with heterogeneous resources in mobile edge,

T. Nishio and R. Yonetani, “Client selection for federated learning with heterogeneous resources in mobile edge,” inIEEE International Conference on Communications (ICC), pp. 1–7, 2019. 21

2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Communication- efficient learning of deep networks from decentralized data,

H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication- efficient learning of deep networks from decentralized data,” inProceedings of the 20th Inter- national Conference on Artificial Intelligence and Statistics (AISTATS), pp. 1273–1282, 2017

work page 2017

[2] [2]

Advances and open problems in federated learning,

P. Kairouz et al., “Advances and open problems in federated learning,”Foundations and Trends in Machine Learning, vol. 14, no. 1-2, pp. 1–210, 2021

work page 2021

[3] [3]

FedNLP: Benchmarking federated learning methods for natural language pro- cessing tasks,

B. Lin et al., “FedNLP: Benchmarking federated learning methods for natural language pro- cessing tasks,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 5749–5757, 2021

work page 2021

[4] [4]

Federated learning for speaker recognition based on self-attention mechanism,

F. Zhu, F. Ye, Y. Fu, Q. Liu, and B. Shen, “Federated learning for speaker recognition based on self-attention mechanism,”IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 28, pp. 2578–2591, 2020

work page 2020

[5] [5]

Federated learning: Strategies for improving communication efficiency,

J. Konečn` y, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,”arXiv preprint arXiv:1610.05492, 2016

arXiv 2016

[6] [6]

Federated learning with non-iid data,

Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” arXiv preprint arXiv:1806.00582, 2018

arXiv 2018

[7] [7]

A joint learning and communica- tions framework for federated learning over wireless networks,

M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communica- tions framework for federated learning over wireless networks,”IEEE Transactions on Wireless Communications, vol. 20, no. 1, pp. 269–283, 2021

work page 2021

[8] [8]

Federated optimization in heterogeneous networks,

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,”Proceedings of Machine Learning and Systems, vol. 2, pp. 429–450, 2020

work page 2020