Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CYBER-0 is a Byzantine-resilient federated zero-order optimization framework that converges for general non-convex heterogeneous objectives while transmitting only a few scalars per client per round.

desk verdict The method and experiments are worth engaging with, but the central convergence proof rests on a transposed and unscaled JL lemma, so the theoretical claims are not currently supported. read the letter →

arxiv 2502.00193 v1 pith:PLAPXQH6 submitted 2025-01-31 cs.LG cs.CRcs.DCstat.ML

Maximilian Egger , Mayank Bakshi , Rawad Bitar This is my paper · ORCID
classification cs.LGcs.CRcs.DCstat.ML
keywords federatedlearningByzantineresiliencezero-orderoptimizationcommunicationefficiencynon-convexdataheterogeneitytransformedrobustaggregationJohnson-Lindenstraussembedding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces CYBER-0, a federated learning method that keeps training on track even when some clients are malicious, while shrinking the per-round communication between clients and server to a few scalars. The central claim is that zero-order gradient estimates, projected onto shared random directions, can be fed through standard robust aggregation rules in that low-dimensional space without losing the Byzantine resilience those rules have on full gradients. The paper proves convergence for general non-convex losses under bounded data heterogeneity, and reports accuracy comparable to full-gradient Byzantine-resilient federated learning with up to seven orders of magnitude less communication. This matters because communication, not computation, is often the bottleneck in federated training, and malicious clients are a realistic deployment threat.

What carries the argument

The central mechanism is transformed robust aggregation in the perturbation space: each client sends the ν-dimensional vector of its zero-order estimates along shared pseudorandom directions, and the federator runs the robust rule on those projected vectors instead of on full gradients. Its work is to keep the global update inside the low-dimensional subspace spanned by the perturbation directions, which is what makes the few-scalar communication possible, and the paper's Lemma B.5, a Johnson-Lindenstrauss-type projection bound, is what transfers the (b,κ)-robustness of the original aggregation rule to this transformed space.

What would settle it

For a fixed set of honest gradients, compute the honest average, project it to one dimension with ν=1, and run coordinate-wise trimmed mean on the projected scalars; if the aggregated scalar deviates from the projection of the honest average by more than κ times the projected honest spread, the robustness-transfer premise fails in exactly the regime the theorems do not cover.

Watch

Extended reading notes

Core claim

The core discovery is that robust aggregation can be performed in the ν-dimensional space of scalar zero-order estimates rather than on reconstructed d-dimensional gradients. CYBER-0 has clients sample ν pseudorandom unit directions from a shared seed, evaluate loss differences along them, and send only the resulting scalars; the federator applies any (b,κ)-robust aggregation rule to those projected vectors, and because the aggregated update lies in the span of the perturbation directions, the downlink also carries only Kν scalars. A double application of a Johnson-Lindenstrauss-type projection lemma shows that the robustness inequality defining (b,κ)-robust aggregation is approximately preserved under this transformation, which closes the convergence proof. The paper proves convergence of the averaged squared gradient norm for general non-convex losses under bounded gradient divergence and a pseudo-Lipschitz heterogeneity condition, and it is explicit that the general theorem requires ν=Θ(d), with a secondary Lipschitz-objective theorem still carrying a d/ν term, even though the experiments converge at ν=1.

Load-bearing premise

The whole convergence story rests on the premise that a (b,κ)-robust aggregation rule stays (b,κ)-robust after the clients' updates are replaced by their projections onto ν random directions, a geometric preservation that the proof only establishes for ν large enough, of order d, not for the ν=1 regime where the communication savings are largest.

Editorial extensions

If this is right

  • If correct, federated fine-tuning of large models can be made Byzantine-robust while clients and server exchange only a few scalars per round, because the aggregated update and the model broadcast both live in the low-dimensional perturbation span.
  • Existing robust aggregation rules with proven (b,κ)-robustness, such as coordinate-wise trimmed mean and Krum, and heterogeneity-aware preprocessing like nearest-neighbor mixing, transfer directly to the zero-order setting without redesign.
  • The convergence guarantee extends Byzantine-resilient federated optimization beyond convex and i.i.d. settings to general non-convex losses under bounded data heterogeneity, matching the regime where federated learning is actually deployed.
  • Resource-constrained clients can participate using forward passes and in-place model perturbation, cutting memory needs compared with backpropagation while still receiving attack-robust training.
  • Reported worst-case accuracy is close to full-gradient robust baselines on MNIST logistic regression and RoBERTa-large fine-tuning, with communication reduced by up to seven orders of magnitude.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The sharpest gap the paper leaves open is its own: the general convergence theorem needs ν=Θ(d), and the Lipschitz theorem keeps a d/ν term, so the ν=1 regime that gives the largest communication savings is not yet covered by the proof; a bound that stays finite at constant ν would close the gap.
  • A testable consequence of Theorem 5.9's d/ν scaling is that, with the total number of perturbation-function evaluations νT held fixed, accuracy should be roughly independent of ν; the paper's fine-tuning results are consistent with this, and checking it across model sizes would test the bound's tightness.
  • The shared-seed projection trick suggests a natural extension: replace uniform random directions with data-dependent subspaces that capture the dominant gradient directions, which could keep ν small without paying the Johnson-Lindenstrauss dimension penalty.
  • Because the experiments attack the projected scalars rather than the full gradients, the reported robustness is specifically about the low-dimensional bottleneck; a follow-up question is whether an adversary who knows the seed and the aggregation rule can craft projected updates that defeat the rule even when the honest projected spread is small.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes CYBER-0, a federated zero-order optimization algorithm that combines random perturbation directions, local epochs, a shared-seed pseudorandom direction protocol, and transformed robust aggregation in a low-dimensional perturbation space. The central claim is that CYBER-0 is Byzantine-resilient under general non-convex losses and bounded data heterogeneity, with convergence proved for arbitrary (b,κ)-robust aggregation rules, and with uplink/downlink communication reduced to a few scalars per round. The empirical section reports accuracy on MNIST logistic regression and RoBERTa-large fine-tuning tasks that is comparable to full-gradient Byzantine-resilient baselines, with up to seven orders of magnitude communication savings in the fine-tuning experiments.

Significance. The algorithmic idea is attractive and timely: combining zero-order estimation with robust aggregation in the perturbation space genuinely reduces communication in a Byzantine setting, and the empirical study is broad, covering multiple attacks, aggregation rules, heterogeneity levels, and LLM fine-tuning tasks. If the convergence theory were valid, this would be a substantial advance for communication-efficient robust FL. The paper also deserves credit for explicitly discussing the design trade-off between unbiased and biased local estimators and for performing sensitivity studies over ν, K, and the Byzantine ratio. However, the theoretical guarantees as stated do not cover the regime (ν = 1) that motivates the strongest communication claims, and a load-bearing proof step applies a Johnson–Lindenstrauss lemma in the wrong dimension direction.

major comments (5)
  1. [Appendix B.1, Lemma B.5 and Eq. (9)] Lemma B.5 is applied in the wrong dimension direction. The lemma states that for Z ∈ R^{ν×d} with rows uniform on S^{d−1}, the map x ↦ Zx nearly preserves norms of x ∈ R^d when ν ≥ 64ε^{-2} log(2/δ). In Algorithm 1, however, Z_{t,ℓ} is d×ν with columns as perturbation vectors, and the robustness term in Eq. (9) (and Eq. (22)) is E‖Z_{t,ℓ}(R_{t,ℓ} − ḡ_H)‖², where R_{t,ℓ} − ḡ_H ∈ R^ν. Bounding this term requires a near-isometry from R^ν to R^d, i.e., a condition of the form Z^T Z ≈ I_ν, which holds when d ≥ C ν ε^{-2} log(...), not when ν ≥ 64ε^{-2} log(...). The condition in the lemma is therefore reversed relative to what the proof needs, and the transfer of (b,κ)-robustness from R^ν to R^d is not established by the written argument.
  2. [Theorem 5.9] Theorem 5.9 does not bridge the gap to ν = 1. The final bound contains explicit d/ν factors, in particular the term 2η φ²G²d/ν (1/(5|H|) + 2Kε′κ + 8/|H|²), so for fixed d and ν = 1 this term is of order d and cannot be made small by the stated condition ν ≥ 64 log(2(|H|−1)TK/Δ) alone. Moreover, Theorem 5.7 already requires ν = Θ(d) for the condition c3 + c4c1 ≤ 1/2 to hold, as the paper itself notes. The claim that the analysis covers the ν = 1 empirical regime is therefore unsupported.
  3. [Theorem 5.9 and Lemma B.13] The symbol φ is never defined. It appears in the statement of Theorem 5.9, in Lemma B.10, and in Lemma B.13, and the convergence bound depends on φ², but the paper does not specify what φ is or what assumptions on it are needed. Without a definition, the theorem cannot be evaluated or used.
  4. [Lemma B.5] The statement of Lemma B.5 contains a sign typo: it reads (1−ε)‖x‖² ≤ ‖Zx‖² ≤ (1−ε)‖x‖², but the upper bound should be (1+ε)‖x‖². In addition, the condition as written allows ε = 1 at ν = 64 log(2/δ), which makes the denominator (1−ε) in Eq. (9) zero; the strict condition ε < 1 requires ν > 64 log(2/δ), not ν ≥ 64 log(2/δ). These are presentation issues in the lemma statement, but they matter because the lemma is the cornerstone of the robustness transfer.
  5. [Appendix B.1, Eq. (9) and Eq. (22)] Even if the dimension direction of Lemma B.5 were corrected, the proof applies the preservation bound to a single draw of Z_{t,ℓ} and then claims a union bound over |H| + 1 projections. The same random matrix is used for all honest clients in a round, so the events are not independent, and the union bound as written does not account for the dependence. The high-probability statement in Theorem 5.7 and Theorem 5.9 therefore needs a more careful probabilistic justification.
minor comments (5)
  1. [Section 3.1] There are typos such as 'aggregagation' in the bullet on transformed robust aggregation and 'al clients' in the same section; these should be corrected.
  2. [Appendix A.4] The text contains typos 'expecially' and 'normlized' in the local-epoch comparison section; please proofread the appendix.
  3. [Theorem 5.7] The condition (b) is stated as ν ≥ 64 log(2(|H|−1)TK/Δ), but the proof requires ε < 1 to keep the factor (1−ε) positive; the condition should be strict (ν > 64 log(...)) or the theorem should state ε < 1 explicitly.
  4. [Appendix B.1.1, Lemma B.7] The extracted lemma from Wang et al. (2025) is stated with a term 3L + 3ζ²‖·‖² that appears dimensionally inconsistent; a factor of 1/|H| or an expectation is likely missing. Please verify the statement against the source.
  5. [Table 6] The table in Appendix A.7 is referenced in the main text as 'Table 6' for i.i.d. results, but the numbering and the preceding sentence in Section 4.3 are confusing; please align the table captions with the in-text references.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the convergence bounds are derived from explicit assumptions and external lemmas, and the self-citations are not load-bearing.

full rationale

The paper's central theoretical claim is not circular. The convergence guarantees in Theorems 5.7 and 5.9 are proven from explicit assumptions (Assumptions 5.1-5.4 and 5.8) through a chain of intermediate lemmas (Lemmas B.1-B.7), with constants c1-c5 obtained as explicit bounds rather than fitted to the empirical results. The robustness transfer is justified by an external Johnson-Lindenstrauss statement (Lemma B.5, quoted as Proposition 8 of Li 2024) and the external (b, kappa)-robustness framework of Allouah et al. The self-citations in the paper are descriptive: the authors note that the preliminary version (Neto et al., 2024) treated a convex i.i.d. special case, but that result is not used to derive the non-convex heterogeneous convergence claim. The heterogeneity model is imported from Wang et al. (2025), which is independent external work. The paper also openly states the limitation that Theorem 5.7 requires nu = Theta(d), and Theorem 5.9 is presented with an explicitly added Assumption 5.8 to address the nu = 1 regime; this is an honest and visible assumption, not a parameter silently renamed as a prediction. The skeptic's concern that Lemma B.5 is applied in the wrong dimension direction would, if substantiated, be a correctness or validity flaw in the proof, not a circularity: the conclusion is not identified with an input by construction. Overall, no claim reduces to its own assumptions, and no fitted value is presented as a prediction.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The method introduces no new physical entities or fitted constants beyond standard hyperparameters. The central theoretical results rest on smoothness, variance, and heterogeneity assumptions borrowed from prior literature, plus a JL-type geometric preservation assumption that is only valid for sufficiently many random projections.

free parameters (4)
  • ν (number of random perturbations) = 1 for LLM fine-tuning; 64 for MNIST experiments
    Controls the communication cost and the variance of the zero-order estimator; the theory requires ν to be large (Θ(d) in Theorem 5.7) while the main experiments use ν=1.
  • η (learning rate) = 0.01 for MNIST, 10^-6 for fine-tuning
    Chosen by hand per task; the proofs require upper bounds on η but do not prescribe values.
  • µ (perturbation scale) = 0.001 for both experiment suites
    Smoothing parameter for the two-point zero-order estimate; set by hand.
  • K (local epochs) = varied, e.g., 1, 3, 5, 10 in ablation
    Number of local updates before communication; a design choice that trades bias and communication.
assumptions (7)
  • domain assumption Assumption 5.1: Each local loss has L-Lipschitz gradients
    Used throughout the proofs to bound function decrease and gradient differences; standard smoothness condition.
  • domain assumption Assumption 5.2: Bounded gradient estimation variance σ^2
    Controls the stochastic noise of mini-batch gradient estimates; needed in Lemmas B.2 and B.6.
  • domain assumption Assumption 5.3: Bounded gradient divergence between each client and the honest average, ζ^2
    Models data heterogeneity; taken from Wang et al. (2025) and used in Lemmas B.1 to B.4.
  • domain assumption Assumption 5.4: Pseudo-Lipschitz condition on averaged gradients with constant D^2
    Borrowed from Wang et al. (2025); it is a global regularity condition on the heterogeneous objectives and is not verified empirically.
  • domain assumption Assumption 5.8: Each local objective is G-Lipschitz
    Added to support Theorem 5.9 for small ν; the resulting bound still depends on d/ν.
  • standard math Johnson-Lindenstrauss embedding preserves Euclidean distances for ν random projections
    Used to transfer robust aggregation guarantees from the original gradient space to the projected scalar space in Lemma B.5 and equations (9) and (22).
  • domain assumption The aggregation rule satisfies Definition 2.2 with (b,κ)-robustness
    CWTM, Krum and NNM compositions are assumed to satisfy this condition; the proofs rely on the definition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning." pith.science (2026). https://pith.science/paper/PLAPXQH6

@misc{pith2026250200193,
  author       = {Pith},
  title        = {Pith review of: Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PLAPXQH6}},
  note         = {Machine review of arXiv:2502.00193}
}
read the original abstract

We introduce CyBeR-0, a Byzantine-resilient federated zero-order optimization method that is robust under Byzantine attacks and provides significant savings in uplink and downlink communication costs. We introduce transformed robust aggregation to give convergence guarantees for general non-convex objectives under client data heterogeneity. Empirical evaluations for standard learning tasks and fine-tuning large language models show that CyBeR-0 exhibits stable performance with only a few scalars per-round communication cost and reduced memory requirements.

Figures

Figures reproduced from arXiv: 2502.00193 by the authors.

Figure 1
Figure 1. Performance of different robust aggregation rules against different attacks for logistic regression on MNIST. results. We give an extensive hyperparameter study for CYBER-0 in fine-tuning tasks. We run our experiments on SST-2 and RoBERTa-large, attacked by FOE. We use non-i.i.d. data with α = 0.1. Please see Appendix A.8 for detailed re￾sults. We show the robustness of CYBER-0 under varying numbers of total and Byz… view at source ↗
Figure 2
Figure 2. Accuracy over epochs for fine-tuning RoBERTa-large on TREC under different attack scenarios for non-i.i.d. data. erogeneous data distribution and general non-convex loss functions. We employ the heterogeneity model introduced by Wang et al. (2025) in our analysis. Challenges and novelty. While Allouah et al. (2023) pro￾vides a roadmap for analyzing robustness in non-convex regimes, the combination with ZO methods in… view at source ↗
Figure 3
Figure 3. Comparisons of Local Epoch Strategies for µ = 0.001 0 50 100 150 200 250 300 350 400 20 40 60 80 Epoch Accuracy µ = 0, K = 1 µ = 0, K = 3 (Biased) µ = 0, K = 3 (Unbiased Compressed) µ = 0, K = 3 (Unbiased) µ = 0, K = 10 (Biased) µ = 0, K = 10 (Unbiased Compressed) µ = 0, K = 10 (Unbiased) (a) Accuracy over Epochs 0 50 100 150 200 250 300 350 400 20 40 60 80 Normalized Communication Cost Accuracy µ = 0, K = 1 µ = 0, … view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Comparisons of Local Epoch Strategies for µ = 0 for ℓ ̸= m ∈ [K]. It suffices for the clients to communicate to the federator PK ℓ=1 gi(wi t,ℓ, Zt,ℓ), thus reducing the communication cost by the same factor of K as for the second approach above. However, this strategy …
Figure 5
Figure 5. Figure 5: Comparison of Zero-Order Optimization for Different Values of ν Compared to the Baseline FedAvg. A.6 Accuracies over Epochs for all Attacks on MNIST We present in the following, extending the case of LF (cf [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: ALIE attack on logistic regression on MNIST. 0 50 100 150 200 250 300 350 400 10 20 30 40 50 60 70 80 90 Epoch Accuracy CYBER-0-CWTM-NNM CYBER-0-Krum-NNM FedAvg-CWTM-NNM FedAvg-Krum-NNM (a) Accuracy over Epochs 10−2 10−1 100 101 102 10 20 30 40 50 60 70 80 90 Normalize…
Figure 7
Figure 7. Figure 7: ALIE-NNM attack on logistic regression on MNIST. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: FOE attack on logistic regression on MNIST. 0 50 100 150 200 250 300 350 400 10 20 30 40 50 60 Epoch Accuracy CYBER-0-CWTM-NNM CYBER-0-Krum-NNM FedAvg-CWTM-NNM FedAvg-Krum-NNM (a) Accuracy over Epochs 10−2 10−1 100 101 102 10 20 30 40 50 60 Normalized Communication Cos…
Figure 9
Figure 9. Figure 9: FOE-NNM attack on logistic regression on MNIST. 0 50 100 150 200 250 300 350 400 0 10 20 30 40 50 60 70 Epoch Accuracy CYBER-0-CWTM CYBER-0-CWTM-NNM CYBER-0-Krum CYBER-0-Krum-NNM FedAvg-CWTM FedAvg-CWTM-NNM FedAvg-Krum FedAvg-Krum-NNM (a) Accuracy over Epochs 10−2 10−1…
Figure 10
Figure 10. Figure 10: SF attack on logistic regression on MNIST. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: LF attack on logistic regression on MNIST. A.7 Accuracies over Epochs for all Attacks on Fine-Tuning Tasks We provide in [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Accuracy comparison of different attacks on fine-tuning RoBERTa-large on SST-2 with i.i.d. data, compared to the baseline. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Accuracy comparison of different attacks on fine-tuning RoBERTa-large on SNLI with i.i.d. data, compared to the baseline. 0 0.5 1 1.5 2 2.5 3 3.5 4 ·104 0.4 0.6 0.8 Epoch Accuracy No Attack ALIE FOE SF TMA [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Accuracy comparison of different attacks on fine-tuning RoBERTa-large on SNLI with i.i.d. data, compared to the baseline. 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 ·104 0.8 0.85 0.9 Epoch Accuracy No Attack ALIE FOE SF TMA [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Accuracy comparison of different attacks on fine-tuning RoBERTa-large on SST-2 with non-i.i.d. data, compared to the baseline. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]
Figure 16
Figure 16. Figure 16: Accuracy comparison of different attacks on fine-tuning RoBERTa-large on SNLI with non-i.i.d. data, compared to the baseline. 0 0.5 1 1.5 2 2.5 3 3.5 4 ·104 0.4 0.6 0.8 Epoch Accuracy No Attack ALIE FOE SF TMA [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Accuracy comparison of different attacks on fine-tuning RoBERTa-large on SNLI with non-i.i.d. data, compared to the baseline. A.8 Hyperparameter Study We provide in the following tables Tables 7 to 10 a sensitivity analysis of CYBER-0 with respect to the number of glo…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Private Aggregation for Byzantine-Resilient Heterogeneous Federated Learning

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Nearest neighbor mixing can be composed with secure aggregation and private information retrieval to give information-theoretic privacy and Byzantine resilience for heterogeneous federated learning.

Reference graph

Works this paper leans on

56 extracted references · 46 canonical work pages · cited by 1 Pith paper

  1. [1]

    Byzantine-resilient non-convex stochastic gradient descent

    Allen-Zhu, Z., Ebrahimianghazani, F., Li, J., and Alistarh, D. Byzantine-resilient non-convex stochastic gradient descent. In International Conference on Learning Representations, 2021

    work page 2021

  2. [2]

    Fixing by mixing: A recipe for optimal byzantine ml under heterogeneity

    Allouah, Y., Farhadkhani, S., Guerraoui, R., Gupta, N., Pinot, R., and Stephan, J. Fixing by mixing: A recipe for optimal byzantine ml under heterogeneity. In International Conference on Artificial Intelligence and Statistics, pp.\ 1232--1300, 2023

    work page 2023

  3. [3]

    Differential privacy has disparate impact on model accuracy

    Bagdasaryan, E., Poursaeed, O., and Shmatikov, V. Differential privacy has disparate impact on model accuracy. Advances in neural information processing systems, 32, 2019

    work page 2019

  4. [4]

    A little is enough: Circumventing defenses for distributed learning

    Baruch, G., Baruch, M., and Goldberg, Y. A little is enough: Circumventing defenses for distributed learning. Advances in Neural Information Processing Systems, 32, 2019

    work page 2019

  5. [5]

    M., Guerraoui, R., and Stainer, J

    Blanchard, P., El Mhamdi, E. M., Guerraoui, R., and Stainer, J. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in neural information processing systems, 30, 2017

    work page 2017

  6. [6]

    R., Angeli, G., Potts, C., and Manning, C

    Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D. A large annotated corpus for learning natural language inference. In Empirical Methods in Natural Language Processing, pp.\ 632--642, 2015

    work page 2015

  7. [7]

    Learning from untrusted data

    Charikar, M., Steinhardt, J., and Valiant, G. Learning from untrusted data. In ACM SIGACT Symposium on Theory of Computing, pp.\ 47--60, 2017

    work page 2017

  8. [8]

    Fine-grained theoretical analysis of federated zeroth-order optimization

    Chen, J., Chen, H., Gu, B., and Deng, H. Fine-grained theoretical analysis of federated zeroth-order optimization. In Neural Information Processing Systems, 2023

    work page 2023

Show all 56 references
  1. [9]

    C., Jordan, M

    Duchi, J. C., Jordan, M. I., Wainwright, M. J., and Wibisono, A. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE Transactions on Information Theory, 61 0 (5): 0 2788--2806, 2015

    2015

  2. [10]

    Private aggregation in wireless federated learning with heterogeneous clusters

    Egger, M., Hofmeister, C., Wachter-Zeh, A., and Bitar, R. Private aggregation in wireless federated learning with heterogeneous clusters. In IEEE International Symposium on Information Theory (ISIT), pp.\ 54--59, 2023

    2023

  3. [11]

    M., Farhadkhani, S., Guerraoui, R., Guirguis, A., Hoang, L.-N., and Rouault, S

    El-Mhamdi, E. M., Farhadkhani, S., Guerraoui, R., Guirguis, A., Hoang, L.-N., and Rouault, S. Collaborative learning in the jungle (decentralized, byzantine, heterogeneous, asynchronous and nonconvex learning). Advances in neural information processing systems, 34: 0 25044--25...

    2021

  4. [12]

    Local model poisoning attacks to byzantine-robust federated learning

    Fang, M., Cao, X., Jia, J., and Gong, N. Local model poisoning attacks to byzantine-robust federated learning. In USENIX security symposium, pp.\ 1605--1622, 2020

    2020

  5. [13]

    N., and Zhou, Y

    Fang, W., Yu, Z., Jiang, Y., Shi, Y., Jones, C. N., and Zhou, Y. Communication-efficient stochastic zeroth-order optimization for federated learning. IEEE Transactions on Signal Processing, 70: 0 5058--5073, 2022

    2022

  6. [14]

    D., Kalai, A

    Flaxman, A. D., Kalai, A. T., and McMahan, H. B. Online convex optimization in the bandit setting: gradient descent without a gradient. In ACM-SIAM Symposium on Discrete Algorithms, pp.\ 385–394, 2005

    2005

  7. [15]

    On the information-adaptive variants of the admm: an iteration complexity perspective

    Gao, X., Jiang, B., and Zhang, S. On the information-adaptive variants of the admm: an iteration complexity perspective. Journal of Scientific Computing, 76: 0 327--363, 2018

    2018

  8. [16]

    and Lan, G

    Ghadimi, S. and Lan, G. Stochastic first- and zeroth-order methods for nonconvex stochastic programming. SIAM Journal on Optimization, 23 0 (4): 0 2341--2368, 2013

    2013

  9. [17]

    Black-box adversarial attacks with limited queries and information

    Ilyas, A., Engstrom, L., Athalye, A., and Lin, J. Black-box adversarial attacks with limited queries and information. In International Conference on Machine Learning, pp.\ 2137--2146, 2018

    2018

  10. [18]

    A., Li, S., and Caire, G

    Jahani-Nezhad, T., Maddah-Ali, M. A., Li, S., and Caire, G. SwiftAgg+ : Achieving asymptotically optimal communication loads in secure aggregation for federated learning. IEEE Journal on Selected Areas in Communications, 41 0 (4): 0 977--989, 2023

    2023

  11. [19]

    and Lindenstrauss, J

    Johnson, W. and Lindenstrauss, J. Extensions of lipschitz maps into a hilbert space. Contemporary Mathematics, 26: 0 189--206, 01 1984

    1984

  12. [20]

    P., Rebjock, Q., Stich, S., and Jaggi, M

    Karimireddy, S. P., Rebjock, Q., Stich, S., and Jaggi, M. Error feedback fixes SignSGD and other gradient compression schemes. In International Conference on Machine Learning, pp.\ 3252--3261, 2019

    2019

  13. [21]

    P., He, L., and Jaggi, M

    Karimireddy, S. P., He, L., and Jaggi, M. Byzantine-robust learning on heterogeneous datasets via bucketing. In International Conference on Learning Representations, 2022

    2022

  14. [22]

    and Wolfowitz, J

    Kiefer, J. and Wolfowitz, J. Stochastic Estimation of the Maximum of a Regression Function . The Annals of Mathematical Statistics, 23 0 (3): 0 462 -- 466, 1952

    1952

  15. [23]

    Learning to detect malicious clients for robust federated learning

    Li, S., Cheng, Y., Wang, W., Liu, Y., and Chen, T. Learning to detect malicious clients for robust federated learning. arXiv preprint arXiv:2002.00211, 2020

    2002 arXiv

  16. [24]

    Simple, unified analysis of johnson-lindenstrauss with applications

    Li, Y. Simple, unified analysis of johnson-lindenstrauss with applications. arXiv preprint arXiv:2402.10232, 2024

    2024 arXiv

  17. [25]

    O., and Varshney, P

    Liu, S., Chen, P.-Y., Kailkhura, B., Zhang, G., Hero III, A. O., and Varshney, P. K. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, 37 0 (5): 0 43--54, 2020

    2020

  18. [26]

    Liu, S., Gupta, N., and Vaidya, N. H. Approximate byzantine fault-tolerance in distributed optimization. In ACM Symposium on Principles of Distributed Computing, pp.\ 379--389, 2021

    2021

  19. [27]

    Roberta: A robustly optimized bert pretraining approach

    Liu, Y. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364, 2019

    1907 arXiv

  20. [28]

    An efficient statistical-based gradient compression technique for distributed training systems

    M Abdelmoniem, A., Elzanaty, A., Alouini, M.-S., and Canini, M. An efficient statistical-based gradient compression technique for distributed training systems. Proceedings of Machine Learning and Systems, 3: 0 297--322, 2021

    2021

  21. [29]

    V., Bondaschi, M., Vogels, T., Jaggi, M., Kim, H., and Gastpar, M

    Makkuva, A. V., Bondaschi, M., Vogels, T., Jaggi, M., Kim, H., and Gastpar, M. LASER : Linear compression in wireless distributed optimization. In International Conference on Machine Learning, 2024

    2024

  22. [30]

    D., Chen, D., and Arora, S

    Malladi, S., Gao, T., Nichani, E., Damian, A., Lee, J. D., Chen, D., and Arora, S. Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems, 36: 0 53038--53075, 2023

    2023

  23. [31]

    McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp.\ 1273--1282, 2017

    2017

  24. [32]

    Neto, A. d. S. D., Egger, M., Bakshi, M., and Bitar, R. Communication-efficient byzantine-resilient federated zero-order optimization. submitted to ICML 2024; arXiv preprint arXiv:2406.14362, 2024

    2024 arXiv

  25. [33]

    Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes

    Qin, Z., Chen, D., Qian, B., Ding, B., Li, Y., and Deng, S. Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes. arXiv preprint arXiv:2312.06353, 2023

    2023 arXiv

  26. [34]

    Zeroth-order methods for nondifferentiable, nonconvex, and hierarchical federated optimization

    Qiu, Y., Shanbhag, U., and Yousefian, F. Zeroth-order methods for nondifferentiable, nonconvex, and hierarchical federated optimization. Advances in Neural Information Processing Systems, 36, 2023

    2023

  27. [35]

    V., Herrera, F., and Mart \' nez-C \'a mara, E

    Rodr \' guez-Barroso, N., Jim \'e nez-L \'o pez, D., Luz \'o n, M. V., Herrera, F., and Mart \' nez-C \'a mara, E. Survey on federated learning threats: Concepts, taxonomy on attacks and defences, experimental study and challenges. Information Fusion, 90: 0 148--173, 2023

    2023

  28. [36]

    Evolution strategies as a scalable alternative to reinforcement learning

    Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864, 2017

    2017 arXiv

  29. [37]

    K., Moraes, M

    Salmon, J. K., Moraes, M. A., Dror, R. O., and Shaw, D. E. Parallel random numbers: as easy as 1, 2, 3. In International conference for high performance computing, networking, storage and analysis, pp.\ 1--12, 2011

    2011

  30. [38]

    Schlegel, R., Kumar, S., Rosnes, E., and i Amat, A. G. CodedPaddedFL and CodedSecAgg : Straggler mitigation and secure aggregation in federated learning. IEEE Transactions on Communications, 2023

    2023

  31. [39]

    Auror: Defending against poisoning attacks in collaborative deep learning systems

    Shen, S., Tople, S., and Saxena, P. Auror: Defending against poisoning attacks in collaborative deep learning systems. In Annual Conference on Computer Security Applications, pp.\ 508--519, 2016

    2016

  32. [40]

    D., Ng, A

    Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In Conference on empirical methods in natural language processing, pp.\ 1631--1642, 2013

    2013

  33. [41]

    Multivariate stochastic approximation using a simultaneous perturbation gradient approximation

    Spall, J. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control, 37 0 (3): 0 332--341, 1992

    1992

  34. [42]

    Private fine-tuning of large language models with zeroth-order optimization

    Tang, X., Panda, A., Nasr, M., Mahloujifar, S., and Mittal, P. Private fine-tuning of large language models with zeroth-order optimization. CoRR, 2024 a

    2024

  35. [43]

    Distributed zero-order algorithms for nonconvex multiagent optimization

    Tang, Y., Zhang, J., and Li, N. Distributed zero-order algorithms for nonconvex multiagent optimization. IEEE Transactions on Control of Network Systems, 8 0 (1): 0 269--281, 2020

    2020

  36. [44]

    z -SignFedAvg : A unified stochastic sign-based compression for federated learning

    Tang, Z., Wang, Y., and Chang, T.-H. z -SignFedAvg : A unified stochastic sign-based compression for federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 15301--15309, 2024 b

    2024

  37. [45]

    A hybrid approach to privacy-preserving federated learning

    Truex, S., Baracaldo, N., Anwar, A., Steinke, T., Ludwig, H., Zhang, R., and Zhou, Y. A hybrid approach to privacy-preserving federated learning. In ACM workshop on artificial intelligence and security, pp.\ 1--11, 2019

    2019

  38. [46]

    P., and Jaggi, M

    Vogels, T., Karimireddy, S. P., and Jaggi, M. PowerSGD : Practical low-rank gradient compression for distributed optimization. Advances in Neural Information Processing Systems, 32, 2019

    2019

  39. [47]

    Voorhees, E. M. and Tice, D. M. Building a question answering test collection. In Annual international ACM SIGIR conference on Research and development in information retrieval, pp.\ 200--207, 2000

    2000

  40. [48]

    Wang, J., Liu, Q., Liang, H., Joshi, G., and Poor, H. V. A novel framework for the analysis and design of heterogeneous federated learning. IEEE Transactions on Signal Processing, 69: 0 5234--5249, 2021

    2021

  41. [49]

    A new theoretical perspective on data heterogeneity in federated optimization

    Wang, J., Wang, S., Chen, R.-R., and Ji, M. A new theoretical perspective on data heterogeneity in federated optimization. In International Conference on Machine Learning, 2025

    2025

  42. [50]

    H., Farokhi, F., Jin, S., Quek, T

    Wei, K., Li, J., Ding, M., Ma, C., Yang, H. H., Farokhi, F., Jin, S., Quek, T. Q., and Poor, H. V. Federated learning with differential privacy: Algorithms and performance analysis. IEEE Transactions on Information Forensics and Security, 15: 0 3454--3469, 2020

    2020

  43. [51]

    TernGrad : Ternary gradients to reduce communication in distributed deep learning

    Wen, W., Xu, C., Yan, F., Wu, C., Wang, Y., Chen, Y., and Li, H. TernGrad : Ternary gradients to reduce communication in distributed deep learning. Advances in neural information processing systems, 30, 2017

    2017

  44. [52]

    Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation

    Xie, C., Koyejo, O., and Gupta, I. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation. In Uncertainty in Artificial Intelligence, pp.\ 261--270, 2020

    2020

  45. [53]

    Byzantine-robust distributed learning: Towards optimal statistical rates

    Yin, D., Chen, Y., Kannan, R., and Bartlett, P. Byzantine-robust distributed learning: Towards optimal statistical rates. In International conference on machine learning, pp.\ 5650--5659, 2018

    2018

  46. [54]

    Federated learning with non-iid data

    Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., and Chandra, V. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582, 2018

    2018 arXiv

  47. [55]

    Federated learning on non-iid data: A survey

    Zhu, H., Xu, J., Liu, S., and Jin, Y. Federated learning on non-iid data: A survey. Neurocomputing, 465: 0 371--390, 2021

    2021

  48. [56]

    and Philip, S

    Zhu, T. and Philip, S. Y. Applying differential privacy mechanism in artificial intelligence. In IEEE International Conference on Distributed Computing Systems (ICDCS), pp.\ 1601--1609, 2019

    2019

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.