Pith. sign in

REVIEW 2 major objections 5 minor 39 references

Constant Stepsize Local GD for Logistic Regression: Acceleration by Instability

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Local GD on separable logistic regression converges for every positive step size and every communication interval, and the instability of large updates accelerates the tail rate.

desk verdict Real result, solid proof structure, but the transition time τ in Theorem 4.2 is understated by a misapplied log factor, making Lemma 4.9 (and hence Theorems 4.2 and Corollary 4.3) unproven as stated; the rate likely survives with a corrected τ. read the letter →

arxiv 2506.13974 v1 pith:Z5LNDR3M submitted 2025-06-16 cs.LG

Michael Crawshaw , Blake Woodworth , Mingrui Liu This is my paper · ORCID
classification cs.LG MSC 68W4090C25
keywords localgradientdescentfederatedaveraginglogisticregressionconstantstepsizelinearseparabilityheterogeneousdataconvergencerateinstability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Local gradient descent, where clients take K gradient steps before averaging, has only been analyzed under small step sizes that keep the loss monotonically decreasing. This paper claims that for separable logistic regression no such restriction is needed: any positive step size and any communication interval converge. The mechanism is that instability itself accelerates the algorithm: large local updates produce an initial phase of non-monotonic loss, followed by a stable phase of monotone decrease. If correct, the result gives an O~(1/$R^{2}$) rate after a short unstable phase, improving on the O(1/R) worst-case bound for general convex objectives and matching a known phenomenon in single-machine large-stepsize gradient descent.

What carries the argument

The load-bearing object is the coefficient $\beta^m_{r,i}=\frac1K\sum_{k=0}^{K-1}|\ell'(\langle w^m_{r,k},x^m_i\rangle)|/|\ell'(\langle w_r,x^m_i\rangle)|$, which measures how much a single data point's contribution grows during $K$ local steps. Lower bounding $\beta$ by $1/K$ and upper bounding it by $\exp(\|w_r\|)$ lets the authors adapt the split-comparator and gradient-potential arguments from large-stepsize single-machine gradient descent to the distributed setting during the unstable phase. In the stable phase, the refined descent inequality $F_m(w')\le F_m(w)+\langle\nabla F_m(w),w'-w\rangle+4F_m(w)\|w'-w\|^2$, together with $\|\nabla^2F_m(w)\|\le F_m(w)$ and $\|\nabla F_m(w)\|\le F_m(w)$, turns a small objective value into monotone decrease.

What would settle it

Train Local GD on a linearly separable synthetic dataset with known margin $\gamma$, choose $\eta K = c\gamma^3 R/M$ for large $R$, and measure $F(w_R)$; if the error does not decay like $1/R^2$ (or if the loss diverges for very large $\eta$ with $K$ fixed), the central acceleration-by-instability claim fails.

Watch

Extended reading notes

Core claim

The central claim is that Local GD on linearly separable, heterogeneous logistic regression converges for every positive step size $\eta$ and every number of local steps $K$. Averaging over the first $r$ rounds, the objective is at most $O((\|w_0\|^2+1+\log^2(K+\eta K\gamma^2 r)+\eta^2K^2)/(\eta \gamma^4 r))$; after a transition time $\tau$, it decreases monotonically and $F(w_r)\le 16/(\eta\gamma^2 K(r-\tau))$. Choosing $\eta K=\tilde\Theta(\gamma^3 R/M)$ yields $F(w_R)\le \tilde O(M/(\gamma^5 R^2))$, which improves on the $O(1/R)$ worst-case baseline. The proof compares the trajectory of Local GD to that of single-machine GD by writing each round's update as a weighted sum of data points and bounding the per-data-point coefficient ratios.

Load-bearing premise

The global dataset is linearly separable, so the margin $\gamma$ is positive; all the rates scale with inverse powers of $\gamma$, and without separability the guarantees become vacuous.

Editorial extensions

If this is right

  • For any fixed step size and communication interval, the average loss over the first $r$ rounds tends to zero, so no step-size cap is needed for convergence.
  • Once the loss falls below a small threshold, every subsequent round decreases the objective monotonically, at rate $16/(\eta\gamma^2 K(r-\tau))$.
  • With $\eta K$ chosen as $\tilde\Theta(\gamma^3 R/M)$, the error after $R$ rounds is $\tilde O(M/(\gamma^5 R^2))$, an improvement over the $O(1/R)$ worst-case rate for general smooth convex objectives.
  • The rate depends on the global margin $\gamma$, not on how the data are distributed among clients; heterogeneous splits do not change the bound.
  • The transition time to the stable phase is proportional to $\eta K$ and scales with $1/(\gamma^2\psi)$, so large step sizes and long intervals delay stability but still pay off in the final rate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the rate is tight, the practical recipe for separable federated problems is to push the product $\eta K$ into the unstable regime; the loss spikes early but the tail convergence is faster than any monotone small-step schedule.
  • Because the bound treats $\eta$ and $K$ only through the product $\eta K$, it does not distinguish local steps from a single larger global step; the experiments suggest $K$ may shorten the unstable phase, so a sharper transition-time analysis could be the place to prove a genuine benefit of local steps.
  • A natural extension is to Local SGD: the same $\beta$-coefficient comparison should carry over, since a stochastic analogue of the single-machine large-stepsize analysis exists for logistic regression.
  • The reliance on margin suggests a testable boundary: as data approach non-separability ($\gamma\to 0$), the predicted time to stability diverges, so large-stepsize Local GD should visibly slow down or diverge on nearly non-separable datasets.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper analyzes constant-stepsize Local GD for distributed logistic regression on linearly separable, heterogeneous data. The main results are Theorem 4.1, an average-iterate bound valid for every step size eta > 0 and communication interval K >= 1; Theorem 4.2, a last-iterate bound F(w_r) = O(1/(eta K (r - tau))) after a transition time tau; and Corollary 4.3, which derives the error O~(M/(gamma^5 R^2)) after tuning eta K ~ gamma^3 R/M. The proof decomposes a Local GD round into per-data-point coefficients beta^m_{r,i}, compares the trajectory with gradient descent via a split-comparator argument, and uses a gradient potential to show that the objective eventually enters a stable phase with monotone decrease. Experiments on synthetic, MNIST, and CIFAR-10 data support the qualitative phenomenon of acceleration through instability.

Significance. If the results are correct, the paper gives the first convergence guarantee for vanilla Local GD with unrestricted step size and communication interval in a non-worst-case problem class, improving on the worst-case O(1/R) rate for general smooth convex objectives. The appendix contains detailed proofs with explicit constants, and the paper is candid about its limitations: the rate is worse than single-machine GD in M and 1/gamma, no advantage of K > 1 is shown, and the separability assumption gamma > 0 is essential. These strengths make the contribution worth preserving, but one load-bearing point in the transition-time argument needs repair before the stated theorems are supported.

major comments (2)
  1. [Appendix A.2, Lemma 4.9 / Theorem 4.2] The transition time tau printed in Theorem 4.2 does not satisfy the sufficient condition derived in Lemma 4.9. With A = (2 gamma ||w0|| + sqrt(2) + eta)/(eta gamma^2 psi), B = 1/(eta gamma^2 psi), and C = eta gamma^2 K, Lemma B.6 requires r >= 2A + B log(1 + B sqrt(C)), i.e., an additive term (1/(eta gamma^2 psi)) log(1 + sqrt(K)/(sqrt(eta) gamma psi)). The manuscript instead writes log(1 + sqrt(K) sqrt(eta gamma psi)) in Eq. (6) and Eq. (130). These arguments differ by the factor 1/(eta gamma^2 psi), which can be arbitrarily large, so the stated tau is not guaranteed to satisfy Eq. (128), and the proof of Lemma 4.9 does not establish the existence of r <= tau with F(w_r) <= gamma/(70 eta K M). Theorem 4.2 and Corollary 4.3 are therefore unsupported as printed. The fix appears local: replace the logarithmic term by log(1 + sqrt(K)/(sqrt(eta) gamma psi)); the tilde-rate in Corollary 4.3 should survive.
  2. [Appendix A.1, Eq. (64)-(65)] In the proof of Lemma 4.4, the transition from Eq. (64) to Eq. (65) appears to drop a factor of 2: the bound derived in Eq. (64) is (2 log(1 + eta gamma^2 K r^2) + eta + sqrt(2))/gamma, while Lemma 4.4 claims (log(1 + eta gamma^2 K r^2) + eta + sqrt(2))/gamma. Either the proof must justify the reduction or the lemma should state the larger bound. Since the term is logarithmic, the asymptotic results are unaffected, but the proof as written does not establish the stated lemma.
minor comments (5)
  1. [Appendix A.2, Eq. (120)] Equation (120) uses |ell'(<w^m_{r,k}, x^{m,i}>)| inside the beta-weighted term without a summation over k and with an unspecified k; consistency with the definition of beta^m_{r,i} requires |ell'(<w_r, x^{m,i}>)|. This appears to be an indexing typo.
  2. [Lemma 4.7] Lemma 4.7's condition F(w_r) <= 1/(eta K M) does not imply the condition F(w_r) <= 1/(4 eta M) needed to invoke Lemma 4.6 when K < 4. The later applications of Lemma 4.7 use the stronger threshold gamma/(70 eta K M), so the main proof is unaffected, but the lemma statement or proof should be adjusted.
  3. [Theorem 4.1] Theorem 4.1 is stated for every r >= 0, but the left-hand side is an average over r rounds and is undefined at r = 0; it should state r >= 1. Similarly, the transition time tau in Theorem 4.2 should be read as an integer ceiling.
  4. [Abstract] The abstract says the unstable phase lasts O~(eta K M) rounds, but the proof of Corollary 4.3 gives tau = O~(max(eta K M/gamma^3, M n/gamma^2)); the dependence on gamma and the second term should be reflected in the abstract for accuracy.
  5. [Section 4.2, Eq. (16)] Equation (16) has mismatched parentheses in <w_r, x^m_i)>; it should be <w_r, x^m_i>.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the Local GD convergence bounds are derived from the algorithm's update equations and standard logistic-loss lemmas; the same-author lemmas cited are parameter-free and do not assume the target results.

full rationale

The derivation chain is not circular. Theorems 4.1 and 4.2 are proven from the algorithm's update equations (Algorithm 1, Eqs. (9)-(12)), the logistic-loss identities ||∇F|| ≤ F and ||∇F|| ≥ (γ/2)F under separability, and the split-comparator technique of Wu et al. (2024a), which is external to this paper. The only same-author inputs are Lemmas B.1-B.3, quoted from Crawshaw et al. (2025); these are parameter-free properties of the logistic loss under explicit assumptions (||x_i|| ≤ 1 and linear separability) that do not include any of the present convergence conclusions, so they are independent support rather than a smuggled ansatz. The β-coefficient decomposition (Eqs. (10)-(12)) is a bookkeeping device, not a fitted quantity, and the lower bound β ≥ 1/K (Eq. (14)) is derived by inspection of the sum, not assumed as a target-dependent condition. The transition-time Lemma 4.9 follows from a potential-function argument (Eqs. (116)-(131)) using the same β bound; the quantities ψ and τ in Theorem 4.2 are explicit and not tuned to match the claimed rate. Corollary 4.3's choice ηK = Θ~(γ^3 R/M) is a theoretical tuning calculation (Appendix A.3), not a fit to the data being predicted. The paper candidly states limitations (Section 6: the bound is worse than GD; no proven benefit of K > 1; implicit bias unknown, Section 4.3), which further cuts against circularity. The skeptic's concern about the logarithmic argument in Lemma B.6 and the printed τ is an algebraic-correctness issue, not a circularity; even if τ were miscalculated, the derivation would not reduce to its own inputs. Overall, no prediction or first-principles result is equivalent by construction to a fitted parameter or to a self-citation chain.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fit; the analysis is a pure worst-case bound. The proof imports three logistic-loss-specific lemmas from a self-cited prior paper, and assumes separability and bounded features. No new physical or algorithmic entities are introduced.

assumptions (3)
  • domain assumption The global dataset is linearly separable, so the margin gamma > 0 (Eq. 1-2).
    All rates in Theorems 4.1, 4.2 and Corollary 4.3 depend on 1/gamma; if gamma = 0 the bounds are vacuous. This restriction is stated in Section 3.
  • domain assumption Each data point has norm ||x_i^m|| <= 1 (Section 3).
    Used via Cauchy-Schwarz and to bound H <= 1/4; can be enforced by rescaling, so it is a normalization rather than a substantive restriction.
  • domain assumption Lemmas B.1-B.3 from Crawshaw et al. (2025): ||grad F_m(w)|| <= F_m(w), ||grad F(w)|| <= F(w), ||grad F(w)|| >= (gamma/2) F(w) when all margins are positive, and Hessian growth bounds.
    These logistic-loss-specific bounds are imported without proof; they are proven in a prior ICLR 2025 paper by overlapping authors. They are used in the stable phase and the potential argument.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Constant Stepsize Local GD for Logistic Regression: Acceleration by Instability." pith.science (2026). https://pith.science/paper/Z5LNDR3M

@misc{pith2026250613974,
  author       = {Pith},
  title        = {Pith review of: Constant Stepsize Local GD for Logistic Regression: Acceleration by Instability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z5LNDR3M}},
  note         = {Machine review of arXiv:2506.13974}
}
abstract

Existing analysis of Local (Stochastic) Gradient Descent for heterogeneous objectives requires stepsizes $\eta \leq 1/K$ where $K$ is the communication interval, which ensures monotonic decrease of the objective. In contrast, we analyze Local Gradient Descent for logistic regression with separable, heterogeneous data using any stepsize $\eta > 0$. With $R$ communication rounds and $M$ clients, we show convergence at a rate $\mathcal{O}(1/\eta K R)$ after an initial unstable phase lasting for $\widetilde{\mathcal{O}}(\eta K M)$ rounds. This improves upon the existing $\mathcal{O}(1/R)$ rate for general smooth, convex objectives. Our analysis parallels the single machine analysis of~\cite{wu2024large} in which instability is caused by extremely large stepsizes, but in our setting another source of instability is large local updates with heterogeneous objectives.

Figures

Figures reproduced from arXiv: 2506.13974 by the authors.

Figure 1
Figure 1. Objective gap when varying one of η, K and keeping the other fixed. In general, Local GD converges faster when η and K are larger, despite the initial instability in early rounds. 1 11 1 1  1 1 1 1  K K 1   K    K 1   K    1 11 1 1  1 11 1 11  K K 1   K    K 1   K    [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Objective gap for different values of K with tuned η. Left: Synthetic data. Right: MNIST data. tends to decrease as K increases, which still suggests that there may be room for improvement in our bound of the transition time in Theorem 4.2. Together, our experimental results confirm that instability is an important ingredient for the fast convergence of Local GD for logistic regression. Further, they suggest that Lo… view at source ↗
Figure 4
Figure 4. Train loss of Local GD (step size η, communication interval K) with the CIFAR-10 dataset. Overall, we observe that Local GD converges faster in the long run by choosing a larger step size/communication interval, despite unstable/slow optimization in early iterations. For (a), we first fix K = 16 while varying η, then fix η = 29 while varying K. and γ1 = 1, γ2 = 1/g. By choosing δ close to zero and g with large magni… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Three splits of a synthetic dataset. Binary labels are shown in red/blue, and client indices for each data point [PITH_FULL_IMAGE:figures/full_fig_p027_5.png]
Figure 6
Figure 6. Figure 6: Results of Local GD on three splits of the synthetic dataset pictured in Figure [PITH_FULL_IMAGE:figures/full_fig_p028_6.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 20 canonical work pages

  1. [1]

    and Shamir, O

    Arjevani, Y. and Shamir, O. Communication complexity of distributed convex learning and optimization. Advances in neural information processing systems, 28, 2015

    work page 2015

  2. [2]

    F., Blum, A., Fine, S., and Mansour, Y

    Balcan, M. F., Blum, A., Fine, S., and Mansour, Y. Distributed learning, communication complexity and privacy. In Conference on Learning Theory, pp.\ 26--1. JMLR Workshop and Conference Proceedings, 2012

    work page 2012

  3. [3]

    Cai, Y., Wu, J., Mei, S., Lindsey, M., and Bartlett, P. L. Large stepsize gradient descent for non-homogeneous two-layer networks: Margin improvement and fast optimization. arXiv preprint arXiv:2406.08654, 2024

    arXiv 2024

  4. [4]

    Z., and Talwalkar, A

    Cohen, J., Kaur, S., Li, Y., Kolter, J. Z., and Talwalkar, A. Gradient descent on neural networks typically occurs at the edge of stability. In International Conference on Learning Representations, 2021

    2021

  5. [5]

    Local steps speed up local gd for heterogeneous distributed logistic regression

    Crawshaw, M., Woodworth, B., and Liu, M. Local steps speed up local gd for heterogeneous distributed logistic regression. International Conference on Learning Representations, 2025

    work page 2025

  6. [6]

    Damian, A., Nichani, E., and Lee, J. D. Self-stabilization: The implicit bias of gradient descent at the edge of stability. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=nhKHA59gXz

    2023

  7. [7]

    Optimal distributed online prediction using mini-batches

    Dekel, O., Gilad-Bachrach, R., Shamir, O., and Xiao, L. Optimal distributed online prediction using mini-batches. Journal of Machine Learning Research, 13 0 (1), 2012

    2012

  8. [8]

    R., Yuan, H., and Ma, T

    Glasgow, M. R., Yuan, H., and Ma, T. Sharp bounds for federated averaging (local sgd) and continuous perspective. In International Conference on Artificial Intelligence and Statistics, pp.\ 9050--9090. PMLR, 2022

    work page 2022

Show all 39 references
  1. [9]

    Characterizing implicit bias in terms of optimization geometry

    Gunasekar, S., Lee, J., Soudry, D., and Srebro, N. Characterizing implicit bias in terms of optimization geometry. In International Conference on Machine Learning, pp.\ 1832--1841. PMLR, 2018

    2018

  2. [10]

    and Mahdavi, M

    Haddadpour, F. and Mahdavi, M. On the convergence of local descent methods in federated learning. arXiv preprint arXiv:1910.14425, 2019

    1910 arXiv

  3. [11]

    The break-even point on optimization trajectories of deep neural networks

    Jastrzebski, S., Szymczak, M., Fort, S., Arpit, D., Tabor, J., Cho*, K., and Geras*, K. The break-even point on optimization trajectories of deep neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=r1g87C4KwB

    2020

  4. [12]

    and Telgarsky, M

    Ji, Z. and Telgarsky, M. The implicit bias of gradient descent on nonseparable data. In Beygelzimer, A. and Hsu, D. (eds.), Proceedings of the Thirty-Second Conference on Learning Theory, volume 99 of Proceedings of Machine Learning Research, pp.\ 1772--1798. PMLR, 25--28 Jun ...

    2019

  5. [13]

    Fast margin maximization via dual acceleration

    Ji, Z., Srebro, N., and Telgarsky, M. Fast margin maximization via dual acceleration. In International Conference on Machine Learning, pp.\ 4860--4869. PMLR, 2021

    2021

  6. [14]

    B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A

    Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. Advances and open problems in federated learning. Foundations and trends in machine learning , 14 0 (1--2): 0 1--210, 2021

    2021

  7. [15]

    P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A

    Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A. T. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning, pp.\ 5132--5143. PMLR, 2020

    2020

  8. [16]

    Tighter theory for local sgd on identical and heterogeneous data

    Khaled, A., Mishchenko, K., and Richt \'a rik, P. Tighter theory for local sgd on identical and heterogeneous data. In International Conference on Artificial Intelligence and Statistics, pp.\ 4519--4529. PMLR, 2020

    2020

  9. [17]

    A unified theory of decentralized sgd with changing topology and local updates

    Koloskova, A., Loizou, N., Boreiri, S., Jaggi, M., and Stich, S. A unified theory of decentralized sgd with changing topology and local updates. In International Conference on Machine Learning, pp.\ 5381--5393. PMLR, 2020

    2020

  10. [18]

    Efficient large-scale distributed training of conditional maximum entropy models

    Mcdonald, R., Mohri, M., Silberman, N., Walker, D., and Mann, G. Efficient large-scale distributed training of conditional maximum entropy models. Advances in neural information processing systems, 22, 2009

    2009

  11. [19]

    Distributed training strategies for the structured perceptron

    McDonald, R., Hall, K., and Mann, G. Distributed training strategies for the structured perceptron. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp.\ 456--464. Association for Computa...

    2010

  12. [20]

    McMahan, B., Moore, E., Ramage, D., Hampson, S., and Arcas, B. A. y. Communication-Efficient Learning of Deep Networks from Decentralized Data . In Singh, A. and Zhu, J. (eds.), Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume ...

    2017

  13. [21]

    S., Srebro, N., and Soudry, D

    Nacson, M. S., Srebro, N., and Soudry, D. Stochastic gradient descent on separable data: Exact convergence with a fixed learning rate. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.\ 3051--3059. PMLR, 2019

    2019

  14. [22]

    K., Glasgow, M., Wang, L., Joshi, N., and Srebro, N

    Patel, K. K., Glasgow, M., Wang, L., Joshi, N., and Srebro, N. On the still unreasonable effectiveness of federated averaging for heterogeneous distributed learning. In Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities, 2023. UR...

    2023

  15. [23]

    K., Glasgow, M., Zindari, A., Wang, L., Stich, S

    Patel, K. K., Glasgow, M., Zindari, A., Wang, L., Stich, S. U., Cheng, Z., Joshi, N., and Srebro, N. The limits and potentials of local sgd for distributed heterogeneous learning with intermittent communication. In Agrawal, S. and Roth, A. (eds.), Proceedings of Thirty Seventh...

    2024

  16. [24]

    J., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Kone c n \'y , J., Kumar, S., and McMahan, H

    Reddi, S. J., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Kone c n \'y , J., Kumar, S., and McMahan, H. B. Adaptive federated optimization. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=LkFG3lB13U5

    2021

  17. [25]

    and Srebro, N

    Shamir, O. and Srebro, N. Distributed stochastic optimization and learning. In 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp.\ 850--857. IEEE, 2014

    2014

  18. [26]

    S., Gunasekar, S., and Srebro, N

    Soudry, D., Hoffer, E., Nacson, M. S., Gunasekar, S., and Srebro, N. The implicit bias of gradient descent on separable data. Journal of Machine Learning Research, 19 0 (70): 0 1--57, 2018. URL http://jmlr.org/papers/v19/18-188.html

    2018

  19. [27]

    Stich, S. U. Local sgd converges fast and communicates little. In ICLR 2019-International Conference on Learning Representations, 2019

    2019

  20. [28]

    Communication-efficient distributed deep learning: A comprehensive survey

    Tang, Z., Shi, S., Wang, W., Li, B., and Chu, X. Communication-efficient distributed deep learning: A comprehensive survey. arXiv preprint arXiv:2003.06307, 2020

    2003 arXiv

  21. [29]

    Verbraeken, J., Wolting, M., Katzy, J., Kloppenburg, J., Verbelen, T., and Rellermeyer, J. S. A survey on distributed machine learning. Acm computing surveys (csur), 53 0 (2): 0 1--33, 2020

    2020

  22. [30]

    B., Al-Shedivat, M., Andrew, G., Avestimehr, S., Daly, K., Data, D., et al

    Wang, J., Charles, Z., Xu, Z., Joshi, G., McMahan, H. B., Al-Shedivat, M., Andrew, G., Avestimehr, S., Daly, K., Data, D., et al. A field guide to federated optimization. arXiv preprint arXiv:2107.06917, 2021

    2021 arXiv

  23. [31]

    On the unreasonable effectiveness of federated averaging with heterogeneous data

    Wang, J., Das, R., Joshi, G., Kale, S., Xu, Z., and Zhang, T. On the unreasonable effectiveness of federated averaging with heterogeneous data. arXiv preprint arXiv:2206.04723, 2022

    2022 arXiv

  24. [32]

    K., Stich, S., Dai, Z., Bullins, B., Mcmahan, B., Shamir, O., and Srebro, N

    Woodworth, B., Patel, K. K., Stich, S., Dai, Z., Bullins, B., Mcmahan, B., Shamir, O., and Srebro, N. Is local sgd better than minibatch sgd? In International Conference on Machine Learning, pp.\ 10334--10343. PMLR, 2020 a

    2020

  25. [33]

    E., Patel, K

    Woodworth, B. E., Patel, K. K., and Srebro, N. Minibatch vs local sgd for heterogeneous distributed learning. Advances in Neural Information Processing Systems, 33: 0 6281--6292, 2020 b

    2020

  26. [34]

    L., Telgarsky, M., and Yu, B

    Wu, J., Bartlett, P. L., Telgarsky, M., and Yu, B. Large stepsize gradient descent for logistic loss: Non-monotonicity of the loss improves optimization efficiency. In Agrawal, S. and Roth, A. (eds.), Proceedings of Thirty Seventh Conference on Learning Theory, volume 247 of P...

    2024

  27. [35]

    Wu, J., Braverman, V., and Lee, J. D. Implicit bias of gradient descent for logistic regression at the edge of stability. Advances in Neural Information Processing Systems, 36, 2024 b

    2024

  28. [36]

    Federated learning of gboard language models with differential privacy

    Xu, Z., Zhang, Y., Andrew, G., Choquette, C., Kairouz, P., Mcmahan, B., Rosenstock, J., and Zhang, Y. Federated learning of gboard language models with differential privacy. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: I...

    2023

  29. [37]

    I., and Wainwright, M

    Zhang, Y., Duchi, J., Jordan, M. I., and Wainwright, M. J. Information-theoretic lower bounds for distributed statistical estimation with communication constraints. Advances in Neural Information Processing Systems, 26, 2013

    2013

  30. [38]

    Zinkevich, M., Weimer, M., Li, L., and Smola, A. J. Parallelized stochastic gradient descent. In Advances in neural information processing systems, pp.\ 2595--2603, 2010

    2010

  31. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.