REVIEW 3 major objections 5 minor 1 cited by
Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper proves convergence of local-step federated methods under (L0,L1)-smoothness without data-homogeneity, bounded-variance, or cosine-relatedness assumptions.
desk verdict Solid theory, shaky bridge to experiments: the paper proves first convergence rates for local-step FL with reshuffling under (L0,L1)-smoothness, but the implemented pseudogradient stepsize is not covered by the proofs. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the generalized-smoothness coefficient $\hat a_p = L_0 + L_1\|\nabla f(\hat x_{t_p})\|$ (with local variants $a_p$ and $\tilde a_t$), paired with a server stepsize constrained to the interval $\gamma_p \in [\zeta/\hat a_p, 1/(4\hat a_p)]$ and a pseudogradient $g_p = \frac{1}{\alpha_p(v-t_p)}(\hat x_{t_p} - \frac1M\sum_m x^m_{t_{p+1}-1})$ formed from the local iterates. This coupling acts as smoothed clipping: when the gradient is large, the server step is automatically short, while the small client stepsize $\alpha_p \le 1/(2H a_p)$ bounds client drift. The proofs use a descent lemma for $(L_0,L_1)$-smooth functions and a deviation bound showing the average squared distance of local iterates from the synchronization point is at most $O((H-1)^2 a_p^3 \alpha_p^2(\delta_p+\Delta^\star))$; the argument then splits each epoch according to whether $\|\nabla f\|$ is below or above $L_0/L_1$, which is what converts the stationarity measure into the stated rates. For symmetric smoothness, the proof additionally controls the exponential factor in the descent inequality by an induction that bounds all local gradient norms along the epoch.
What would settle it
On a two-client $(L_0,L_1)$-smooth problem where one client's local gradients are far larger than the other's, compute $a_p = L_0 + L_1 \max_m \|\nabla f_m(\hat x_{t_p})\|$ along the trajectory and check whether the tuned inner stepsize satisfies $\alpha_p \le 1/(2H a_p)$ at every epoch; because $a_p$ is not observable to the user, a run that converges while violating this condition would show the theorem's sufficient conditions are not the operative mechanism, while a run that diverges under the condition would refute the claimed guarantee for the implemented method.
Extended reading notes
Core claim
On the paper's own terms, the central claim is that local-step federated methods can converge under $(L_0,L_1)$-smoothness without extra restrictive assumptions, as long as the server takes a clipped, larger step along the averaged pseudogradient while clients take small steps. Concretely, Theorem 1 bounds $\min_{0\le p\le P-1} \frac{\zeta}{8}\min\{\|\nabla f(\hat x_{t_p})\|^2/L_0, \|\nabla f(\hat x_{t_p})\|/L_1\}$ by a term of order $O(L_0\delta_0/(\zeta \varepsilon))$ for Clip-LocalGDJ, and Corollary 1 converts this into an iteration complexity matching standard LocalGD when $L_1=0$. Theorems 3 and 4 give the analogous guarantees for CLERR (local steps with random reshuffling) and Clipped RR-CLI (local steps, reshuffling, and partial participation); for each method the paper also proves a linear rate under the Polyak–Łojasiewicz condition, recovering the standard $O((L_0/\mu)\ln(2\delta_0/\varepsilon))$ rate when $L_1=0$. These are the first results of this kind for partial participation under generalized smoothness, according to the paper's literature survey.
Load-bearing premise
For the proved rates to apply, stepsizes must be set using the exact global gradient and the largest local gradient norm among all clients at every synchronization point, values a federated user would not have; the implemented algorithms instead use a pseudogradient heuristic that the theorems do not cover.
Editorial extensions
If this is right
- For non-convex $(L_0,L_1)$-smooth distributed problems, Clip-LocalGDJ requires $O(L_0\delta_0/(\zeta\varepsilon))$ synchronization epochs to reach the generalized stationarity measure $\min\{\|\nabla f\|^2/L_0, \|\nabla f\|/L_1\}\le\varepsilon$, matching the standard LocalGD rate when $L_1=0$.
- CLERR gives a random-reshuffling convergence rate for generalized smooth objectives that does not assume bounded variance or cosine relatedness, under the paper's claims.
- Clipped RR-CLI extends the guarantee to partial participation of clients, with a rate that again becomes $O(L_0\delta_0/(\zeta\varepsilon))$ when the inner stepsizes are small.
- Under the Polyak–Łojasiewicz condition, all three methods converge linearly with complexity $O(L_0/\mu \ln(2\delta_0/\varepsilon))$ in the $L_1=0$ limit, and the theorem shows a large-gradient phase of at most $64\delta_0 L_1^2/(\mu\zeta)$ epochs before the linear decay begins.
- The results indicate that clipping once per epoch, rather than at every local step, is sufficient in theory for generalized smoothness, and the paper's synthetic experiments support that design choice.
Reading between the lines
- If $(L_0,L_1)$-smoothness is the right model for deep-network losses, the paper's design principle suggests that federated systems should spend their clipping budget on the aggregated server step rather than on every local gradient; the theory proves this for the once-per-epoch variants, but a head-to-head comparison with per-step clipping under the same assumptions would test whether the design p
- Because the theorems' stepsize conditions involve quantities unavailable to the user (the full gradient and the largest local gradient norm at each synchronization point), an adaptive scheme that estimates these quantities online, or a parameter-free version, is the natural next step before the guarantees can be applied directly to the implemented algorithms.
- The same two-case 'small gradient / large gradient' proof technique could plausibly extend to local methods with communication compression or incomplete local epochs, which the paper itself lists as future work.
- The experimental heuristic of replacing $\nabla f(x_t)$ by the pseudogradient $g_t$ in the server stepsize is covered only approximately by the appendix's perturbation argument; a full proof for that practical variant would close the gap between the implemented and the analyzed methods.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes three federated optimization algorithms for non-convex objectives satisfying the generalized (L0,L1)-smoothness condition: Clip-LocalGDJ (Algorithm 1), CLERR (Algorithm 2), and Clipped RR-CLI (Algorithm 3). The methods combine local steps, random reshuffling, partial client participation, and server-side gradient clipping. The main theorems (Theorems 1-4) give convergence rates for the non-convex setting, and additional theorems in the appendices cover the Polyak-Lojasiewicz case. The paper claims these are the first such results without restrictive assumptions such as data homogeneity, bounded variance, or cosine relatedness, and the rates recover standard smooth rates when L1=0. Experiments on a synthetic shifted-fourth-order problem, ResNet-18 on CIFAR-10, and logistic regression datasets are reported in support of the theory.
Significance. If the theorems are correct as stated with their full hypotheses, the paper fills a genuine gap: it extends federated learning methods with local steps, random reshuffling, and partial participation to (L0,L1)-smooth non-convex problems without imposing the homogeneity or variance assumptions used in prior work. The recovery of known L-smooth rates when L1=0 is a useful consistency check and is not a circular derivation from the target result. The analysis follows established proof templates and the appendix contains detailed derivations. However, the practical impact is currently limited by two gaps: the stepsize conditions in the theorems require oracle quantities that are not available to the server, and the experiments run a version of the algorithms whose analysis is explicitly omitted. These issues are fixable but need to be addressed before the claims can be accepted as stated.
major comments (3)
- [Section 4 and Appendix F] The convergence theorems analyze Algorithms 1-3 with a global stepsize of the form 1/(c0+c1||grad f(x_t)||), where grad f is the full gradient. The experiments, however, replace this by the pseudogradient g_p (Algorithm 1) or g_t (Algorithms 2 and 3), as Appendix F explicitly states. Appendix F further says that the theoretical analysis of this practical version was deliberately not included. Appendix E only shows that ||g_p|| is close to ||grad f(hat x_{t_p})|| for small local stepsizes; it does not re-derive the descent lemmas used in Theorems 1-4. In particular, for CLERR and Clipped RR-CLI the stepsize becomes a random variable that is correlated with the random permutation through g_t, so the conditional-expectation step E[f(x_{t+1})|x_t] with a deterministic gamma_t is no longer justified. As written, the paper proves convergence for an algorithm that is not the one run in the experiments, and the experimental support for the theory is therefore incomplete.
- [Section 4, Theorems 1-4] The stepsize conditions in the theorems require quantities that are not available to the server in a federated setting. For example, Theorem 1 requires alpha_p to be bounded in terms of a_p = L0 + L1 max_m ||grad f_m(hat x_{t_p})||, and gamma_p to lie in an interval defined by hat a_p = L0 + L1 ||grad f(hat x_{t_p})||. The server would need to know L0, L1, and the maximum local gradient norm over all clients at every synchronization point, as well as delta0, Delta*, H, and P. The pseudocode only says 'choose small local stepsizes' without giving a procedure for computing or estimating these quantities. If these quantities are not available, the theorems do not apply to a practical implementation. The paper should state explicitly that the stepsizes are theoretical, or provide a procedure for setting them from computable quantities.
- [Section 4, Corollary 4] The statement of Corollary 4 says that if T >= 72 delta0 / (zeta epsilon) and gamma_t, eta_t are small enough, then the method reaches epsilon accuracy. The proof in Appendix D.1, however, requires T >= max{72 delta0 / (zeta epsilon), 12 Delta* / (zeta epsilon), 6 Delta* / (zeta epsilon)} and precise upper bounds on eta_t and gamma_t that depend on these quantities. As stated, the corollary omits the dependence on Delta* and Delta*, so if these constants are large the claimed iteration complexity is an underestimate. This is a load-bearing issue for the partial-participation result and should be corrected.
minor comments (5)
- [Section 4, Theorem 1] The main-text statement of Theorem 1 does not include the upper bound on alpha_p that is imposed in the appendix (alpha_p <= min{1/(2H a_p), 1/(c a_p) sqrt(hat a_p / a_p)} with c >= sqrt(P)). The main text says only 'choose small local stepsizes,' which is too vague for the theorem to be self-contained; please include the condition or provide an explicit reference to Appendix B.1.
- [Section 5.1.1 and Appendix F] The experiments are run over 3 or 10 random seeds, but Figures 2, 6, 7, and 8 show only mean curves without error bars or shaded regions. Adding variance information would make the comparisons between methods more convincing.
- [Section 5.3 and Appendix F.3] The abbreviation 'CRR-CLI' is used in Figure 4 and in Appendix F.3, while the method is named 'Clipped RR-CLI' in the main text. Please use a single consistent name for Algorithm 3.
- [Appendix F, first paragraph] The sentence 'Although, we decided not to include it to keep the presentation more concise' contains a typo; 'Although' should be 'However'.
- [Section 1.1] The contribution bullet says the results are 'tight,' but the paper only shows recovery of known rates in the special case L1=0; no lower bounds are established for the (L0,L1)-smooth setting. Consider softening the claim to avoid suggesting optimality in the broader class.
Circularity Check
No circularity: the convergence rates are derived from the stated (L0,L1)-smoothness assumptions via standard descent lemmas, and the consistency checks with L-smooth rates are special-case verifications rather than inputs.
full rationale
The paper's central derivation chain is not circular. Theorems 1-4 all follow the same structure: apply the generalized-smooth descent inequality (Lemma 1, cited to Zhang et al. 2020b and Gorbunov et al. 2024) to bound f(x_{t+1}), control the client-drift or reshuffling-bias terms through Lemmas 4, 7, and the V_t bound in Appendix D, and then invoke the standard recurrence lemma of Mishchenko et al. (2020, Lemma 6). The final rates are consequences of explicit choices such as P >= 32*delta_0/(zeta*epsilon) and small enough inner stepsizes; none of the quantities defining the target bound are inserted into the assumptions or fitted from the data. The claim that the rates recover O(L0*delta_0/epsilon) for LocalGD, RR, and clipped GD when L1 = 0 is a consistency check, not an assumption: it is obtained by specializing the same bound, not by assuming the final rate. The self-citations are auxiliary rather than load-bearing: Lemma 1's second statement is a short proof from the descent inequality, the RR-CLI base algorithm is cited as prior work being extended, and the RR variance lemma from Malinovsky et al. (2022) is a standard sampling-without-replacement fact. None of these citations assume the paper's conclusion. The genuine limitations are non-circular correctness/implementability gaps: the stepsize conditions in Theorems 1-4 require oracle quantities such as hat_a_p = L0 + L1||grad f(hat_x_{t_p})|| and a_p = L0 + L1 max_m ||grad f_m(hat_x_{t_p})||, and Appendix F explicitly states that the theoretical analysis of the 'practical' pseudogradient stepsize used in the experiments was deliberately omitted, while Appendix E provides only a norm-closeness heuristic. These are gaps between the proved algorithm and the implemented one, not circular reasoning.
Assumptions & free parameters
free parameters (1)
- zeta
assumptions (6)
- domain assumption Assumption 1: f, f_m, f_mj are bounded below
- domain assumption Assumption 2 (asymmetric (L0,L1)-smoothness) applied to f, each f_m, and each f_mj
- domain assumption Assumption 3 (symmetric (L0,L1)-smoothness) used for some theorems
- domain assumption Assumption 4 (Polyak-Lojasiewicz condition) for linear-rate results
- standard math Lemma 1 and Lemma 2 from earlier papers (Zhang et al. 2020b; Gorbunov et al. 2024; Chen et al. 2023)
- standard math Lemma 6 from Mishchenko et al. (2020) for unrolling recurrences
Cite this review
Pith. "Pith review of Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization." pith.science (2026). https://pith.science/paper/LWBP2ZUT
@misc{pith2026241202781,
author = {Pith},
title = {Pith review of: Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/LWBP2ZUT}},
note = {Machine review of arXiv:2412.02781}
}
abstract
Non-convex Machine Learning problems typically do not adhere to the standard smoothness assumption. Based on empirical findings, Zhang et al. (2020b) proposed a more realistic generalized $(L_0, L_1)$-smoothness assumption, though it remains largely unexplored. Many existing algorithms designed for standard smooth problems need to be revised. However, in the context of Federated Learning, only a few works address this problem but rely on additional limiting assumptions. In this paper, we address this gap in the literature: we propose and analyze new methods with local steps, partial participation of clients, and Random Reshuffling without extra restrictive assumptions beyond generalized smoothness. The proposed methods are based on the proper interplay between clients' and server's stepsizes and gradient clipping. Furthermore, we perform the first analysis of these methods under the Polyak-{\L} ojasiewicz condition. Our theory is consistent with the known results for standard smooth problems, and our experimental results support the theoretical insights.
Figures
Figures from the paper (7 more)
Forward citations
Cited by 1 Pith paper
-
Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization
A thesis proving communication-acceleration guarantees for local-step, compressed, Byzantine-robust, and low-rank federated optimization methods, assembled from the author's own published papers.
Reference graph
Works this paper leans on
-
[1]
For such p, we have L0 + L1 ∇f ˆxtp = ˆap ≤ 2L1 ∇f ˆxtp
Let ˜P be the number of steps p, so that ∇f ˆxtp ≥ L0 L1 . For such p, we have L0 + L1 ∇f ˆxtp = ˆap ≤ 2L1 ∇f ˆxtp . Therefore, we get µζ f (ˆxtp ) − f ⋆ 4L1 ∇f ˆxtp ≤ f ˆxtp − f ˆxtp+1 + 3(H − 1)2a3 pα2 p f (ˆxtp ) − f ⋆ + ∆⋆ 2ˆap . Notice that the relation ˆap ≤ 2L1 ∇f ˆxtp and Lemma 1 together imply ∇f ˆxtp 4L1 ≤ ∇f ˆxtp 2 2ˆap ≤ f ˆxtp − f ⋆. Hence, w...
-
[2]
For such p, we have L0 + L1 ∇f ˆxtp = ˆap ≤ 2L0
Suppose now that ∇f ˆxtp ≤ L0 L1 . For such p, we have L0 + L1 ∇f ˆxtp = ˆap ≤ 2L0. Hence, µζ f (ˆxtp ) − f ⋆ 4L0 ≤ f ˆxtp − f ˆxtp+1 + 3(H − 1)2a3 pα2 p f (ˆxtp ) − f ⋆ + ∆⋆ 2ˆap . Subtracting f ⋆ on both sides and introducing δp def = f ˆxtp − f ⋆, we obtain δp+1 ≤ δpρ + 3(H − 1)2a3 pα2 p f (ˆxtp ) − f ⋆ + ∆⋆ 2ˆap , where ρ def = 1 − µζ 4L0 . 25 Publish...
work page 2025
-
[7]
16 Published as a conference paper at ICLR 2025 Quoc Tran-Dinh, Nhan H
ISBN 978-1-57735-876-3. 16 Published as a conference paper at ICLR 2025 Quoc Tran-Dinh, Nhan H. Pham, D. Phan, and Lam M. Nguyen. Feddr - randomized douglas- rachford splitting algorithms for nonconvex federated composite optimization. In Neural Information Processing Systems , 2021. URL https://api.semanticscholar.org/ CorpusID:235376727. Bohan Wang, Yus...
arXiv 2025
-
[8]
doi: 10.1109/TSP.2018.2878551. Yang You, Jing Li, Sashank J. Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. Large batch optimization for deep learning: Training bert in 76 minutes. arXiv: Learning, 2019. Bohang Zhang, Jikai Jin, Cong Fang, and Liwei Wang. Improved analysis of clipping...
arXiv 2018
-
[10]
Despite the fact that both Nastya and CLERR choose bigger outer stepsizes compared to inner stepsize, jumping does not have any impact on this problem. CLERR clips outer gradients at the level of 10, so this also does not help method to converge to a better area. Moreover, we provide results of heuristically modified Algorithm 2, where we fix clipping lev...
work page 2022
-
[11]
For such p, we have L0 + L1 ∇f ˆxtp = ˆap ≤ 2L1 ∇f ˆxtp
Let ˜P be the number of steps p, so that ∇f ˆxtp ≥ L0 L1 . For such p, we have L0 + L1 ∇f ˆxtp = ˆap ≤ 2L1 ∇f ˆxtp . Therefore, we get µζ f (ˆxtp ) − f ⋆ 4L1 ∇f ˆxtp ≤ f ˆxtp − f ˆxtp+1 + 7(H − 1)2a3 pα2 p f (ˆxtp ) − f ⋆ + ∆⋆ ˆap . Notice that the relation ˆap ≤ 2L1 ∇f ˆxtp and Lemma 1 together imply ∇f ˆxtp 4L1 ≤ ∇f ˆxtp 2 2ˆap ≤ f ˆxtp − f ⋆. Hence, we...
-
[12]
For such p, we have L0 + L1 ∇f ˆxtp = ˆap ≤ 2L0
Suppose now that ∇f ˆxtp ≤ L0 L1 . For such p, we have L0 + L1 ∇f ˆxtp = ˆap ≤ 2L0. Hence, µζ f (ˆxtp ) − f ⋆ 4L0 ≤ f ˆxtp − f ˆxtp+1 + 7(H − 1)2a3 pα2 p f (ˆxtp ) − f ⋆ + ∆⋆ ˆap . Subtracting f ⋆ on both sides and introducing δp def = f ˆxtp − f ⋆, we obtain δp+1 ≤ δpρ + 7(H − 1)2a3 pα2 p f (ˆxtp ) − f ⋆ + ∆⋆ ˆap , where ρ def = 1 − µζ 4L0 . Let αp def =...
work page 2025
-
[13]
For such t, we haveL0+L1 ∥∇f (xt)∥ = ˆat ≤ 2L1 ∥∇f (xt)∥
Let ˜T be the number of stepst, so that ∥∇f (ˆxt)∥ ≥L0 L1 . For such t, we haveL0+L1 ∥∇f (xt)∥ = ˆat ≤ 2L1 ∥∇f (xt)∥ . Therefore, we get µζ (f (xt) − f ⋆) 4L1 ∥∇f (xt)∥ ≤ f (xt) − f (xt+1) + 3α2 t ˜at 3 8ˆat δt((N − 1)(2N − 1) + 2(N + 1)) + 2(N + 1)∆ ⋆ . Notice that the relation ˆat ≤ 2L1 ∥∇f (xt)∥ and Lemma 1 together imply ∥∇f (xt)∥ 4L1 ≤ ∥∇f (xt)∥2 2ˆa...
Show all 20 references
-
[14]
For such t, we have L0 +L1 ∥∇f (xt)∥ = ˆap ≤ 2L0
Suppose now that ∥∇f (xt)∥ ≤L0 L1 . For such t, we have L0 +L1 ∥∇f (xt)∥ = ˆap ≤ 2L0. Hence, µζ (f (xt) − f ⋆) 4L0 ≤ f (xt) − f (xt+1) + 3α2 t ˜at 3 8ˆat δt((N − 1)(2N − 1) + 2(N + 1)) + 2(N + 1)∆ ⋆ . Subtracting f ⋆ on both sides and introducing δt def = f (xt) − f ⋆, we obta...
2025
-
[15]
For such t, we haveL0+L1 ∥∇f (xt)∥ = ˆat ≤ 2L1 ∥∇f (xt)∥
Let ˜T be the number of stepst, so that ∥∇f (ˆxt)∥ ≥L0 L1 . For such t, we haveL0+L1 ∥∇f (xt)∥ = ˆat ≤ 2L1 ∥∇f (xt)∥ . Therefore, we get µζ (f (xt) − f ⋆) 4L1 ∥∇f (xt)∥ ≤ f (xt) − f (xt+1) + α2 t ˜a3 t 2ˆat δt((N − 1)(2N − 1) + 2(N + 1)) + 2(N + 1)∆ ⋆ . Notice that the relatio...
2025
-
[16]
For such t, we have L0 +L1 ∥∇f (xt)∥ = ˆap ≤ 2L0
Suppose now that ∥∇f (xt)∥ ≤L0 L1 . For such t, we have L0 +L1 ∥∇f (xt)∥ = ˆap ≤ 2L0. Hence, µζ (f (xt) − f ⋆) 4L0 ≤ f (xt) − f (xt+1) + α2 t ˜a3 t 2ˆat δt((N − 1)(2N − 1) + 2(N + 1)) + 2(N + 1)∆ ⋆ . Subtracting f ⋆ on both sides and introducing δt def = f (xt) − f ⋆, we obtai...
2025
-
[17]
For such t, we haveL0+L1 ∥∇f (xt)∥ = ˆat ≤ 2L1 ∥∇f (xt)∥
Let ˜T be the number of stepst, so that ∥∇f (ˆxt)∥ ≥L0 L1 . For such t, we haveL0+L1 ∥∇f (xt)∥ = ˆat ≤ 2L1 ∥∇f (xt)∥ . Therefore, we get µζ (f (xt) − f ⋆) 4L1 ∥∇f (xt)∥ ≤ f (xt) − f (xt+1) + 2ˆat˜a2 t + ˆa3 t 4ˆa2 t (f (xt) − f ⋆) η2 t at + η2 t R2ˆat + γ2 t N ˜at + η2 t Rat +...
-
[18]
practical
Suppose now that ∥∇f (xt)∥ ≤L0 L1 . For such t, we have L0 +L1 ∥∇f (xt)∥ = ˆap ≤ 2L0. Hence, µζ (f (xt) − f ⋆) 4L0 ≤ f (xt) − f (xt+1) + 2ˆat˜a2 t + ˆa3 t 4ˆa2 t (f (xt) − f ⋆) η2 t at + η2 t R2ˆat + γ2 t N ˜at + η2 t Rat + 2ˆat˜a2 t + ˆa3 t 4ˆa2 t η2 t at∆⋆ + γ2 t N ˜at∆ ⋆ + ...
2016
-
[20]
And CE-FedAvg-PP has server stepsize equal to 10
Speaking of outer steps, Algorithm 3 chooses global stepsize equal to 5 · 10−7 with clipping level 1016. And CE-FedAvg-PP has server stepsize equal to 10. The grids of hyperparameters are provided in Table 5. 64 Published as a conference paper at ICLR 2025 Method Cl. Stepsize ...
2011
-
[2012]
ISBN 9781450312851
Omnipress. ISBN 9781450312851. Benjamin Recht and Christopher R ´e. Parallel stochastic gradient algorithms for large-scale matrix completion. Mathematical Programming Computation , 5:201 – 226, 2013. URL https:// api.semanticscholar.org/CorpusID:17109415. Herbert Robbins and ...
2013
-
[2019]
12 Published as a conference paper at ICLR 2025 Priya Goyal, Piotr Doll ´ar, Ross B
URL https://proceedings.mlr.press/v97/qian19b.html. 12 Published as a conference paper at ICLR 2025 Priya Goyal, Piotr Doll ´ar, Ross B. Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Tra...
2025 arXiv
-
[2020]
Xufeng Cai, Cheuk Yin Lin, and Jelena Diakonikolas
URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf. Xufeng Cai, Cheuk Yin Lin, and Jelena Diakonikolas. Empirical risk minimization with shuffled SGD: A primal-dual perspective and improved bounds. CoRR, abs/2306.12498, 2...
-
[2021]
Wenlin Chen, Samuel Horv ´ath, and Peter Richt ´arik
URL https://proceedings.neurips.cc/paper_files/paper/2021/ file/ab9ebd57177b5106ad7879f0896685d4-Paper.pdf. Wenlin Chen, Samuel Horv ´ath, and Peter Richt ´arik. Optimal client sampling for federated learn- ing. Trans. Mach. Learn. Res., 2022, 2020. URL https://api.semanticsch...
2021
-
[2022]
Eduard Gorbunov, Konstantin P
URL https://proceedings.mlr.press/v151/glasgow22a.html. Eduard Gorbunov, Konstantin P. Burlachenko, Zhize Li, and Peter Richtarik. Marina: Faster non- convex distributed learning with compression. In Marina Meila and Tong Zhang (eds.), Proceed- ings of the 38th International C...
2024 arXiv
-
[2023]
Dan Alistarh, Torsten Hoefler, Mikael Johansson, Sarit Khirirat, Nikola Konstantinov, and C ´edric Renggli
URL https://api.semanticscholar.org/CorpusID:263605847. Dan Alistarh, Torsten Hoefler, Mikael Johansson, Sarit Khirirat, Nikola Konstantinov, and C ´edric Renggli. The convergence of sparsified gradient methods. In Proceedings of the 32nd Inter- national Conference on Neural I...
2018 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.