REVIEW 2 major objections 5 minor 2 cited by
Revisiting Randomized Smoothing: Nonsmooth Nonconvex Optimization Beyond Global Lipschitz Continuity
T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Randomized smoothing provably works when the objective has no global Lipschitz constant.
desk verdict A real extension of randomized smoothing rates to a locally Lipschitz class, but the main theorems silently assume exact knowledge of α,β,σ,ℓ, which a zeroth-order algorithm cannot have. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the $(\alpha,\beta)$ subgradient growth condition and its propagation through spherical smoothing. It converts an unbounded local Lipschitz constant into a point-dependent smoothness constant of $f_\delta$, which then feeds directly into the step-size formulas (4.1)--(4.3). The variance-reduced loop of Algorithm 3 uses periodic large batches to reduce gradient-estimator variance, which is what lowers the $\epsilon$-exponent from $-4$ to $-3$.
What would settle it
Take $f(x)=\exp(|x|)-x^3$ in $d=1$ with $\alpha(x)=\exp(|x|)+3x^2$ and run RS-NVRGF with finite-difference estimates of $\alpha$ and $\beta$ replacing the exact values in the step-size formulas: if no such implementable variant reaches a $(10^{-4},\epsilon)$-Goldstein point with $O(\epsilon^{-3})$ oracle calls, the hidden-exactness premise is doing the work; conversely, a concrete trajectory where the exact-schedule theorem fails to hold would refute the claimed bound.
Extended reading notes
Core claim
The central claim is that the global Lipschitz assumption can be replaced by a strictly weaker local growth condition without losing the convergence rates of randomized smoothing. A function $f$ satisfies the $(\alpha,\beta)$ subgradient growth condition if every Clarke subgradient at $x$ has norm at most $\alpha(x)$, and $|\alpha(x)-\alpha(y)| \le \beta(x,\|x-y\|)$ with $\beta$ non-decreasing in the radius. Spherical smoothing $f_\delta(x)=\mathbb{E}_{w\sim U(B_1(0))} f(x+\delta w)$ then gives a continuously differentiable function whose gradient lies in the Goldstein $\delta$-subdifferential, and the paper proves a local descent inequality with smoothness constant $\ell(x,r)=c\sqrt{d}(2\alpha(x)+\beta(x,\delta)+\beta(x,r+\delta))/(2\delta)$. Under level-boundedness of $f_\delta$ and with step sizes that adapt to $\alpha(x_t)$, $\beta(x_t,\cdot)$, $\sigma(x_t)$, and $\ell(x_t,\cdot)$, the paper proves high-probability convergence to a $(\delta,\epsilon)$-Goldstein stationary point for three algorithms; the variance-reduced RS-NVRGF attains the $\epsilon^{-3}$ sample complexity previously tied to global Lipschitz assumptions.
Load-bearing premise
The load-bearing premise is that the algorithm can be handed exact values, at each step, of the local constants that describe how steep the function is and how smooth its smoothed version is near the current point; the theorems assume these values are available to set step sizes, and the paper only notes at the end that a parameter-free method is left for future work.
Editorial extensions
If this is right
- If the central claim is correct, any level-bounded objective satisfying the $(\alpha,\beta)$ growth condition can be optimized by zeroth-order randomized smoothing at essentially the same $\epsilon$-rate as globally Lipschitz objectives.
- The three rates are $\tilde{\mathcal{O}}(d^{5/2}\delta^{-1}\epsilon^{-4})$ for RS-GF, $\tilde{\mathcal{O}}(d^{3/2}\delta^{-1}\epsilon^{-4})$ for RS-NGF, and $\tilde{\mathcal{O}}(d^{3/2}\delta^{-1}\epsilon^{-3})$ for RS-NVRGF, showing that normalization improves the dimension dependence and variance reduction improves the $\epsilon$-dependence.
- The stationarity guarantee is a $(\delta,\epsilon)$-Goldstein stationary point, the standard tractable surrogate for Clarke stationarity in nonsmooth nonconvex problems.
- The non-polynomial example $f(x)=\exp(|x|)-x^3$ shows the function class is not limited to polynomial subgradient bounds, so the analysis covers objectives that previously fell outside the known randomized-smoothing theory.
- The high-probability rates depend on exact evaluation of $\alpha(x_t)$, $\beta(x_t,\cdot)$, $\sigma(x_t)$, and $\ell(x_t,\cdot)$ at every iterate; without that access, the stated complexity bounds are not tied to an implementable procedure.
Reading between the lines
- A natural next step is a parameter-free variant that estimates $\alpha$ and $\beta$ online from function-value queries; the step-size logic suggests that conservative estimates keeping iterates inside the level set would be enough to preserve the rates.
- The analysis suggests the real barrier is not global Lipschitzness itself but control of subgradient growth over the bounded level set, so other function classes with such local control may inherit the same rates.
- For composite objectives that arise in adversarial attacks and sensor-network localization, replacing exact growth quantities with finite-difference estimates would make the variance-reduced rate testable in practice, although the paper does not supply that analysis.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an (α,β) subgradient growth condition as a replacement for the global Lipschitz assumption in nonsmooth nonconvex optimization, and shows that spherical randomized smoothing of such functions yields a locally smooth surrogate. It then introduces three zeroth-order algorithms—RS-GF, RS-NGF, and RS-NVRGF—and claims high-probability convergence to (δ,ε)-Goldstein stationary points with sample complexities Õ(d^{5/2}δ^{-1}ε^{-4}), Õ(d^{3/2}δ^{-1}ε^{-4}), and Õ(d^{3/2}δ^{-1}ε^{-3}), respectively. The last rate is advertised as matching the optimal ε-dependence previously known only for globally Lipschitz functions. The paper includes detailed appendix proofs, examples of the new function class, and numerical experiments on a sensor-network-like problem and on black-box CIFAR attacks.
Significance. If the central theorems were valid for an implementable zeroth-order procedure, the paper would make a useful extension of randomized-smoothing theory to locally Lipschitz, possibly non-globally-Lipschitz objectives. The local smoothness proposition for fδ and the high-probability machinery are nontrivial and are supported by detailed derivations. The main limitation is that the algorithms' step sizes and batch sizes depend on exact values of α(x_t), β(x_t,·), σ(x_t), and ℓ(x_t,·), which are not outputs of a zeroth-order oracle and are not stated as assumptions in the formal theorems. The paper's own conclusion concedes this, but the abstract and theorem statements overclaim by presenting the rates as unconditional zeroth-order oracle complexities. Because the mathematical core appears sound under an explicit idealized oracle, the contribution is salvageable with a substantial revision.
major comments (2)
- [Section 4, Theorems 1–3, Algorithms 1–3; Conclusion, Section 6] The step-size and batch-size formulas (4.1)–(4.3) and their formal versions in Appendix D require exact evaluation at every iterate of α(x_t), β(x_t,δ), β(x_t,r), σ(x_t), and ℓ(x_t,r). These quantities are not produced by the stated zeroth-order oracle, which returns only function values, and no estimation procedure or oracle model for them is provided. The formal theorems list only Assumption 1 (level-boundedness of fδ) as the substantive assumption. The Conclusion states that 'our result relies on the assumption of knowing α and β,' but this limitation is not incorporated into the theorem statements or the abstract. The experiments do not close the gap because step sizes are tuned on a grid and batch sizes are selected from a small set rather than set by the theorem's formulas. As stated, the high-probability ε^{-3} rate is a conditional statement about a parameter-idealized algorithm, not a guarantee for a fully specified zeroth-order method.
- [Appendix D, proof of Theorem 2, around Eq. (D.23)] The proof asserts that C_t = O(Δ d^{-1/4}δ^{1/2}) and uses this to conclude the iteration complexity T = Õ(d^{1/2}δ^{-1}ε^{-2}). This is not a valid derivation as written: with ℓ(x,r) = c√d(2α(x)+β(x,δ)+β(x,r+δ))/(2δ), the term √(Δ/(3ℓ(x,·))) is at least of order √(Δδ/(√d M)), where M = sup_{x∈Q}(2α(x)+β(x,·)+β(x,·)) over the level set Q. The displayed bound omits M and also conflates an upper bound with the lower bound actually needed to control 1/C_path. The claimed d-, δ-, and ε-scaling may still be correct after absorbing problem-dependent constants, but the proof as written is incomplete and should explicitly introduce and track such constants.
minor comments (5)
- [Section 4.1 and Appendix D, Theorem 1] The informal step-size formula (4.1) uses c_{t,δ} = α(x_t)+β(x_t,3δ) consistently, while the formal version of Theorem 1 uses β(x_t,δ) in the second term of C_t and β(x_t,3δ) in the others; the notation should be aligned.
- [Algorithm 3 and Theorem 3] The normalized update x_{t+1} = x_t - η_t m_t/‖m_t‖ is undefined when m_t = 0; a convention for that zero-probability or degenerate event should be stated for completeness.
- [Notation and Assumption 1] The paper states early that f is lower bounded by f* and that a Δ > fδ(x1)-f* is available, but these requirements are not listed in the formal theorem hypotheses alongside Assumption 1; they should be made explicit.
- [Appendix D, proof of Theorem 1] In the Bernstein argument the text uses both log(2/p) and log(2T/p) in the step-size bound and in the definition of c; these should be made consistent because the final probability p/T depends on the T-dependent logarithmic factor.
- [Appendix C, Lemma 11 proof] The displayed expression for the recursion of m_t contains a sign ambiguity in the terms involving g(x_{t-1},S) and g(x_t,S); the intended centered random variable should be written explicitly to avoid confusion.
Circularity Check
No circularity: the convergence rates are derived from the stated (alpha,beta) growth assumptions; the need to know alpha and beta is an acknowledged implementability limitation, not a logical loop.
full rationale
The paper's derivation chain is self-contained given its assumptions. The (alpha,beta) subgradient growth condition is a definition of the function class, and the generalized smoothness of the smoothed surrogate f_delta, the gradient-estimator variance bounds, and the descent inequalities are all derived from that definition. The step sizes in (4.1)-(4.3) depend on alpha(x_t), beta(x_t,delta), sigma(x_t), and ell(x_t,r), but these quantities are properties of the assumed function class, not outputs of the convergence claim; the target Goldstein-stationarity guarantee is not used to construct the step sizes. The paper explicitly acknowledges the resulting implementability gap in Section 6: 'our result relies on the assumption of knowing alpha and beta. A possible direction is to design a parameter-free method.' This is a real limitation of the algorithmic claim, but it is not circularity: no fitted parameter is renamed as a prediction, and no equation used to prove the rate is identical by construction to the rate itself. The proof reuses standard results from Lin et al. (2022), Chen et al. (2023a), Hubler et al. (2024), and Chezhegov et al. (2024), which are external published results; the authors' self-citations, such as Lin et al. (2024) for an experimental setup, are not load-bearing for the main theorems. The experimental section tunes step sizes and batch sizes from finite grids, but the paper does not claim these experiments instantiate the theorem's exact parameter choices, and the core contribution is the theoretical convergence analysis. Consequently, no specific circular reduction can be exhibited, and the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (3)
- Step-size multiplier η (experiments) =
Grid-tuned: 2^{-2i+1} for i in [-6,6] in simulation; {0.0005,0.001,0.005,0.01,0.05,0.1,0.5} in adversarial attack
- Batch sizes B_t, b_t and period q (experiments) =
Grid-tuned: [2,4,8,16,32] for simulation; [16,32,64,128,256] for attack; q from [2,4,8,16,32] or [200,400,800,1600,3200]
- Smoothing radius δ =
10^{-4} in simulation, 10^{-3} in adversarial attack
assumptions (7)
- domain assumption f satisfies sup_{ζ∈∂f(x)} ||ζ|| ≤ α(x) and |α(x)-α(y)| ≤ β(x,||x-y||) with β non-decreasing in r
- domain assumption For every ν, the level set {x : fδ(x) ≤ ν} is bounded
- domain assumption f is lower bounded by f⋆
- ad hoc to paper The optimizer can evaluate α(x_t), β(x_t,r), σ(x_t), and ℓ(x_t,r) exactly to set η_t
- standard math Clarke mean value theorem for Lipschitz functions
- standard math Bernstein inequality for martingale difference sequences
- standard math Sub-Gaussian concentration for Lipschitz functions on the sphere
Cite this review
Pith. "Pith review of Revisiting Randomized Smoothing: Nonsmooth Nonconvex Optimization Beyond Global Lipschitz Continuity." pith.science (2026). https://pith.science/paper/6532TYKR
@misc{pith2026250813496,
author = {Pith},
title = {Pith review of: Revisiting Randomized Smoothing: Nonsmooth Nonconvex Optimization Beyond Global Lipschitz Continuity},
year = {2026},
howpublished = {\url{https://pith.science/paper/6532TYKR}},
note = {Machine review of arXiv:2508.13496}
}
abstract
Randomized smoothing is a widely adopted technique for optimizing nonsmooth objective functions. However, its efficiency analysis typically relies on global Lipschitz continuity, a condition rarely met in practical applications. To address this limitation, we introduce a new subgradient growth condition that naturally encompasses a wide range of locally Lipschitz functions, with the classical global Lipschitz function as a special case. Under this milder condition, we prove that randomized smoothing yields a differentiable function that satisfies certain generalized smoothness properties. To optimize such functions, we propose novel randomized smoothing gradient algorithms that, with high probability, converge to $(\delta, \epsilon)$-Goldstein stationary points and achieve a sample complexity of $\tilde{\mathcal{O}}(d^{5/2}\delta^{-1}\epsilon^{-4})$. By incorporating variance reduction techniques, we further improve the sample complexity to $\tilde{\mathcal{O}}(d^{3/2}\delta^{-1}\epsilon^{-3})$, matching the optimal $\epsilon$-bound under the global Lipschitz assumption, up to a logarithmic factor. Experimental results validate the effectiveness of our proposed algorithms.
Figures
Forward citations
Cited by 2 Pith papers
-
On computing Goldstein approximate second-order stationary points of structured nonsmooth nonconvex programs
A randomized first-order algorithm computes Goldstein approximate second-order stationary points of L-smooth nonconvex functions with oracle complexity Õ(ΔL⁸n²/ε⁹ + ΔL⁶n³/ε⁷).
-
A Gaussian smoothing-based zeroth-order method for Goldstein second-order stationarity
Introduces the Goldstein second-order δ-subdifferential for C¹,¹ functions and a Gaussian-smoothing cubic-regularization zeroth-order method that provably finds (ε₁, ε₂, δ)-second-order stationary points under a coerc...
Reference graph
Works this paper leans on
-
[1]
Variance reduction for faster non-convex optimization
Zeyuan Allen-Zhu and Elad Hazan. Variance reduction for faster non-convex optimization. In International conference on machine learning, pages 699--707. PMLR, 2016
work page 2016
-
[2]
Introduction to Nonsmooth Optimization: theory, practice and software, volume 12
Adil Bagirov, Napsu Karmitsa, and Marko M M \"a kel \"a . Introduction to Nonsmooth Optimization: theory, practice and software, volume 12. Springer, 2014
work page 2014
-
[3]
Krishnakumar Balasubramanian and Saeed Ghadimi. Zeroth-order nonconvex stochastic optimization: Handling constraints, high dimensionality, and saddle points. Foundations of Computational Mathematics, 22 0 (1): 0 35--76, 2022
work page 2022
-
[4]
Faster gradient-free algorithms for nonsmooth nonconvex stochastic optimization
Lesi Chen, Jing Xu, and Luo Luo. Faster gradient-free algorithms for nonsmooth nonconvex stochastic optimization. In International Conference on Machine Learning, pages 5219--5233. PMLR, 2023 a
work page 2023
-
[5]
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pages 15--26, 2017
work page 2017
-
[6]
Generalized-smooth nonconvex optimization is as efficient as smooth nonconvex optimization
Ziyi Chen, Yi Zhou, Yingbin Liang, and Zhaosong Lu. Generalized-smooth nonconvex optimization is as efficient as smooth nonconvex optimization. In International Conference on Machine Learning, pages 5396--5427. PMLR, 2023 b
work page 2023
-
[7]
Gradient clipping improves adagrad when the noise is heavy-tailed
Savelii Chezhegov, Yaroslav Klyukin, Andrei Semenov, Aleksandr Beznosikov, Alexander Gasnikov, Samuel Horv \'a th, Martin Tak \'a c , and Eduard Gorbunov. Gradient clipping improves adagrad when the noise is heavy-tailed. arXiv preprint arXiv:2406.04443, 2024
arXiv 2024
-
[8]
Optimization and nonsmooth analysis
Frank H Clarke. Optimization and nonsmooth analysis. SIAM, 1990
work page 1990
Show all 46 references
-
[9]
Complexity guarantees for an implicit smoothing-enabled method for stochastic mpecs
Shisheng Cui, Uday V Shanbhag, and Farzad Yousefian. Complexity guarantees for an implicit smoothing-enabled method for stochastic mpecs. Mathematical Programming, 198 0 (2): 0 1153--1225, 2023
2023
-
[10]
Modern nonconvex nondifferentiable optimization
Ying Cui and Jong-Shi Pang. Modern nonconvex nondifferentiable optimization. SIAM, 2021
2021
-
[11]
Momentum improves normalized sgd
Ashok Cutkosky and Harsh Mehta. Momentum improves normalized sgd. In International conference on machine learning, pages 2260--2268. PMLR, 2020
2020
-
[12]
Momentum-based variance reduction in non-convex sgd
Ashok Cutkosky and Francesco Orabona. Momentum-based variance reduction in non-convex sgd. Advances in neural information processing systems, 32, 2019
2019
-
[13]
Randomized smoothing for stochastic optimization
John C Duchi, Peter L Bartlett, and Martin J Wainwright. Randomized smoothing for stochastic optimization. SIAM Journal on Optimization, 22 0 (2): 0 674--701, 2012
2012
-
[14]
Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator
Cong Fang, Chris Junchi Li, Zhouchen Lin, and Tong Zhang. Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator. Advances in neural information processing systems, 31, 2018
2018
-
[15]
Online convex optimization in the bandit setting: gradient descent without a gradient
Abraham D Flaxman, Adam Tauman Kalai, and H Brendan McMahan. Online convex optimization in the bandit setting: gradient descent without a gradient. In Proceedings of the sixteenth annual ACM-SIAM symposium on Discrete algorithms, pages 385--394, 2005
2005
-
[16]
Stochastic weakly convex optimization beyond lipschitz continuity
Wenzhi Gao and Qi Deng. Stochastic weakly convex optimization beyond lipschitz continuity. arXiv preprint arXiv:2401.13971, 2024
2024 arXiv
-
[17]
Stochastic first-and zeroth-order methods for nonconvex stochastic programming
Saeed Ghadimi and Guanghui Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM journal on optimization, 23 0 (4): 0 2341--2368, 2013
2013
-
[18]
Convergence rates for deterministic and stochastic subgradient methods without lipschitz continuity
Benjamin Grimmer. Convergence rates for deterministic and stochastic subgradient methods without lipschitz continuity. SIAM Journal on Optimization, 29 0 (2): 0 1350--1365, 2019
2019
-
[19]
From gradient clipping to normalization for heavy tailed sgd
Florian H \"u bler, Ilyas Fatkhullin, and Niao He. From gradient clipping to normalization for heavy tailed sgd. arXiv preprint arXiv:2410.13849, 2024
2024 arXiv
-
[20]
Non-convex optimization for machine learning
Prateek Jain, Purushottam Kar, et al. Non-convex optimization for machine learning. Foundations and Trends in Machine Learning , 10 0 (3-4): 0 142--363, 2017
2017
-
[21]
An algorithm with optimal dimension-dependence for zero-order nonsmooth nonconvex stochastic optimization
Guy Kornowski and Ohad Shamir. An algorithm with optimal dimension-dependence for zero-order nonsmooth nonconvex stochastic optimization. arXiv preprint arXiv:2307.04504, 2023
2023 arXiv
-
[22]
Derivative-free optimization methods
Jeffrey Larson, Matt Menickelly, and Stefan M Wild. Derivative-free optimization methods. Acta Numerica, 28: 0 287--404, 2019
2019
-
[23]
Subdifferentially polynomially bounded functions and gaussian smoothing-based zeroth-order optimization
Ming Lei, Ting Kei Pong, Shuqin Sun, and Man-Chung Yue. Subdifferentially polynomially bounded functions and gaussian smoothing-based zeroth-order optimization. arXiv preprint arXiv:2405.04150, 2024
2024 arXiv
-
[24]
Convex and non-convex optimization under generalized smoothness
Haochuan Li, Jian Qian, Yi Tian, Alexander Rakhlin, and Ali Jadbabaie. Convex and non-convex optimization under generalized smoothness. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[25]
Gradient-free methods for deterministic and stochastic nonsmooth nonconvex optimization
Tianyi Lin, Zeyu Zheng, and Michael Jordan. Gradient-free methods for deterministic and stochastic nonsmooth nonconvex optimization. Advances in Neural Information Processing Systems, 35: 0 26160--26175, 2022
2022
-
[26]
Decentralized gradient-free methods for stochastic non-smooth non-convex optimization
Zhenwei Lin, Jingfan Xia, Qi Deng, and Luo Luo. Decentralized gradient-free methods for stochastic non-smooth non-convex optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17477--17486, 2024
2024
-
[27]
A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications
Sijia Liu, Pin-Yu Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III, and Pramod K Varshney. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, 37 0 (5): 0 43--54, 2020
2020
-
[28]
Nonconvex stochastic optimization under heavy-tailed noises: Optimal convergence without gradient clipping
Zijian Liu and Zhengyuan Zhou. Nonconvex stochastic optimization under heavy-tailed noises: Optimal convergence without gradient clipping. arXiv preprint arXiv:2412.19529, 2024
2024 arXiv
-
[29]
Near-optimal non-convex stochastic optimization under generalized smoothness
Zijian Liu, Srikanth Jagabathula, and Zhengyuan Zhou. Near-optimal non-convex stochastic optimization under generalized smoothness. arXiv preprint arXiv:2302.06032, 2023
2023 arXiv
-
[30]
relative continuity
Haihao Lu. “relative continuity” for non-lipschitz nonsmooth convex optimization using stochastic (or deterministic) mirror descent. INFORMS Journal on Optimization, 1 0 (4): 0 288--303, 2019
2019
-
[31]
Stability and convergence of stochastic gradient clipping: Beyond lipschitz continuity and smoothness
Vien V Mai and Mikael Johansson. Stability and convergence of stochastic gradient clipping: Beyond lipschitz continuity and smoothness. In International Conference on Machine Learning, pages 7325--7335. PMLR, 2021
2021
-
[32]
Fine-tuning language models with just forward passes
Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems, 36: 0 53038--53075, 2023
2023
-
[33]
Directional smoothness and gradient methods: Convergence and adaptivity
Aaron Mishkin, Ahmed Khaled, Yuanhao Wang, Aaron Defazio, and Robert M Gower. Directional smoothness and gradient methods: Convergence and adaptivity. arXiv preprint arXiv:2403.04081, 2024
2024 arXiv
-
[34]
A simplex method for function minimization
John A Nelder and Roger Mead. A simplex method for function minimization. The computer journal, 7 0 (4): 0 308--313, 1965
1965
-
[35]
Random gradient-free minimization of convex functions
Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17: 0 527--566, 2017
2017
-
[36]
Sum of squares method for sensor network localization
Jiawang Nie. Sum of squares method for sensor network localization. Computational Optimization and Applications, 43 0 (2): 0 151--179, 2009
2009
-
[37]
Variance-reduced clipping for non-convex optimization
Amirhossein Reisizadeh, Haochuan Li, Subhro Das, and Ali Jadbabaie. Variance-reduced clipping for non-convex optimization. arXiv preprint arXiv:2303.00883, 2023
2023 arXiv
-
[38]
Simulation and the Monte Carlo method
Reuven Y Rubinstein and Dirk P Kroese. Simulation and the Monte Carlo method. John Wiley & Sons, 2016
2016
-
[39]
An optimal algorithm for bandit and zero-order convex optimization with two-point feedback
Ohad Shamir. An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. Journal of Machine Learning Research, 18 0 (1): 0 1703--1713, 2017
2017
-
[40]
Gradient normalization with (out) clipping ensures convergence of nonconvex sgd under heavy-tailed noise with improved results
Tao Sun, Xinwang Liu, and Kun Yuan. Gradient normalization with (out) clipping ensures convergence of nonconvex sgd under heavy-tailed noise with improved results. arXiv preprint arXiv:2410.16561, 2024
2024
-
[41]
Toward a unified theory of gradient descent under generalized smoothness
Alexander Tyurin. Toward a unified theory of gradient descent under generalized smoothness. arXiv preprint arXiv:2412.11773, 2024
2024 arXiv
-
[42]
High-dimensional statistics: A non-asymptotic viewpoint, volume 48
Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cambridge university press, 2019
2019
-
[43]
Why gradient clipping accelerates training: A theoretical justification for adaptivity
Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie. Why gradient clipping accelerates training: A theoretical justification for adaptivity. arXiv preprint arXiv:1905.11881, 2019
1905 arXiv
-
[44]
Complexity of finding stationary points of nonconvex nonsmooth functions
Jingzhao Zhang, Hongzhou Lin, Stefanie Jegelka, Suvrit Sra, and Ali Jadbabaie. Complexity of finding stationary points of nonconvex nonsmooth functions. In International Conference on Machine Learning, pages 11173--11182. PMLR, 2020
2020
-
[45]
Stochastic nested variance reduction for nonconvex optimization
Dongruo Zhou, Pan Xu, and Quanquan Gu. Stochastic nested variance reduction for nonconvex optimization. Journal of machine learning research, 21 0 (103): 0 1--63, 2020
2020
-
[46]
A unified analysis for the subgradient methods minimizing composite nonconvex, nonsmooth and non-lipschitz functions
Daoli Zhu, Lei Zhao, and Shuzhong Zhang. A unified analysis for the subgradient methods minimizing composite nonconvex, nonsmooth and non-lipschitz functions. arXiv preprint arXiv:2308.16362, 2023
2023
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.