Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Optimization via First-Order Switching Methods: Skew-Symmetric Dynamics and Optimistic Discretization

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For convex smooth f and g, Switching Gradient Method still needs O(ε^{-2}) iterations; the paper traces this to its discontinuous, skew-symmetric flow rather than to loose analysis.

desk verdict Theorem 2 is a real first—a formal O(epsilon^-2) complexity for smooth SGM—but the paper's stronger claim that this rate is fundamental is not backed by a lower bound and should be softened or proven. read the letter →

arxiv 2505.09146 v1 pith:S7NYEOKH submitted 2025-05-14 math.OC

Antesh Upadhyay , Sang Bin Moon , Abolfazl Hashemi This is my paper · ORCID
classification math.OC MSC 90C2590C3065K05
keywords switchinggradientmethodfunctionalconstraintsconvexsmoothoptimizationiterationcomplexityskew-symmetricdynamicsoptimisticdiscretizationproximalpointsoft
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the Switching Gradient Method (SGM) for functional-constraint problems min f(w) s.t. g(w)≤0 inherits the faster convergence that gradient descent enjoys when f and g are smooth. Its answer, for convex L-smooth functions, is no: SGM still needs O($ε^{{-2}}$) iterations to return an ε-solution, the same order as the nonsmooth case, whereas unconstrained gradient descent would need only O($ε^{{-1}}$). The paper argues this is not proof slack but a property of the algorithm: SGM's continuous-time limit is a skew-symmetric flow, and SGM is its forward Euler discretization, a scheme that rotates around the solution instead of settling. From that diagnosis it designs soft-switching and optimistic (backward-Euler / proximal) variants, proving rates for them and showing an optimistic implicit method can reach O($ε^{{-1}}$) when its fixed-point subproblems are solvable.

What carries the argument

The load-bearing object is the smoothed switching operator $F(w) = \sigma_\beta(g(w)-\epsilon)\nabla g(w) + (1-\sigma_\beta(g(w)-\epsilon))\nabla f(w)$ and its Jacobian identity: the term $\sigma_\beta'(g(w)-\epsilon)(\nabla g(w)-\nabla f(w))\nabla g(w)^\top$ is an outer product that is symmetric only when the two gradients are aligned, so generically the flow has a skew-symmetric part. For the algorithmic guarantees, the trimmed hinge $\sigma_\beta(x) = \mathrm{Proj}_{[0,1]}(1+\beta x)$ replaces the sigmoid so that the soft switching methods (SSGM, SSPPM, SSPPM-E, SPPM) are Lipschitz; forward Euler discretization reproduces SGM and SSGM, while backward Euler corresponds to an optimistic proximal-point update. The quantity $\Delta_{\max}$ acts as the effective variance that degrades the rate, and the Lipschitz constant of the soft flow, bounded in Proposition 2 by $2(L + G^2\beta)$, controls when the implicit SSPPM update is guaranteed to exist.

What would settle it

Run SGM on a fixed convex L-smooth instance, for example two random quadratics in $R^{10}$ as in the paper, and record for a range of small ε the first T at which the averaged iterate satisfies both f(w̄)-f(w*) ≤ ε and g(w̄) ≤ ε. If these T values grow like $ε^{{-1}}$ for any such instance, the fundamentality claim is refuted; a matching example forcing T ~ $ε^{{-2}}$, or a formal lower bound for all first-order switching rules, would confirm it.

Watch

Extended reading notes

Core claim

Under the assumption that f and g are convex and L-smooth, SGM has iteration complexity O($ε^{{-2}}$), with the parameters in Theorem 2: η = min{1/(2L), $\sqrt$(D²/(2L Δ_max T))} and ε = 2LD²/T + $\sqrt$(8LD²Δ_max/T). The extra term Δ_max, the largest gap between the value at the optimum and the global minimum of f or g, plays the role that gradient noise plays in stochastic gradient descent: it is a variance created by the switching dynamics, not by randomness. The paper identifies the mechanism behind this ceiling by smoothing SGM's discontinuous update into the flow $\dot w = -F(w)$ with $F(w) = \sigma_\beta(g(w)-\epsilon)\nabla g(w) + (1-\sigma_\beta(g(w)-\epsilon))\nabla f(w)$. The Jacobian of this flow has a nonvanishing skew-symmetric component whenever $\nabla f$ and $\nabla g$ are linearly independent, and that skew part grows as the trajectory approaches the constraint boundary and as β grows. Since SGM is the forward Euler discretization of the limit of this flow, it inherits the slow convergence characteristic of discretizing rotating, non-gradient dynamics; the same viewpoint motivates the paper's new algorithms, backward-Euler optimistic discretization and soft switching via a trimmed hinge.

Load-bearing premise

The load-bearing premise is that the sigmoid-smoothed ODE (20) faithfully represents the actual discontinuous SGM, so conclusions about skew-symmetric flows transfer to the discrete method; the upper bound of Theorem 2 stands regardless, but the paper's 'inherent limitation' claim collapses if this smoothing transfer is unfaithful, and the omitted proofs of Theorems 4 and 7 plus the fixed-point solvability assumption for SSPPM are secondary verification points.

Editorial extensions

If this is right

  • SGM's smooth-case rate is O(ε^{-2}), so smoothness does not automatically improve switching methods the way it improves gradient descent; the paper's continuous-time argument says the obstruction is structural.
  • Soft switching (SSGM) keeps the optimal O(ε^{-2}) rate under Lipschitz convexity while producing smoother trajectories that oscillate less, as the paper's experiments show.
  • When the implicit update exists, optimistic discretization (SSPPM) reaches O(ε^{-1}) under pure convexity; guaranteeing existence through contractivity costs back to O(ε^{-2}) in the worst case, but with constants independent of L.
  • The explicit optimistic method SSPPM-E and the hard-switching proximal variant SPPM both recover optimal O(ε^{-2}) guarantees without solving nonlinear systems at each step.
  • The paper's analysis connects switching methods to variational inequalities and min-max optimization, where optimism is the established cure for skew-symmetric flows, so those techniques can be transplanted to functional-constraint problems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • In my reading, since the Δ_max term resembles a variance in SGD, variance-reduction schedules such as gradient averaging across switching states are a natural next step the paper does not take; if they work, SGM-type methods might recover O(ε^{-1}) in the smooth case.
  • The skew-symmetric analysis suggests that ergodic averages are the right success criterion for switching methods, but last-iterate convergence is the harder and more informative target; backward-Euler methods should be tested for last-iterate convergence directly.
  • A testable design principle follows from Proposition 1: switching rules whose smoothed flow has a vanishing skew-symmetric component should escape the ε^{-2} barrier, and constructing such rules is an open problem.
  • The paper's own conjecture that β can be chosen independent of ε would, if true, make SSGM and SSPPM-E strictly more attractive and likely to show O(ε^{-1}) empirical behavior; this is directly checkable on its quadratic test instances. Verification points the paper leaves open include the omitted proofs of Theorems 4 and 7 and the fixed-point solvability assumption for SSPPM.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies the Switching Gradient Method (SGM) for convex functional-constrained problems of the form min f(w) subject to g(w) <= 0. Theorem 1 gives the classical O(epsilon^{-2}) rate for G-Lipschitz nonsmooth f and g, while Theorem 2 provides an explicit O(epsilon^{-2}) rate when both f and g are convex and L-smooth, with a fixed step size and tolerance depending on the initialization distance D and the suboptimality gap Delta_max. The authors interpret this as showing that smoothness does not improve SGM's worst-case rate, and they offer two informal arguments for why this is fundamental: a penalty-function analogy (GD on f + rho [g - epsilon]_+) and a continuous-time sigmoid-smoothed ODE whose Jacobian contains a skew-symmetric term (Proposition 1). The paper then introduces soft switching and optimistic discretization variants (SSGM, SSPPM, SSPPM-E, SPPM) and states convergence guarantees for them, with Theorems 4 and 7 given without proofs. A short experiment section illustrates oscillatory behavior and compares the methods on random convex quadratics.

Significance. The upper-bound part of the paper is a solid, self-contained contribution: the proof of Theorem 2 is careful, the parameter choices are explicit and derived analytically, and the soft-switching and optimistic variants are natural and potentially useful algorithmic ideas. No fitted constants appear in the analysis, which is a strength; the rates follow from convexity, smoothness, and Lipschitz assumptions. If the proofs of Theorems 4 and 7 are supplied and the 'fundamental limitation' claim is downgraded to a worst-case upper bound, the paper would be a useful addition to the first-order constrained-optimization literature. At present, however, the advertised conceptual conclusion—that the O(epsilon^{-2}) rate is inherent to SGM's dynamics—is not supported by the arguments in Sections 3.3 and 3.4, and two of the stated theorems are unproved.

major comments (2)
  1. [§3.3 and §3.4, Eqs. (17)–(21)] The paper's central claim that SGM's O(epsilon^{-2}) rate under smoothness is 'fundamental' and 'not an artifact of our analysis' is not established. Proposition 1 computes the Jacobian of the sigmoid-smoothed flow (20), but the skew-symmetric outer-product term sigma'_beta(...)(nabla g - nabla f)(nabla g)^T in (21) is created by the smoothing and has no counterpart in the hard-switch field (19): wherever (19) is differentiable its Jacobian is either nabla^2 f or nabla^2 g, both symmetric. No approximation or limit theorem is proved that transfers a qualitative property of the smoothed ODE (20) to the iteration complexity of the discontinuous update (SGM). The penalty argument in Section 3.3 analyzes gradient descent on the nonsmooth penalty (17), not SGM, and no Omega(epsilon^{-2}) lower bound is provided. The manuscript should either prove a genuine lower bound or rephrase the conclusion as a worst-case upper bound (e.g., 'SGM can require O(epsilon^{-2}) iterations under this analysis'), removing the 'fundamental/inherent' language from the abstract and from Section 3.
  2. [Theorems 4 and 7] Theorem 4 (smooth SSGM) and Theorem 7 (SPPM) are stated with their proofs omitted 'for brevity.' Because these are substantive convergence claims, not simple lemmas with immediately obvious proofs, the current manuscript does not establish them. For a journal version, full proofs must be included in the appendix, or the statements should be explicitly labeled as conjectures; otherwise the claimed guarantees for SSGM in the smooth case and for SPPM are unsupported.
minor comments (5)
  1. [Section 6, Fig. 1 caption] The caption says 'For each method, we used the best constant step-size epsilon = 0.001'; this should read 'step size eta = 0.001', since epsilon denotes the tolerance throughout the paper.
  2. [Section 3.2, Eqs. (11)–(15)] The dual/stochastic-oracle analogy is informal, as the text acknowledges, but the notation E[nabla ell_tilde(w_t, z)] in (15) and the claim that SGM 'acts as a stochastic approximation' could be misread as a proof; suggest adding a sentence stating explicitly that this is only intuition and is not used in the derivation of Theorem 2.
  3. [Appendix F, Eq. (69)] The equality h_t(w_t) - h_t(w_{t+1}) = |h_t(w_t) - h_t(w_{t+1})| is not generally true; the intended argument is |h_t(w_t) - h_t(w_{t+1})| <= G||w_t - w_{t+1}||, and the proof should be rewritten accordingly.
  4. [Section 4, Eq. (24)] The symbol sigma_beta(.) is used for both the sigmoid in (20) and the trimmed hinge in (24); this notational reuse is confusing, particularly because Theorem 3 and the experiments use a different smoothing function from the one discussed in Section 3.4.
  5. [Section 5.1] The derivation of epsilon = DG/sqrt(T) after Proposition 2 is compressed; expanding the algebra would improve readability and make the pessimistic O(epsilon^{-2}) conclusion easier to verify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SGM upper bounds are proved from convexity/smoothness with analytically chosen constants; the 'fundamental limitation' claim is an unsupported extrapolation, not a reduction of output to input.

full rationale

Theorem 1 and Theorem 2 are self-contained proofs: the step size and tolerance constants in Eq. (7) and Eq. (10) are chosen analytically from the assumed Lipschitz/smoothness parameters, T, D, and Δ_max, to make the telescoping-sum bound O(ε^-2), and no quantity is fitted to data or to the desired rate. The SGM update is not defined in terms of the convergence conclusion, and the averaged-iterate solution is not used to set any parameter. The informal SGD-duality discussion in Section 3.2 and the penalty-function analogy in Section 3.3 are heuristic interpretations of the already-proved upper bound; they do not feed values back into Theorem 2, so they are not circular. Section 3.4's Proposition 1 computes the Jacobian of the sigmoid-smoothed ODE (20); whether or not that computation rigorously transfers to the discontinuous SGM field (19) is a question of evidential support for the 'inherent limitation' claim, not circularity. No load-bearing uniqueness theorem is imported from the authors' own prior work; the self-citations in the paper are background or method references and do not supply the central convergence results. The trimmed-hinge and implicit methods are analyzed directly in Theorems 3-7, again with analytic constants. The omitted proofs of Theorem 4 and Theorem 7 are acknowledged brevity limitations, not circular steps. Thus the central derivation is self-contained against external assumptions, and no circular step is identified.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central theorems rest on standard convexity and smoothness assumptions plus a few hand-chosen algorithm parameters. The most fragile additions are the sigmoid-smoothing proxy and the penalty-based tightness argument, which are used to claim that SGM's rate is inherent but are not formal lower-bound proofs. No new physical or mathematical entities are introduced.

free parameters (3)
  • beta (smoothing parameter for soft switching) = 2/epsilon in Theorems 3-7; beta = 1 in experiments
    Chosen by hand through the pessimistic bound sum sigma < T in the proofs. The hoped-for O(epsilon^-1) complexity for SSPPM would require beta to be treated as a constant independent of epsilon, which the paper only conjectures.
  • rho (penalty parameter in the tightness argument) = epsilon^-alpha for alpha > 0
    Introduced in Section 3.3 to argue that SGM behaves like gradient descent on a nonsmooth penalty and therefore cannot beat O(epsilon^-2). This parameter is used in an informal argument, not in a theorem.
  • Step size eta = D/(G sqrt(T)), min{1/(2L), sqrt(D^2/(2L Delta_max T))}, D/(G sqrt(2T)), and 1/(2(L+G^2 beta)) depending on the theorem
    The step sizes are analytical choices fixed before the proofs, not fitted to data, but the derived rates depend on these choices and they are hand-selected.
assumptions (6)
  • domain assumption f and g are convex and differentiable, with L-smoothness or G-Lipschitzness as assumed per theorem.
    Standard problem assumptions stated in Section 2 and used throughout the proofs, especially Lemma 2 in Appendix B.
  • domain assumption f and g are bounded below so that tilde f, tilde g, and Delta_max are finite.
    Equation (8) in Section 3.2 defines Delta_max using global minima; Lemma 2 requires a finite lower bound. The paper handles affine L=0 cases by convention.
  • domain assumption The SSPPM nonlinear equation w = wt - eta F(w) has a solution for all t.
    Explicitly assumed in Theorem 5. Section 5.1 provides sufficient conditions via non-expansiveness of eta F, but for the desired O(epsilon^-1) rate these conditions force beta to scale with 1/epsilon, restoring O(epsilon^-2).
  • ad hoc to paper The sigmoid-smoothed ODE (20) is an adequate continuous-time proxy for SGM's dynamics, and its skew-symmetric Jacobian explains SGM's rates.
    Section 3.4 and Proposition 1. This modeling step underpins the claim that the O(epsilon^-2) rate is fundamental, but it is a heuristic transfer from a smoothed ODE to a discontinuous discrete method, not a proof for SGM itself.
  • ad hoc to paper The penalty reformulation f + rho[g-epsilon]_+ has the same epsilon-solutions as problem (P) for large rho, and its nonsmoothness forces Omega(epsilon^-2) for SGM.
    Section 3.3 argues tightness by analogy with a nonsmooth penalty. The equivalence for large rho is standard, but the lower-bound conclusion is informal and no rigorous Omega(epsilon^-2) proof is given.
  • standard math Standard fixed-point, ODE existence, and convex-analysis results hold: polarization identity, descent lemma, Caratheodory existence, Picard-Lindelof, Banach and Krasnoselskii-Mann theorems.
    Invoked in Appendices A-F and Section 5.1. These are classical background results, not new assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimization via First-Order Switching Methods: Skew-Symmetric Dynamics and Optimistic Discretization." pith.science (2026). https://pith.science/paper/S7NYEOKH

@misc{pith2026250509146,
  author       = {Pith},
  title        = {Pith review of: Optimization via First-Order Switching Methods: Skew-Symmetric Dynamics and Optimistic Discretization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S7NYEOKH}},
  note         = {Machine review of arXiv:2505.09146}
}
read the original abstract

Large-scale constrained optimization problems are at the core of many tasks in control, signal processing, and machine learning. Notably, problems with functional constraints arise when, beyond a performance{\nobreakdash-}centric goal (e.g., minimizing the empirical loss), one desires to satisfy other requirements such as robustness, fairness, etc. A simple method for such problems, which remarkably achieves optimal rates for non-smooth, convex, strongly convex, and weakly convex functions under first-order oracle, is Switching Gradient Method (SGM): in each iteration depending on a predetermined constraint violation tolerance, use the gradient of objective or the constraint as the update vector. While the performance of SGM is well-understood for non-smooth functions and in fact matches its unconstrained counterpart, i.e., Gradient Descent (GD), less is formally established about its convergence properties under the smoothness of loss and constraint functions. In this work, we aim to fill this gap. First, we show that SGM may not benefit from faster rates under smoothness, in contrast to improved rates for GD under smoothness. By taking a continuous-time limit perspective, we show the issue is fundamental to SGM's dynamics and not an artifact of our analysis. Our continuous-time limit perspective further provides insights towards alleviating SGM's shortcomings. Notably, we show that leveraging the idea of optimism, a well-explored concept in variational inequalities and min-max optimization, could lead to faster methods. This perspective further enables designing a new class of ``soft'' switching methods, for which we further analyze their iteration complexity under mild assumptions.

Figures

Figures reproduced from arXiv: 2505.09146 by the authors.

Figure 1
Figure 1. Verifying experiments on an instance of (P) with conv [PITH_FULL_IMAGE:figures/full_fig_p019_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

    cs.LG 2026-03 conditional novelty 6.0 of 10

    A single-loop softmax-weighted switching-gradient method solves constrained federated minimax problems at Õ(ε^{-4}) oracle complexity with high-probability guarantees and partial-participation analysis.

Reference graph

Works this paper leans on

55 extracted references · 51 canonical work pages · cited by 1 Pith paper

  1. [1]

    Parallel and distributed methods for constrained nonconvex optimization—part i: Th eory,

    G. Scutari, F. Facchinei, and L. Lampariello, “Parallel and distributed methods for constrained nonconvex optimization—part i: Th eory,” IEEE Transactions on Signal Processing , vol. 65, no. 8, pp. 1929–1944, 2016

    work page 1929

  2. [2]

    Penalty dual deco mposi- tion method for nonsmooth nonconvex optimization—Part II: Applica- tions,

    Q. Shi, M. Hong, X. Fu, and T.-H. Chang, “Penalty dual deco mposi- tion method for nonsmooth nonconvex optimization—Part II: Applica- tions,” IEEE Transactions on Signal Processing, vol. 68, pp. 4242–4257, 2020

    work page 2020

  3. [3]

    Applied and computational optimal control,

    K. L. Teo, B. Li, C. Yu, V. Rehbock, et al., “Applied and computational optimal control,” Optimization and Its Applications , 2021

    work page 2021

  4. [4]

    Stable and fair classification,

    L. Huang and N. Vishnoi, “Stable and fair classification, ” in Interna- tional Conference on Machine Learning , pp. 2879–2890, PMLR, 2019

    work page 2019

  5. [5]

    Neyman-pearson classification , convexity and stochastic constraints,

    P. Rigollet and X. Tong, “Neyman-pearson classification , convexity and stochastic constraints,” Journal of Machine Learning Research , 2011

    work page 2011

  6. [6]

    Fairness constraints: A flexible approach for fair classifi cation,

    M. B. Zafar, I. Valera, M. Gomez-Rodriguez, and K. P. Gumm adi, “Fairness constraints: A flexible approach for fair classifi cation,” The Journal of Machine Learning Research , vol. 20, no. 1, pp. 2737–2778, 2019

    work page 2019

  7. [7]

    Oracle complexity of single-loop sw itching sub- gradient methods for non-smooth weakly convex functional c onstrained optimization,

    Y. Huang and Q. Lin, “Oracle complexity of single-loop sw itching sub- gradient methods for non-smooth weakly convex functional c onstrained optimization,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023

    work page 2023

  8. [8]

    D. P. Bertsekas, Constrained optimization and Lagrange multiplier methods. Academic press, 2014

    2014

Show all 55 references
  1. [9]

    A first-order primal-dual algo rithm for convex problems with applications to imaging,

    A. Chambolle and T. Pock, “A first-order primal-dual algo rithm for convex problems with applications to imaging,” Journal of mathemat- ical imaging and vision , vol. 40, pp. 120–145, 2011

    2011

  2. [10]

    Solving convex smooth function con strained optimization is almost as easy as unconstrained optimizati on,

    Z. Zhang and G. Lan, “Solving convex smooth function con strained optimization is almost as easy as unconstrained optimizati on,” arXiv preprint arXiv:2210.05807, 2022. 20

    2022

  3. [11]

    Stochastic first-order met hods for con- vex and nonconvex functional constrained optimization,

    D. Boob, Q. Deng, and G. Lan, “Stochastic first-order met hods for con- vex and nonconvex functional constrained optimization,” Mathematical Programming, pp. 1–65, 2022

    2022

  4. [12]

    A fast single-loop primal-dual algorithm for non-convex functional constrai ned optimiza- tion,

    J. G. Kim, A. Chandra, A. Hashemi, and C. Brinton, “A fast single-loop primal-dual algorithm for non-convex functional constrai ned optimiza- tion,” arXiv preprint arXiv:2406.17107 , 2024

    2024

  5. [13]

    Nesterov, Introductory Lectures on Convex Optimization: A Basic Course

    Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course. Springer, New York, 2004

    2004

  6. [14]

    Parallel and distributed methods for constrained noncon- vex optimization-Part II: Applications in communications and machine learning,

    G. Scutari, F. Facchinei, L. Lampariello, S. Sardellit ti, and P. Song, “Parallel and distributed methods for constrained noncon- vex optimization-Part II: Applications in communications and machine learning,” IEEE Transactions on Signal Processing , vol. 65, no. 8, pp. 1945–1...

    1945

  7. [15]

    N. Z. Shor, Minimization methods for non-differentiable functions , vol. 3. Springer Science & Business Media, 2012

    2012

  8. [16]

    Complexity of problems an d effective- ness of methods of optimization(russian book),

    A. Nemirovskii and D. IUDIN, “Complexity of problems an d effective- ness of methods of optimization(russian book),” Moscow, Izdatel’stvo Nauka, 1979. 384 , 1979

    1979

  9. [17]

    Algorithms for stochastic optimiza tion with func- tion or expectation constraints,

    G. Lan and Z. Zhou, “Algorithms for stochastic optimiza tion with func- tion or expectation constraints,” Computational Optimization and Ap- plications, vol. 76, no. 2, pp. 461–498, 2020

    2020

  10. [18]

    Mirror descent and convex optimization problems with non- smooth inequality constraints,

    A. Bayandina, P. Dvurechensky, A. Gasnikov, F. Stonyak in, and A. Titov, “Mirror descent and convex optimization problems with non- smooth inequality constraints,” Large-scale and distributed optimiza- tion, pp. 181–213, 2018

    2018

  11. [19]

    A single-loop spider-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth o ptimiza- tion,

    W. Liu and Y. Xu, “A single-loop spider-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth o ptimiza- tion,” arXiv preprint arXiv:2501.19214 , 2025

    2025 arXiv

  12. [20]

    Crpo: A new approach for safe rein- forcement learning with convergence guarantee,

    T. Xu, Y. Liang, and G. Lan, “Crpo: A new approach for safe rein- forcement learning with convergence guarantee,” in International Con- ference on Machine Learning , pp. 11480–11491, PMLR, 2021

    2021

  13. [21]

    Stochastic model-based minimization of weakly convex functions,

    D. Davis and D. Drusvyatskiy, “Stochastic model-based minimization of weakly convex functions,” SIAM Journal on Optimization , vol. 29, no. 1, pp. 207–239, 2019. 21

    2019

  14. [22]

    Introduction to optimization. optimiza tion software,

    B. T. Polyak, “Introduction to optimization. optimiza tion software,” Inc., Publications Division, New York , vol. 1, no. 32, p. 1, 1987

    1987

  15. [23]

    Subgradient methods for huge-scale opti mization prob- lems,

    Y. Nesterov, “Subgradient methods for huge-scale opti mization prob- lems,” Mathematical Programming, vol. 146, no. 1, pp. 275–297, 2014

    2014

  16. [24]

    On modification of an adaptive stochasti c mirror de- scent algorithm for convex optimization problems with func tional con- straints,

    M. S. Alkousa, “On modification of an adaptive stochasti c mirror de- scent algorithm for convex optimization problems with func tional con- straints,” Computational Mathematics and Applications , pp. 47–63, 2020

    2020

  17. [25]

    Adap- tive mirror descent algorithms for convex and strongly conv ex opti- mization problems with functional constraints,

    F. S. Stonyakin, M. Alkousa, A. N. Stepanov, and A. A. Tit ov, “Adap- tive mirror descent algorithms for convex and strongly conv ex opti- mization problems with functional constraints,” Journal of Applied and Industrial Mathematics , vol. 13, no. 3, pp. 557–574, 2019

    2019

  18. [26]

    On some methods for strongly convex optimization problems with one functional constraint,

    F. S. Stonyakin, M. S. Alkousa, A. A. Titov, and V. V. Pisk unova, “On some methods for strongly convex optimization problems with one functional constraint,” in Mathematical Optimization Theory and Op- erations Research: 18th International Conference, MOTOR 2019 , Eka- terinb...

    2019

  19. [27]

    Mirror de- scent for constrained optimization problems with large sub gradient val- ues of functional constraints,

    F. Stonyakin, A. Stepanov, A. Gasnikov, A. Titov, et al. , “Mirror de- scent for constrained optimization problems with large sub gradient val- ues of functional constraints,” Computer research and modeling, vol. 12, no. 2, pp. 301–317, 2020

    2020

  20. [28]

    Analogues of switching subgradient schemes for relatively lipschitz-continuous convex programming problems,

    A. A. Titov, F. S. Stonyakin, M. S. Alkousa, S. S. Ablaev, and A. V. Gasnikov, “Analogues of switching subgradient schemes for relatively lipschitz-continuous convex programming problems,” in International Conference on Mathematical Optimization Theory and Operatio ns Re- sear...

    2020

  21. [29]

    Mirror descent and constrained online optimization probl ems,

    A. A. Titov, F. S. Stonyakin, A. V. Gasnikov, and M. S. Alk ousa, “Mirror descent and constrained online optimization probl ems,” in In- ternational Conference on Optimization and Applications , pp. 64–78, Springer, 2018

    2018

  22. [30]

    Stochastic first-and zeroth-ord er methods for nonconvex stochastic programming,

    S. Ghadimi and G. Lan, “Stochastic first-and zeroth-ord er methods for nonconvex stochastic programming,” SIAM journal on optimization , vol. 23, no. 4, pp. 2341–2368, 2013. 22

    2013

  23. [31]

    A method for solving the convex programmi ng problem with convergence rate o (1/k2),

    Y. Nesterov, “A method for solving the convex programmi ng problem with convergence rate o (1/k2),” in Dokl akad nauk Sssr , vol. 269, p. 543, 1983

    1983

  24. [32]

    Smooth minimization of non-smooth funct ions,

    Y. Nesterov, “Smooth minimization of non-smooth funct ions,” Mathe- matical programming, vol. 103, pp. 127–152, 2005

    2005

  25. [33]

    Random gradient-free min imization of convex functions,

    Y. Nesterov and V. Spokoiny, “Random gradient-free min imization of convex functions,” Foundations of Computational Mathematics , vol. 17, no. 2, pp. 527–566, 2017

    2017

  26. [34]

    O’Regan, Existence theory for nonlinear ordinary differential equa- tions, vol

    D. O’Regan, Existence theory for nonlinear ordinary differential equa- tions, vol. 398. Springer Science & Business Media, 1997

    1997

  27. [35]

    Optimization with mome ntum: Dy- namical, control-theoretic, and symplectic perspectives ,

    M. Muehlebach and M. I. Jordan, “Optimization with mome ntum: Dy- namical, control-theoretic, and symplectic perspectives ,” Journal of Machine Learning Research, vol. 22, no. 73, pp. 1–50, 2021

    2021

  28. [36]

    J. C. Butcher, Numerical methods for ordinary differential equations . John Wiley & Sons, 2016

    2016

  29. [37]

    Monotone operators and the proxima l point al- gorithm,

    R. T. Rockafellar, “Monotone operators and the proxima l point al- gorithm,” SIAM journal on control and optimization , vol. 14, no. 5, pp. 877–898, 1976

    1976

  30. [38]

    Two remarks on the method of suc cessive ap- proximations,

    M. A. Krasnosel’ski ˘ ı, “Two remarks on the method of suc cessive ap- proximations,” Uspekhi matematicheskikh nauk , vol. 10, no. 1, pp. 123– 127, 1955

    1955

  31. [39]

    Mean value methods in iteration,

    W. R. Mann, “Mean value methods in iteration,” Proceedings of the American Mathematical Society , vol. 4, no. 3, pp. 506–510, 1953

    1953

  32. [40]

    On the operations in abstract sets and their application to the equations,

    S. Banach, “On the operations in abstract sets and their application to the equations,” Fundamenta mathematicae, vol. 3, no. 1, pp. 133–181, 1922

    1922

  33. [41]

    Convergence rates w ith inexact non- expansive operators,

    J. Liang, J. Fadili, and G. Peyr´ e, “Convergence rates w ith inexact non- expansive operators,” Mathematical Programming, vol. 159, pp. 403– 434, 2016

    2016

  34. [42]

    A unified model for large-scale inexact fixe d-point iter- ation: A stochastic optimization perspective,

    A. Hashemi, “A unified model for large-scale inexact fixe d-point iter- ation: A stochastic optimization perspective,” IEEE Transactions on Automatic Control, 2024. 23

    2024

  35. [43]

    Stochastic fixed-point iter ations for non- expansive maps: Convergence and error bounds,

    M. Bravo and R. Cominetti, “Stochastic fixed-point iter ations for non- expansive maps: Convergence and error bounds,” SIAM Journal on Control and Optimization , vol. 62, no. 1, pp. 191–219, 2024

    2024

  36. [44]

    Stochastic quasi-f ej´ er block- coordinate fixed point iterations with random sweeping,

    P. L. Combettes and J.-C. Pesquet, “Stochastic quasi-f ej´ er block- coordinate fixed point iterations with random sweeping,” SIAM Journal on Optimization , vol. 25, no. 2, pp. 1221–1248, 2015

    2015

  37. [45]

    Stochastic (approximate) proxi mal point meth- ods: Convergence, optimality, and adaptivity,

    H. Asi and J. C. Duchi, “Stochastic (approximate) proxi mal point meth- ods: Convergence, optimality, and adaptivity,” SIAM Journal on Op- timization, vol. 29, no. 3, pp. 2257–2290, 2019

    2019

  38. [46]

    No- regret learning in dynamic stackelberg games,

    N. Lauffer, M. Ghasemi, A. Hashemi, Y. Savas, and U. Topcu, “No- regret learning in dynamic stackelberg games,” IEEE Transactions on Automatic Control, vol. 69, no. 3, pp. 1418–1431, 2023

    2023

  39. [47]

    First-order al gorithms for nonlinear generalized nash equilibrium problems,

    M. I. Jordan, T. Lin, and M. Zampetakis, “First-order al gorithms for nonlinear generalized nash equilibrium problems,” Journal of Machine Learning Research, vol. 24, no. 38, pp. 1–46, 2023

    2023

  40. [48]

    Independent learning in constrained markov potential games,

    P. Jordan, A. Barakat, and N. He, “Independent learning in constrained markov potential games,” in International Conference on Artificial In- telligence and Statistics , pp. 4024–4032, PMLR, 2024

    2024

  41. [49]

    Optimization, learning, and games with predictable sequences,

    S. Rakhlin and K. Sridharan, “Optimization, learning, and games with predictable sequences,” Advances in Neural Information Processing Systems, vol. 26, 2013

    2013

  42. [50]

    Optimistic regret bounds for online learn- ing in adversarial markov decision processes,

    S. B. Moon and A. Hashemi, “Optimistic regret bounds for online learn- ing in adversarial markov decision processes,” in The 40th Conference on Uncertainty in Artificial Intelligence , 2024

    2024

  43. [51]

    Distributed subgradient met hods for multi- agent optimization,

    A. Nedic and A. Ozdaglar, “Distributed subgradient met hods for multi- agent optimization,” IEEE Transactions on Automatic Control , vol. 54, no. 1, pp. 48–61, 2009

    2009

  44. [52]

    Momentum-based variance r eduction in non-convex sgd,

    A. Cutkosky and F. Orabona, “Momentum-based variance r eduction in non-convex sgd,” Advances in neural information processing systems , vol. 32, 2019

    2019

  45. [53]

    A h ybrid stochastic optimization framework for composite nonconve x optimiza- tion,

    Q. Tran-Dinh, N. H. Pham, D. T. Phan, and L. M. Nguyen, “A h ybrid stochastic optimization framework for composite nonconve x optimiza- tion,” Mathematical Programming, vol. 191, no. 2, pp. 1005–1071, 2022. 24

    2022

  46. [54]

    An optimal hybri d variance- reduced algorithm for stochastic composite nonconvex opti mization,

    D. Liu, L. M. Nguyen, and Q. Tran-Dinh, “An optimal hybri d variance- reduced algorithm for stochastic composite nonconvex opti mization,” arXiv preprint arXiv:2008.09055 , 2020

    2008 arXiv

  47. [55]

    Faster non-convex federated learning via global and lo- cal momentum,

    R. Das, A. Acharya, A. Hashemi, S. Sanghavi, I. S. Dhillo n, and U. Topcu, “Faster non-convex federated learning via global and lo- cal momentum,” in Uncertainty in Artificial Intelligence , pp. 496–506, PMLR, 2022. Appendix A Proof of Theorem 1 We first state a well-known resul...

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.