Pith. sign in

REVIEW 3 major objections 3 minor 59 references

Second-order methods for provably escaping strict saddle points in composite nonconvex and nonsmooth optimization

T0 review · 3 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces two second-order optimization methods that provably converge to second-order stationary points of composite nonsmooth objectives from any initialization, using the forward-backward envelope to transfer smooth…

desk verdict Genuinely new FBE-based second-order theory with a real scope gap: the unqualified 'regardless of initialization' claim is false because the stopping test can certify a weakly active strict saddle. read the letter →

arxiv 2506.22332 v1 pith:OQO3CPKL submitted 2025-06-27 math.OC

Alexander Bodard , Masoud Ahookhosh , Panagiotis Patrinos This is my paper · ORCID
classification math.OC MSC 90C2690C3049J5265K05
keywords compositeoptimizationforward-backwardenvelopestrictsaddlepointssecond-orderstationarytrust-regionmethodscurvilinearlinesearchpartialsmoothnessnonsmoothnonconvex
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles composite optimization problems of the form "minimize f(x) + g(x)" where f is smooth but possibly nonconvex and g is nonsmooth, and asks whether second-order methods can be designed that provably avoid strict saddle points—points satisfying first-order optimality but with negative curvature—no matter where they start. The authors establish that the forward-backward envelope, a smooth surrogate of the original objective, has second-order stationary points exactly matching those of the original nonsmooth problem, provided the nonsmooth term has a generalized quadratic second-order epi-derivative at the limit point. On this basis they propose two algorithms, a nonsmooth trust-region method and a curvilinear linesearch method, and prove both converge to second-order stationary points of the original composite problem, with local superlinear rates near strong minimizers. Because the guarantees hold regardless of initialization, the work promises to extend the saddle-escaping behavior of classical smooth second-order methods to a broad class of nonsmooth problems such as sparse PCA and phase retrieval.

What carries the argument

The load-bearing object is the forward-backward envelope $\varphi_\gamma(x) = \inf_u \{ f(x) + \langle \nabla f(x), u-x\rangle + g(u) + \tfrac{1}{2\gamma}\|u-x\|^2 \}$, which for small $\gamma$ is a real-valued strictly continuous surrogate with the same critical points, minimizers, and (by Theorem 3.6) second-order stationary points as $\varphi$. The second piece is the set-valued generalized Hessian $\hat{\partial}^2\varphi_\gamma(x) = \{ \gamma^{-1}Q(I-PQ) : P \in \partial_C \mathrm{prox}_{\gamma g}(x-\gamma\nabla f(x)),\ Q \in I-\gamma\partial_C\nabla f(x) \}$, a Gauss-Newton approximant that is single-valued and equal to $\nabla^2\varphi_\gamma$ near critical points under the stated assumptions, but never requires third derivatives of $f$. The proof machinery combines the equivalence between twice epi-differentiability of $g$ and differentiability of $\mathrm{prox}_{\gamma g}$, a tilting argument, and classical trust-region and negative-curvature conditions.

What would settle it

Minimize a nonconvex quadratic over the nonnegative orthant from an initialization whose limit point has a weakly active constraint (zero gradient component at an active bound), so strict complementarity fails, and check whether Algorithm 1 either terminates at a non-second-order point or stalls with $\lambda_{\min}(B_k)$ staying negative—either outcome would violate the local smoothness hypothesis of Theorem 4.7.

Watch

Extended reading notes

Core claim

The central claim is that second-order stationarity of the nonsmooth composite $\varphi$ and of its forward-backward envelope $\varphi_\gamma$ coincide: Theorem 3.6 shows that for a critical point $x^\star$ where $g$ is twice epi-differentiable with a generalized quadratic second-order epi-derivative, $d^2\varphi(x^\star,0)[s] \ge 0$ for all $s$ if and only if $\nabla^2\varphi_\gamma(x^\star) \succeq 0$. This transfers the well-developed machinery of smooth trust-region and curvilinear methods to nonsmooth problems, since iterating on the FBE with a Gauss-Newton-type generalized Hessian avoids third-order derivatives while agreeing with the true Hessian at critical points. Under Assumptions 5, 6 and a local smoothness assumption at the limit, Algorithm 1 converges to a second-order stationary point of the original problem (Theorem 4.7), and Algorithm 2 achieves the same under weaker global assumptions, without level-boundedness (Theorem 5.3, Corollary 5.4). For $\mathcal{C}^2$-partly smooth $g$, the needed proximal differentiability follows from strict complementarity (Theorem 3.4), covering $\ell_1$, nuclear norm, total variation, and indicators of polyhedral and conic constraint sets.

Load-bearing premise

The guarantee needs the nonsmooth term at the meeting point to have a very specific second-order structure (a "generalized quadratic" epi-derivative), which the algorithms never check; for partly smooth functions this reduces to strict complementarity, meaning no weakly active constraints.

Editorial extensions

If this is right

  • For any composite problem whose nonsmooth term satisfies the epi-differentiability condition at the limit, both algorithms provably reach points that are second-order stationary for the original objective, not merely first-order critical points.
  • Algorithm 1 achieves this under the standard trust-region hypotheses (level-boundedness plus bounded model Hessians), while Algorithm 2 removes the level-boundedness assumption entirely.
  • For $\mathcal{C}^2$-partly smooth regularizers—$\ell_1$, $\ell_{1,2}$, $\ell_\infty$, nuclear norm, total variation, $\ell_0$, and indicators of polyhedral or conic sets—strict complementarity suffices for all the needed local smoothness, so the guarantees cover the most widely used nonsmooth penalties.
  • Near strong local minimizers with a positive-definite envelope Hessian, both methods eventually take full Newton steps and converge superlinearly, and quadratically whenever $\nabla^2\varphi_\gamma$ is locally Lipschitz.
  • The numerical comparisons show the methods match or outperform an L-BFGS accelerated proximal gradient baseline on sparse PCA and phase retrieval, with Algorithm 2 converging to better stationary points on phase retrieval when its negative-curvature scaling is tuned.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence not spelled out in the paper: on problems with benign landscapes, the initialization-independent guarantee suggests these methods should find global minimizers from any random start whenever every strict saddle is genuinely nonsmooth, extending the classical smooth strict-saddle story to regularized problems.
  • The equivalence in Theorem 3.6 points to a general recipe—any smooth surrogate whose second-order stationary points match those of the original nonsmooth problem inherits classical second-order algorithms; constructing other envelopes with the same matching property could yield similar initialization-independent guarantees.
  • The strict complementarity requirement could in principle be bypassed by an active-set identification phase: once the active manifold is discovered, one could run a constrained second-order method on the smooth restriction, avoiding weakly active constraints altogether.
  • Empirically, the sensitivity of Algorithm 2's solution quality to the negative-curvature scaling parameter suggests a practical extension: adaptively tune the scaling by a trust-region-like acceptance rule rather than fixing it in advance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper studies second-order methods for the composite nonsmooth problem (P): minimize φ = f + g with f smooth and g nonsmooth. It analyzes the forward-backward envelope (FBE) φγ, proves conditions under which φγ is locally C2 around critical points, and establishes an equivalence between second-order stationary points of φ and of φγ. Based on this, it proposes a nonsmooth trust-region method (Algorithm 1) and a curvilinear linesearch proximal gradient method (Algorithm 2), and proves global convergence, convergence to second-order stationary points under additional assumptions, and local superlinear rates. The advertised headline is that these are the first second-order methods that provably escape nonsmooth strict saddle points regardless of initialization.

Significance. The conditional results are significant. The local C2 analysis of the FBE around critical points, the generalized Hessian construction in (5), and the equivalence theorem (Theorem 3.6) are valuable extensions of the existing FBE theory, and the algorithmic framework is novel for composite nonsmooth problems. The proofs are detailed and the paper is transparent about many assumptions, such as the strict complementarity condition (SC) for C2-partly smooth functions. The numerical experiments illustrate the intended saddle-escaping behavior. However, the advertised claim of escaping nonsmooth strict saddle points 'regardless of initialization' is stronger than what the theorems actually establish, and the stopping criteria can certify a strict saddle point as a solution. The central contribution is defensible only after the claims and stopping conditions are qualified with the local smoothness/SC assumptions.

major comments (3)
  1. [Abstract, Section 1.1, Algorithm 1 line 4, Algorithm 2 line 5] The headline claim that the methods provably escape nonsmooth strict saddle points 'regardless of initialization' is false as stated. Consider f(x,y) = -(x^2+y^2)/2 and g = δ_{R_+^2}, with x0 = (0,0). Then Tγ(0) = 0, so r0 = 0, and P0 = 0 is a valid selection from ∂C(prox_{γg})(0), because the Clarke Jacobian at 0 contains all diagonal matrices diag(p1,p2) with p_i ∈ [0,1]. With Q0 = (1+γ)I, one obtains B0 = γ^{-1}(1+γ)I ≻ 0, so both Algorithm 1 and Algorithm 2 terminate at x* = 0. Yet 0 ∈ ∂φ(0) and d^2φ(0,0)[d] = -||d||^2 for d ∈ R_+^2, so x* is a nonsmooth strict saddle point under Definition 2.3(ii). The formal theorems are conditional on Assumption 4 at the limit point (equivalently SC for C2-partly smooth g), but the abstract, the contributions section, and the stopping tests do not carry this qualification, and no algorithmic mechanism verifies it. Please re-scope the central claim and either modify the stopping criteria or state explicitly that termination is only certified under Assumption 4/SC.
  2. [Theorem 4.7 and Section 4.2] Theorem 4.5 only establishes lim_k ∥∇φγ(xk)∥ = 0; combined with Fact 2 and nonsingularity of Qγ, this gives Rγ(xk) → 0 and hence subsequential criticality of bounded iterates. Theorem 4.7, however, assumes that the full sequence of iterates converges to a critical point x⋆ satisfying Assumption 4, an assumption that is never established. The paper should state the second-order convergence result subsequentially, i.e., for every limit point of the iterates that satisfies Assumption 4, or prove full-sequence convergence. As written, the gap between Theorem 4.5 and Theorem 4.7 is load-bearing for the claimed convergence to second-order stationary points.
  3. [Theorem 3.6, Assumption 3, Corollary 5.4] The equivalence between second-order stationary points of φ and φγ (Theorem 3.6) requires Assumption 3(i) at the critical point, which for C2-partly smooth g is guaranteed by SC. This condition is not preserved by the algorithms and can fail at a limit point, as the example in the first major comment shows; there, prox_{γg} is not C1 at the forward point, so Assumption 4(ii) fails and the equivalence does not apply. Consequently, the conclusion of Corollary 5.4 that 'the limit points of (x̄k) are second-order stationary points of (P)' holds only under an unverified structural condition on the limit point. The paper should either add an algorithmic mechanism to certify this condition at termination or state all second-order stationarity results as conditional on Assumption 3(i)/Assumption 4 holding at the limit point.
minor comments (3)
  1. [Section 4.2, Theorem 4.7 proof] In the proof of Theorem 4.7, inequality (10) uses boundedness of ∇3f near x⋆; this is guaranteed by Assumption 4(i), but it would help to state explicitly that the constant is uniform on the neighborhood provided by Theorem 3.2.
  2. [Section 5.2, Theorem 5.3 and Corollary 5.4] Theorem 5.3 assumes boundedness of the direction sequences but not boundedness of the iterates; Proposition 5.2 only yields criticality of limit points when the iterates are bounded. Without a boundedness or coercivity assumption, Corollary 5.4 may be vacuously true if no limit points exist, which is weaker than the announced convergence. Please state the boundedness assumption explicitly.
  3. [Throughout] There are several minor typographical issues, e.g., 'Suppose that that the iterates' in Theorem 4.9, and the notation 'N[1,N]' in Section 6.3 is nonstandard; please clean these up.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the FBE second-order theory and algorithm convergence are proven from epi-differentiability and variational-analysis assumptions rather than assumed into existence; self-citations are published technical lemmas, not target-assuming inputs.

full rationale

The paper's derivation chain is not circular. Lemma 3.1 establishes the equivalence between second-order epi-differentiability of g (Assumption 3) and differentiability of prox_gamma_g by tilting to a global minimizer and invoking Poliquin-Rockafellar, which are external results. Theorem 3.2 then differentiates phi_gamma via the product rule applied to Nabla phi_gamma = Q_gamma R_gamma, and Theorem 3.6 proves the SOSP equivalence by writing d^2g as a generalized quadratic and reducing the PSD condition on Nabla^2 phi_gamma to Pi_S(M + Nabla^2 f)Pi_S >= 0 through a Schur complement argument. The formulas P_gamma = Pi_S(I+gamma M)^{-1}Pi_S and positive definiteness of I+gamma M are quoted from the same group's published papers [49,55], but they are parameter-free lemmas whose assumptions do not include the target SOSP equivalence, so under the review rules they count as real evidence rather than circular self-citation. The generalized Hessian (5) is explicitly attributed to [53] and used as an algorithmic model; convergence is then obtained by standard trust-region and negative-curvature arguments. No fitted constants or data-dependent parameters are renamed as predictions. The main caveat is that Theorem 4.7 and Corollary 5.4 assume Assumption 4 holds at the limit point, which the algorithms do not verify; the abstract's 'regardless of initialization' is therefore broader than the theorems. This is a scope/correctness concern, not a circular one, because the assumptions do not define the conclusions.

Assumptions & free parameters 1 free parameters · 8 assumptions · 0 invented entities

The paper is a theory paper. The main assumptions are explicit domain assumptions on f and g; no data-fitted constants appear in the theory. The core burden is Assumption 3(i) plus the strict complementarity condition (SC), which are not guaranteed by the algorithms. The only hand-chosen numerical parameter is the scaling s-bar in Algorithm 2.

free parameters (1)
  • s-bar (negative curvature direction scaling) = 1 (default; varied as 1e-2 and 1e-4 in experiments)
    Hand-chosen scaling factor for negative curvature directions in Algorithm 2; the paper states it significantly impacts solution quality in phase retrieval (Table 2).
assumptions (8)
  • domain assumption Assumption 1: f is C1,1 with Lf-Lipschitz gradient; g is proper, lsc, and gamma-g-prox-bounded; arg min phi is nonempty
    Defines the problem class in (P) and ensures the FBE and proximal mappings are well-behaved.
  • domain assumption Assumption 2: g is rho-weakly convex
    Ensures the Clarke Jacobian of prox-gamma-g is nonempty and bounded along iterates, needed for the generalized Hessian (5).
  • domain assumption Assumption 3(i): g is twice epi-differentiable at the critical point with generalized quadratic second-order epi-derivative
    Load-bearing for Theorem 3.6, the equivalence of SOSP of phi and phi-gamma.
  • domain assumption Assumption 4: f is C3 around the critical point and prox-gamma-g is C1 around the forward point
    Needed for the local C2 property of phi-gamma (Theorem 3.2) and for Hessian approximation error bounds in Theorems 4.7 and 5.3.
  • domain assumption Assumption 5: f is C2+ plus Assumptions 1 and 2
    Standard trust-region assumption ensuring phi-gamma is C1+ globally.
  • domain assumption Assumption 6: phi has bounded sublevel sets
    Used for global convergence of Algorithm 1; not needed for Algorithm 2.
  • domain assumption Strict complementarity (SC): -grad f(x-star) in relint dg(x-star) for C2-partly smooth g
    Sufficient condition for prox-gamma-g to be C1 around the forward point (Theorem 3.4).
  • standard math Rockafellar-Wets variational analysis facts (prox-regularity, partial smoothness, subderivatives)
    Standard machinery used throughout the paper without proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Second-order methods for provably escaping strict saddle points in composite nonconvex and nonsmooth optimization." pith.science (2026). https://pith.science/paper/OQO3CPKL

@misc{pith2026250622332,
  author       = {Pith},
  title        = {Pith review of: Second-order methods for provably escaping strict saddle points in composite nonconvex and nonsmooth optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OQO3CPKL}},
  note         = {Machine review of arXiv:2506.22332}
}
abstract

This study introduces two second-order methods designed to provably avoid saddle points in composite nonconvex optimization problems: (i) a nonsmooth trust-region method and (ii) a curvilinear linesearch method. These developments are grounded in the forward-backward envelope (FBE), for which we analyze the local second-order differentiability around critical points and establish a novel equivalence between its second-order stationary points and those of the original objective. We show that the proposed algorithms converge to second-order stationary points of the FBE under a mild local smoothness condition on the proximal mapping of the nonsmooth term. Notably, for \( \C^2 \)-partly smooth functions, this condition holds under a standard strict complementarity assumption. To the best of our knowledge, these are the first second-order algorithms that provably escape nonsmooth strict saddle points of composite nonconvex optimization, regardless of the initialization. Our preliminary numerical experiments show promising performance of the developed methods, validating our theoretical foundations.

Figures

Figures reproduced from arXiv: 2506.22332 by the authors.

Figure 1
Figure 1. Crosses indicate minimizers, dots correspond to saddle points. The objective [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. The function g(x, y) = |x| + 1 2 y 2 (left) is C 2 -partly smooth at the origin relative to M := {(0, y) | y ∈ R}. Around (0, 0), the restriction of g to M is C 2 smooth (middle), and g behaves “sharply” along normal directions w ∈ NM(0, 0) = {(wx, 0) | wx ∈ R} (right). 2.4 Partial smoothness The class of C 2 -partly smooth functions was initially introduced in [29] and further analyzed in [36, 26, 24, 16, 11, 30, 2… view at source ↗
Figure 3
Figure 3. Iterates of the PGM and Algorithms 1 and 2 for the problems from Examples 2.4 and 2.5, along with the level-curves of the corresponding objectives. 6.2 Sparse principal component analysis Let us consider the sparse principal component analysis (PCA) problem described in [27, §2.1], i.e., minimize x∈Rn − 1 2 x ⊤Σx + κ∥x∥1 + δB¯(0;1)(x), with sample covariance matrix Σ = A⊤A, and sparsity inducing parameter κ ≥ 0. Thi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 57 canonical work pages

  1. [1]

    Trust-Region Methods on Riemannian Mani- folds

    P.-A. Absil, C. Baker, and K. Gallivan. “Trust-Region Methods on Riemannian Mani- folds”. In: Foundations of Computational Mathematics 7.3 (July 2007), pp. 303–330

    work page 2007

  2. [2]

    A Proximal Quasi-Newton Trust-Region Method for Nonsmooth Regularized Optimization

    A. Y. Aravkin, R. Baraldi, and D. Orban. “A Proximal Quasi-Newton Trust-Region Method for Nonsmooth Regularized Optimization”. In: SIAM Journal on Optimization 32.2 (June 2022), pp. 900–929. 34

    work page 2022

  3. [3]

    A proximal trust-region method for nonsmooth opti- mization with inexact function and gradient evaluations

    R. J. Baraldi and D. P. Kouri. “A proximal trust-region method for nonsmooth opti- mization with inexact function and gradient evaluations”. In: Mathematical Program- ming 201.1 (Sept. 2023), pp. 559–598

    work page 2023

  4. [4]

    A Fast Iterative Shrinkage-Thresholding Algorithm for Lin- ear Inverse Problems

    A. Beck and M. Teboulle. “A Fast Iterative Shrinkage-Thresholding Algorithm for Lin- ear Inverse Problems”. In: SIAM Journal on Imaging Sciences 2.1 (Jan. 2009), pp. 183– 202

    work page 2009

  5. [5]

    Global Optimality of Local Search for Low Rank Matrix Recovery

    S. Bhojanapalli, B. Neyshabur, and N. Srebro. “Global Optimality of Local Search for Low Rank Matrix Recovery”. In: Advances in Neural Information Processing Systems . Vol. 29. 2016

    work page 2016

  6. [6]

    PANTR: A Proximal Algorithm With Trust- Region Updates for Nonconvex Constrained Optimization

    A. Bodard, P. Pas, and P. Patrinos. “PANTR: A Proximal Algorithm With Trust- Region Updates for Nonconvex Constrained Optimization”. In: IEEE Control Systems Letters 7 (2023), pp. 2389–2394

    work page 2023

  7. [7]

    Proximal alternating linearized minimization for nonconvex and nonsmooth problems

    J. Bolte, S. Sabach, and M. Teboulle. “Proximal alternating linearized minimization for nonconvex and nonsmooth problems”. In: Mathematical Programming 146.1 (Aug. 2014), pp. 459–494

    work page 2014

  8. [8]

    Bonnans and A

    J. Bonnans and A. Shapiro. Perturbation Analysis of Optimization Problems. Jan. 2000

    work page 2000

Show all 59 references
  1. [9]

    Gradient Descent Provably Escapes Saddle Points in the Training of Shallow ReLU Networks

    P. Cheridito, A. Jentzen, and F. Rossmannek. “Gradient Descent Provably Escapes Saddle Points in the Training of Shallow ReLU Networks”. In: Journal of Optimization Theory and Applications (Sept. 2024)

    2024

  2. [10]

    A. R. Conn, N. I. M. Gould, and P. L. Toint. Trust Region Methods. MOS-SIAM Series on Optimization. Society for Industrial and Applied Mathematics, Jan. 2000

    2000

  3. [11]

    Orthogonal Invariance and Identi- fiability

    A. Daniilidis, D. Drusvyatskiy, and A. S. Lewis. “Orthogonal Invariance and Identi- fiability”. In: SIAM Journal on Matrix Analysis and Applications 35.2 (Jan. 2014), pp. 580–598

    2014

  4. [12]

    Geometrical interpretation of the predictor- corrector type algorithms in structured optimization problems

    A. Daniilidis, W. Hare, and J. Malick. “Geometrical interpretation of the predictor- corrector type algorithms in structured optimization problems”. In: Optimization 55.5-6 (Oct. 2006), pp. 481–503

    2006

  5. [13]

    Escaping Strict Saddle Points of the Moreau Envelope in Nonsmooth Optimization

    D. Davis, M. D ´ ıaz, and D. Drusvyatskiy. “Escaping Strict Saddle Points of the Moreau Envelope in Nonsmooth Optimization”. In: SIAM Journal on Optimization 32.3 (Sept. 2022), pp. 1958–1983

    2022

  6. [14]

    Proximal Methods Avoid Active Strict Saddles of Weakly Convex Functions

    D. Davis and D. Drusvyatskiy. “Proximal Methods Avoid Active Strict Saddles of Weakly Convex Functions”. In: Foundations of Computational Mathematics 22.2 (Apr. 2022), pp. 561–606

    2022

  7. [15]

    Proximal Gradient Algorithms Under Local Lipschitz Gradient Continuity

    A. De Marchi and A. Themelis. “Proximal Gradient Algorithms Under Local Lipschitz Gradient Continuity”. In:Journal of Optimization Theory and Applications 194.3 (Sept. 2022), pp. 771–794

    2022

  8. [16]

    Optimality, identifiability, and sensitivity

    D. Drusvyatskiy and A. S. Lewis. “Optimality, identifiability, and sensitivity”. In: Math- ematical Programming 147.1-2 (Oct. 2014), pp. 467–498

    2014

  9. [17]

    Nonmonotone curvilinear line search methods for unconstrained optimization

    M. C. Ferris, S. Lucid, and M. Roma. “Nonmonotone curvilinear line search methods for unconstrained optimization”. In: Computational Optimization and Applications 6.2 (Sept. 1996), pp. 117–136. 35

    1996

  10. [18]

    No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis

    R. Ge, C. Jin, and Y. Zheng. “No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis”. In: Proceedings of the 34th International Conference on Machine Learning . PMLR, July 2017, pp. 1233–1242

    2017

  11. [19]

    Matrix Completion has No Spurious Local Minimum

    R. Ge, J. D. Lee, and T. Ma. “Matrix Completion has No Spurious Local Minimum”. In: Advances in Neural Information Processing Systems . Vol. 29. 2016

    2016

  12. [20]

    Exploiting negative curva- ture directions in linesearch methods for unconstrained optimization

    N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint. “Exploiting negative curva- ture directions in linesearch methods for unconstrained optimization”. In: Optimization Methods and Software 14.1-2 (Jan. 2000), pp. 75–98

    2000

  13. [21]

    Riemannian trust-region methods for strict saddle func- tions with complexity guarantees

    F. Goyens and C. W. Royer. “Riemannian trust-region methods for strict saddle func- tions with complexity guarantees”. In: Mathematical Programming (Nov. 2024)

    2024

  14. [22]

    N. T. V. Hang and E. Sarabi. A fresh look into variational analysis of $\mathcal Cˆ2$- partly smooth functions . arXiv:2411.00655. Nov. 2024

    2024

  15. [23]

    N. T. V. Hang and E. Sarabi. Smoothness of Subgradient Mappings and Its Applications in Parametric Optimization . arXiv:2311.06026 [math]. Nov. 2023

    2023 arXiv

  16. [24]

    Identifying Active Constraints via Partial Smoothness and Prox-Regularity

    W. L. Hare and A. S. Lewis. “Identifying Active Constraints via Partial Smoothness and Prox-Regularity”. In: Journal of Convex Analysis 11.2 (2004), pp. 251–266

    2004

  17. [25]

    Functions and Sets of Smooth Substructure: Relationships and Examples

    W. L. Hare. “Functions and Sets of Smooth Substructure: Relationships and Examples”. In: Computational Optimization and Applications 33.2-3 (Mar. 2006), pp. 249–270

    2006

  18. [26]

    Nonsmooth Optimization with Smooth Substructure

    W. L. Hare. “Nonsmooth Optimization with Smooth Substructure”. PhD thesis. Simon Fraser University, 2004

    2004

  19. [27]

    Generalized Power Method for Sparse Principal Component Analysis

    M. Journ´ ee, Y. Nesterov, P. Richt´ arik, and R. Sepulchre. “Generalized Power Method for Sparse Principal Component Analysis”. In: Journal of Machine Learning Research 11.15 (2010), pp. 517–553

    2010

  20. [28]

    First- order methods almost always avoid strict saddle points

    J. D. Lee, I. Panageas, G. Piliouras, M. Simchowitz, M. I. Jordan, and B. Recht. “First- order methods almost always avoid strict saddle points”. In:Mathematical Programming 176.1 (July 2019), pp. 311–337

    2019

  21. [29]

    Active Sets, Nonsmoothness, and Sensitivity

    A. S. Lewis. “Active Sets, Nonsmoothness, and Sensitivity”. In: SIAM Journal on Op- timization 13.3 (Jan. 2002), pp. 702–725

    2002

  22. [30]

    Partial Smoothness, Tilt Stability, and Generalized Hes- sians

    A. S. Lewis and S. Zhang. “Partial Smoothness, Tilt Stability, and Generalized Hes- sians”. In: SIAM Journal on Optimization 23.1 (Jan. 2013), pp. 74–94

    2013

  23. [31]

    Convergence Rates of First-Order Operator Splitting Methods

    J. Liang. “Convergence Rates of First-Order Operator Splitting Methods”. PhD thesis. Oct. 2016

    2016

  24. [32]

    An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

    R. Liu, S. Pan, Y. Wu, and X. Yang. “An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization”. In: Computational Optimization and Ap- plications 88.2 (June 2024), pp. 603–641

    2024

  25. [33]

    An Envelope for Davis–Yin Splitting and Strict Saddle-Point Avoidance

    Y. Liu and W. Yin. “An Envelope for Davis–Yin Splitting and Strict Saddle-Point Avoidance”. In: Journal of Optimization Theory and Applications 181.2 (May 2019), pp. 567–587

    2019

  26. [34]

    Curvilinear Stabilization Techniques for Trun- cated Newton Methods in Large Scale Unconstrained Optimization

    S. Lucidi, F. Rochetich, and M. Roma. “Curvilinear Stabilization Techniques for Trun- cated Newton Methods in Large Scale Unconstrained Optimization”. In: SIAM Journal on Optimization 8.4 (Nov. 1998), pp. 916–939. 36

    1998

  27. [35]

    A modification of Armijo’s step-size rule for negative curvature

    G. P. McCormick. “A modification of Armijo’s step-size rule for negative curvature”. In: Mathematical Programming 13.1 (Dec. 1977), pp. 111–115

    1977

  28. [36]

    Primal-Dual Gradient Structured Functions: Second- Order Results; Links to Epi-Derivatives and Partly Smooth Functions

    R. Mifflin and C. Sagastiz´ abal. “Primal-Dual Gradient Structured Functions: Second- Order Results; Links to Epi-Derivatives and Partly Smooth Functions”. In:SIAM Jour- nal on Optimization 13.4 (Jan. 2003), pp. 1174–1194

    2003

  29. [37]

    On the use of directions of negative curvature in a modified newton method

    J. J. Mor´ e and D. C. Sorensen. “On the use of directions of negative curvature in a modified newton method”. In: Mathematical Programming 16.1 (Dec. 1979), pp. 1–20

    1979

  30. [38]

    Nocedal and S

    J. Nocedal and S. J. Wright. Numerical optimization. 2nd ed. Springer series in opera- tions research. New York: Springer, 2006

    2006

  31. [39]

    A trust region-type normal map-based semismooth New- ton method for nonsmooth nonconvex composite optimization

    W. Ouyang and A. Milzarek. “A trust region-type normal map-based semismooth New- ton method for nonsmooth nonconvex composite optimization”. In: Mathematical Pro- gramming (July 2024)

    2024

  32. [40]

    Proximal Newton methods for convex composite op- timization

    P. Patrinos and A. Bemporad. “Proximal Newton methods for convex composite op- timization”. In: 52nd IEEE Conference on Decision and Control . Firenze: IEEE, Dec. 2013, pp. 2358–2363

    2013

  33. [41]

    Generalized Hessian Properties of Regularized Nonsmooth Functions

    R. A. Poliquin and R. T. Rockafellar. “Generalized Hessian Properties of Regularized Nonsmooth Functions”. In: SIAM Journal on Optimization 6.4 (Nov. 1996), pp. 1121– 1137

    1996

  34. [42]

    Prox-regular functions in variational analysis

    R. A. Poliquin and R. T. Rockafellar. “Prox-regular functions in variational analysis”. In: Transactions of the American Mathematical Society 348.5 (1996), pp. 1805–1838

    1996

  35. [43]

    First- and Second-Order Epi-Differentiability in Nonlinear Program- ming

    R. T. Rockafellar. “First- and Second-Order Epi-Differentiability in Nonlinear Program- ming”. In: Transactions of the American Mathematical Society 307.1 (1988), pp. 75– 108

    1988

  36. [44]

    R. T. Rockafellar and R. J. B. Wets. Variational Analysis . Ed. by M. Berger et al. Vol. 317. Grundlehren der mathematischen Wissenschaften. Berlin, Heidelberg: Springer, 1998

    1998

  37. [45]

    On a Class of Nonsmooth Composite Functions

    A. Shapiro. “On a Class of Nonsmooth Composite Functions”. In: Mathematics of Op- erations Research 28.4 (Nov. 2003), pp. 677–692

    2003

  38. [46]

    A Family of Trust-Region-Based Algo- rithms for Unconstrained Minimization with Strong Global Convergence Properties

    G. A. Shultz, R. B. Schnabel, and R. H. Byrd. “A Family of Trust-Region-Based Algo- rithms for Unconstrained Minimization with Strong Global Convergence Properties”. In: SIAM Journal on Numerical Analysis 22.1 (1985), pp. 47–67

    1985

  39. [47]

    Theoretical Insights Into the Opti- mization Landscape of Over-Parameterized Shallow Neural Networks

    M. Soltanolkotabi, A. Javanmard, and J. D. Lee. “Theoretical Insights Into the Opti- mization Landscape of Over-Parameterized Shallow Neural Networks”. In:IEEE Trans- actions on Information Theory 65.2 (Feb. 2019), pp. 742–769

    2019

  40. [48]

    The Conjugate Gradient Method and Trust Regions in Large Scale Op- timization

    T. Steihaug. “The Conjugate Gradient Method and Trust Regions in Large Scale Op- timization”. In: SIAM Journal on Numerical Analysis 20.3 (June 1983), pp. 626–637

    1983

  41. [49]

    Forward–backward quasi-Newton methods for nonsmooth optimization problems

    L. Stella, A. Themelis, and P. Patrinos. “Forward–backward quasi-Newton methods for nonsmooth optimization problems”. In: Computational Optimization and Applications 67.3 (July 2017), pp. 443–487

    2017

  42. [50]

    A simple and efficient algorithm for nonlinear model predictive control

    L. Stella, A. Themelis, P. Sopasakis, and P. Patrinos. “A simple and efficient algorithm for nonlinear model predictive control”. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC) . Dec. 2017, pp. 1939–1944. 37

    2017

  43. [51]

    A Geometric Analysis of Phase Retrieval

    J. Sun, Q. Qu, and J. Wright. “A Geometric Analysis of Phase Retrieval”. In: Founda- tions of Computational Mathematics 18.5 (Oct. 2018), pp. 1131–1198

    2018

  44. [52]

    J. Sun, Q. Qu, and J. Wright. When Are Nonconvex Problems Not Scary? arXiv:1510.06096. Apr. 2016

    2016 arXiv

  45. [53]

    On the Acceleration of Forward-Backward Splitting via an Inexact Newton Method

    A. Themelis, M. Ahookhosh, and P. Patrinos. “On the Acceleration of Forward-Backward Splitting via an Inexact Newton Method”. In: Splitting Algorithms, Modern Operator Theory, and Applications. Cham: Springer International Publishing, 2019, pp. 363–412

    2019

  46. [54]

    A new envelope function for nonsmooth DC optimization

    A. Themelis, B. Hermans, and P. Patrinos. “A new envelope function for nonsmooth DC optimization”. In: 2020 59th IEEE Conference on Decision and Control (CDC) . ISSN: 2576-2370. Dec. 2020, pp. 4697–4702

    2020

  47. [55]

    Forward-Backward Envelope for the Sum of Two Nonconvex Functions: Further Properties and Nonmonotone Linesearch Algo- rithms

    A. Themelis, L. Stella, and P. Patrinos. “Forward-Backward Envelope for the Sum of Two Nonconvex Functions: Further Properties and Nonmonotone Linesearch Algo- rithms”. In: SIAM Journal on Optimization 28.3 (Jan. 2018), pp. 2274–2303

    2018

  48. [56]

    Linear Regularizers Enforce the Strict Saddle Prop- erty

    M. Ubl, M. Hale, and K. Yazdani. “Linear Regularizers Enforce the Strict Saddle Prop- erty”. In: Proceedings of the AAAI Conference on Artificial Intelligence 37.8 (June 2023). Number: 8, pp. 10017–10024

    2023

  49. [57]

    Zhang, ed

    F. Zhang, ed. The Schur Complement and Its Applications . Vol. 4. Numerical Methods and Algorithms. New York: Springer-Verlag, 2005

    2005

  50. [58]

    Proximal gradient algorithm with trust region scheme on Riemannian manifold

    S. Zhao, T. Yan, and Y. Zhu. “Proximal gradient algorithm with trust region scheme on Riemannian manifold”. In: Journal of Global Optimization 88.4 (Apr. 2024), pp. 1051– 1076

    2024

  51. [59]

    Zheng, C.-F

    Y. Zheng, C.-F. Pai, and Y. Tang. Benign Nonconvex Landscapes in Optimal and Robust Control, Part II: Extended Convex Lifting. arXiv:2406.04001 [cs, eess, math]. June 2024. 38

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.