Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

Conformal Symplectic Optimization for Stable Reinforcement Learning

T0 review · 4 major / 3 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RAD derives a stable RL optimizer from relativistic multi-particle dynamics, proving sublinear convergence, and contains ADAM as a limiting case.

desk verdict A promising RL optimizer with a broad empirical study, but the main convergence theorem does not actually cover the recommended hyperparameters used in the experiments. read the letter →

arxiv 2412.02291 v2 pith:UG7FKCUQ submitted 2024-12-03 cs.LG cs.AI

Yao Lyu , Xiangteng Zhang , Shengbo Eben Li , Jingliang Duan , Letian Tao , Qing Xu , Lei He , Keqiang Li This is my paper · ORCID
classification cs.LGcs.AI
keywords conformalHamiltoniansystemsymplecticintegratorrelativisticgradientdescentadaptiveoptimizationreinforcementlearningnonconvexstochastictrainingstabilityADAM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes RAD, an optimizer for deep reinforcement learning that treats each trainable weight as a tiny relativistic particle moving under a damped Hamiltonian force. By discretizing this conformal Hamiltonian system with a symplectic integrator, RAD caps how fast any parameter can move per step, which the authors argue suppresses the wild gradient spikes that destabilize RL training. The paper proves that RAD converges sublinearly to a stationary point for general nonconvex stochastic objectives, provided each gradient coordinate stays bounded, and that smaller gradient variance and larger batches tighten the bound. It also shows that with speed coefficient one and a fixed small symplectic factor, RAD reduces exactly to ADAM, making ADAM a special case of the relativistic update. Across five RL algorithms and twelve environments, RAD reports consistently higher policy returns than nine baselines, including up to 155.1% improvement over ADAM on Seaquest-v4.

What carries the argument

The central object is the conformal symplectic integrator applied to a multi-particle relativistic Hamiltonian. A conformal Hamiltonian system adds damping $-rp$ to the canonical equations, so phase-space area contracts as $e^{-rt}$; a conformal symplectic integrator preserves this contraction after discretization. The paper's update rule (Eq. 13) is the integrator's discrete map, with the symplectic factor $\zeta_k = \min\{\epsilon_k, 1-\beta_2^{k+1}\}$ controlling how much of the relativistic normalization survives: it starts tiny, mimicking ADAM, then anneals upward to restore the dissipative symplectic flow. The speed coefficient $\delta$ sets the per-step cap $|\theta_{k+1,i}-\theta_{k,i}| \leq \alpha/\delta$, which is the relativistic speed limit that suppresses abnormal gradients.

What would settle it

Track the maximum per-coordinate gradient norm across a long RL run (e.g., Seaquest-v4) and check whether it stays below the paper's M while the average squared gradient norm decays as O(1/N). If gradients exceed any finite M but the observed decay still matches the bound, the proof's assumption is unnecessary; if the decay stops precisely when gradients spike, the assumption is load-bearing. Alternatively, run RAD on a synthetic nonconvex problem with heavy-tailed noise and compare its empirical convergence against the O(1/N) prediction.

Watch

Extended reading notes

Core claim

RAD's central claim is that long-term training stability in NN optimization can be engineered by constructing the optimizer as a conformal symplectic integrator of a relativistic multi-particle Hamiltonian. Each parameter $\theta_i$ is an independent one-dimensional particle with momentum $p_i$, and the Hamiltonian is $H = \sum_i c\sqrt{p_i^2 + m^2 c^2} + J(\theta)$. Discretizing the conformal equations with a first-order symplectic Euler map and adding exponential-moving-average momenta with bias correction yields the update $\theta_{k+1,i} = \theta_{k,i} - \alpha \frac{\sqrt{1-\beta_2^{k+1}}}{\sqrt{\delta^2 y_{k+1,i} + \zeta_k}} \frac{v_{k+1,i}}{1-\beta_1^{k+1}}$, where the symplectic factor $\zeta_k$ grows from near zero to $1-\beta_2^{k+1}$. This factor is the paper's key addition: early in training it keeps the algorithm close to ADAM for fast progress, while later it grows to restore symplecticity and long-term stability. The paper proves that under smoothness, per-coordinate bounded gradients, and bounded variance, the expected squared gradient norm averages to $O(1/N)$, and that choosing $\delta=1$ with $\zeta=\epsilon$ recovers ADAM exactly, so the convergence theorem also gives ADAM a convergence guarantee.

Load-bearing premise

RAD's convergence proof assumes that every coordinate of every stochastic gradient that appears during training stays within a fixed bound M, an assumption that can fail when RL gradients spike to arbitrarily large values.

Editorial extensions

If this is right

  • If RAD's claims hold, RL practitioners can replace ADAM with RAD at essentially no memory cost (only the same two momenta) and get flatter, more stable learning curves plus higher final returns.
  • Because ADAM is the $\delta=1$, $\zeta=\epsilon$ limit of RAD, any improvement RAD shows on a task can be read as evidence that the growing symplectic factor, not the momentum scheme, is restoring stability.
  • The theorem guarantees $O(1/N)$ average squared gradient norm under bounded-gradient and bounded-variance assumptions; increasing batch size tightens the constant, giving a practical lever for harder RL problems.
  • The two-stage behavior (fast ADAM-like early search, symplectic late refinement) offers a principled alternative to heuristic switches from ADAM to SGD.
  • RAD's derivation framework extends to other kinetic energies and higher-order integrators, so the same conformal-symplectic recipe can generate new optimizers with built-in stability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The bounded-gradient assumption (Assumption 2) is likely violated in real deep RL, where gradients are heavy-tailed; if so, the $O(1/N)$ proof does not transfer directly, although the normalization step may still limit the damage in practice. This is an editorial extension, not a claim of the paper.
  • The interpretation of $\zeta$ as a mass-energy term suggests a systematic hyperparameter law: choosing $\zeta$ roughly proportional to a fraction of the current second-moment scale should stabilize training without sacrificing adaptivity; we would test this by sweeping $\zeta$ schedules on a single Atari game.
  • The same conformal-symplectic recipe might also tame optimization in non-RL settings with heavy-tailed gradients, such as large language model fine-tuning, where abnormal gradient spikes are known to occur.
  • RAD's connection to ADAM hints that the small rational factor $\epsilon$ in ADAM is not merely a numerical safeguard but an implicit symplectic factor; if true, simply raising $\epsilon$ in ADAM should improve its stability, which the paper's Hopper ablation supports.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes relativistic adaptive gradient descent (RAD), an optimizer derived by discretizing a conformal Hamiltonian system of independent one-dimensional relativistic particles. RAD applies per-coordinate speed limits through a relativistic kinetic energy term, combines them with exponential moving averages and bias correction, and reduces to ADAM when the speed coefficient is one and the symplectic factor is fixed. The paper proves a sublinear convergence bound for RAD under nonconvex stochastic optimization, presents a universal framework connecting conformal symplectic integrators to neural-network updates, and reports empirical gains over nine baseline optimizers across twelve RL environments, including a 155.1% improvement over ADAM on Seaquest-v4.

Significance. The framework is attractive and the derivation is self-contained: the paper gives a concrete recipe for translating conformal Hamiltonian dynamics into iterative optimizers, and the exact algebraic correspondence between RAD and ADAM is a genuine insight. The per-coordinate relativistic formulation is a novel and potentially useful design for stabilizing RL training. If the convergence proof can be repaired so that it actually covers the algorithm as run, and if the recommended hyperparameters are brought inside the theorem's hypotheses, the paper would be a solid contribution. The empirical section, despite relying on a limited number of seeds, supports the practical message that the increasing symplectic factor and speed limit improve training stability on several Atari and MuJoCo benchmarks.

major comments (4)
  1. [Section V (Theorem 1) and Section VI.A] The hypotheses of Theorem 1 are not satisfied by the configuration used in all experiments. Algorithm 3 sets ζ_k = min{ε_k, 1−β2^{k+1}}, and Section VI.A recommends δ=1 with κ=12π in (15), so ζ_0 = exp(−12π) ≈ 4×10^{−17}. With β2=0.999, the condition β2 ≥ 1 − ζ_0/(16M^2δ^2) forces M ≤ sqrt(ζ_0/(16(1−β2))) ≈ 5×10^{−8}, and α ≤ sqrt(ζ_0)/(2L) ≈ 6×10^{−9}/L. The reported learning rates are 10^{−3} to 5×10^{−4}, and no evidence is given that any network loss in the paper has a coordinate-wise gradient bound near 10^{−8}. Thus the O(1/N) bound in Theorem 1 does not apply to the algorithm as run in the experiments, even if Assumption 2 is granted.
  2. [Section V (Theorem 1) and Appendix H] The theorem statement claims convergence for θ_k generated by Algorithm 3, but the proof explicitly omits bias correction and sets β1=0, whereas Algorithm 3 and all experiments use β1=0.9 and bias correction. The paper asserts without proof that the analysis extends to the general case. In addition, the proof conditions on θ_k rather than the full history F_k; since y_k and θ_k depend on the same past batches, the claim that y_k is independent of B_k is not justified under conditioning on θ_k alone. These gaps are fixable, but the stated theorem currently exceeds what is proved.
  3. [Appendix E (proof of Corollary 5)] The proof of Corollary 5 is invalid as written. In bounding Σ_{k=0}^{N−1} 1/(B_k√ζ_k), the proof replaces 1/(k+1) by 1 in the first sum, then treats Σ_{k=0}^{N̂−1} e^{κ/2(1−k/N̂)} as part of an expression claimed to be o(N). With the factor dropped, that sum is Θ(N), so the final expression is O(N), not o(N). The conclusion of Corollary 5 may be salvageable by retaining the 1/(k+1) factor and obtaining O(log N), but the written proof does not establish it. Since this corollary provides the specific schedule used in the experiments, the error is load-bearing.
  4. [Section V (Assumption 2)] Assumption 2, which bounds every coordinate of every stochastic gradient by a uniform constant M, is very strong for deep RL, where gradient norms can spike and coordinate magnitudes are not uniformly bounded. The proof uses this assumption to control the cross term and to assert 0 ≤ y_{k,i} ≤ M^2. The paper provides no empirical verification of such a bound for the benchmark tasks. Even setting aside the quantitative mismatch in the first comment, the convergence result is conditional on a property that is unlikely to hold for the evaluated losses.
minor comments (3)
  1. [Section IV.C] The remark that RAD degrades exactly to ADAM when δ=1 and ζ is a small positive constant is only exact if that fixed ζ is used from the first iteration; in Algorithm 3, ζ_k = min{ε_k, 1−β2^{k+1}}, so one should state that the equivalence holds when ε_k is constant and 1−β2^{k+1} ≥ ε for all k.
  2. [Table I] On Swimmer-v3, RAD and ADAM both report 32±2 and the improvement column shows 0.0%, so the text's claim that RAD 'consistently demonstrates faster convergence and achieves the highest TAR across all MuJoCo tasks' is not supported by this table.
  3. [Appendix E and Figure 7] The proof of Corollary 5 uses both N̂ and N-hat notations inconsistently, and Figure 7's caption refers to 'Algorithm IV' when the RAD pseudocode is Algorithm 3.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the algorithm derivation, convergence theorem, and explicit ADAM reduction are self-contained, and the theory–experiment hyperparameter mismatch is a correctness concern, not a circular step.

full rationale

The paper derives RAD from a conformal Hamiltonian multi-particle system through explicit changes of variables (Section III and IV, Eqs. 5–13), and this derivation does not presuppose the algorithm's convergence or empirical results. Theorem 1 is proved from explicit assumptions (L-smoothness, coordinate-wise gradient bound, bounded variance) via a descent lemma and Lemma 1; the proof does not use the claimed bound as an input. The reduction of RAD to ADAM (Section IV-C) is a direct algebraic identity: setting δ=1 and ζ to a constant in Algorithm 3 yields the update rule of Algorithm 4. Presenting this equivalence as a connection is not circular, since it does not validate RAD's other claims. Self-citations appear in background, platform, and benchmark contexts (e.g., refs. [1], [35], [36]) and are not load-bearing for the central derivation or proof. The skeptic's observation that the recommended hyperparameters (δ=1, κ=12π, α≈1e-3) violate Theorem 1's conditions (α ≤ √ζ0/(2L), β2 ≥ 1−ζ0/(16M²δ²)) is a gap between the theorem's hypotheses and the experimental configuration, and it may be a correctness or rigor issue, but it is not circular reasoning: the paper does not fit parameters to the bound or define the algorithm in terms of the theorem's conclusion. The apparent flaw in Appendix E's proof of Corollary 5 is a mathematical error, not a self-referential step. No quoted step reduces by construction to its own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The algorithm's derivation rests on modeling NN training as a multi-particle relativistic conformal Hamiltonian system and then discretizing with a conformal symplectic integrator; this is a design choice rather than a theorem. The convergence analysis adds three standard stochastic-optimization assumptions: smoothness, coordinate-wise bounded gradients, and bounded variance. The speed coefficient δ and annealing rate κ are free hyperparameters tuned by the authors. No new physical entities are introduced.

free parameters (2)
  • speed coefficient δ = 1 (default), 5 (ablation)
    Controls the upper bound on parameter update magnitude, |Δθ_i| ≤ α/δ. δ=1 is recommended through extensive testing; δ=5 is shown to further reduce oscillation on Humanoid-v3.
  • annealing rate κ for symplectic factor ζ_k = 12π
    Sets the rate at which ζ_k rises from 1-β2 to larger values; chosen by tuning across tasks, described as 'extensive testing' in Section VI-A.
assumptions (4)
  • domain assumption The loss function L is L-smooth (Assumption 1).
    Used to derive the descent inequality in Theorem 1.
  • domain assumption Each coordinate of the stochastic gradient is bounded by M (Assumption 2).
    Critical for bounding cross terms in the proof of Theorem 1; may be violated in deep RL.
  • domain assumption Coordinate-wise gradient variance is bounded by σ_i^2 (Assumption 3).
    Used in Lemma 1 to bound expected squared gradients.
  • domain assumption NN training can be modeled as a conformal Hamiltonian system; a conformal symplectic integrator preserves long-term stability.
    This analogy motivates RAD and the stability claims; the formal convergence theorem does not require it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Conformal Symplectic Optimization for Stable Reinforcement Learning." pith.science (2026). https://pith.science/paper/UG7FKCUQ

@misc{pith2026241202291,
  author       = {Pith},
  title        = {Pith review of: Conformal Symplectic Optimization for Stable Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UG7FKCUQ}},
  note         = {Machine review of arXiv:2412.02291}
}
read the original abstract

Training deep reinforcement learning (RL) agents necessitates overcoming the highly unstable nonconvex stochastic optimization inherent in the trial-and-error mechanism. To tackle this challenge, we propose a physics-inspired optimization algorithm called relativistic adaptive gradient descent (RAD), which enhances long-term training stability. By conceptualizing neural network (NN) training as the evolution of a conformal Hamiltonian system, we present a universal framework for transferring long-term stability from conformal symplectic integrators to iterative NN updating rules, where the choice of kinetic energy governs the dynamical properties of resulting optimization algorithms. By utilizing relativistic kinetic energy, RAD incorporates principles from special relativity and limits parameter updates below a finite speed, effectively mitigating abnormal gradient influences. Additionally, RAD models NN optimization as the evolution of a multi-particle system where each trainable parameter acts as an independent particle with an individual adaptive learning rate. We prove RAD's sublinear convergence under general nonconvex settings, where smaller gradient variance and larger batch sizes contribute to tighter convergence. Notably, RAD degrades to the well-known adaptive moment estimation (ADAM) algorithm when its speed coefficient is chosen as one and symplectic factor as a small positive value. Experimental results show RAD outperforming nine baseline optimizers with five RL algorithms across twelve environments, including standard benchmarks and challenging scenarios. Notably, RAD achieves up to a 155.1% performance improvement over ADAM in Atari games, showcasing its efficacy in stabilizing and accelerating RL training.

Figures

Figures reproduced from arXiv: 2412.02291 by the authors.

Figure 1
Figure 1. Learning curves of CartPole-v1, wherein the TAR indicates the [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Policy performance of SAC on MuJoCo tasks. The solid lines [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 4
Figure 4. Performance comparison among SOTA optimizers. The solid [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Policy performance on Atari games. The solid lines correspond [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]
Figure 5
Figure 5. Figure 5: Policy performance on Walker2d-v3 under observation noise. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 7
Figure 7. Figure 7: Ablation studies on different symplectic factors [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 6
Figure 6. Figure 6: Ablation studies on different speed coefficients [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 8
Figure 8. Figure 8: Policy performance of TD3 on MuJoCo tasks. The solid lines correspond to the mean, and the shaded regions correspond to the 95% [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Under fixed innovation coupling, finite-horizon optimizers admit minimal pathwise realizations and incidence-identifiable Möbius effects, with a five-term readout transfer from hidden relaxation and a closed reduced-v...

Reference graph

Works this paper leans on

38 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    Eben Li, Reinforcement Learning for Sequential Decision and Optimal Control

    S. Eben Li, Reinforcement Learning for Sequential Decision and Optimal Control. Springer Berlin Heidelberg, 2023

    work page 2023

  2. [2]

    Mastering the game of go with deep neural networks and tree search,

    D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V . Panneershelvam, M. Lanctot et al., “Mastering the game of go with deep neural networks and tree search,” nature, vol. 529, no. 7587, pp. 484–489, 2016

    2016

  3. [3]

    Grandmaster level in starcraft ii using multi-agent reinforcement learning,

    O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev et al. , “Grandmaster level in starcraft ii using multi-agent reinforcement learning,” Nature, vol. 575, no. 7782, pp. 350–354, 2019

    2019

  4. [4]

    Addressing function approximation error in actor-critic methods,

    S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in International conference on machine learning. PMLR, 2018, pp. 1587–1596

    2018

  5. [5]

    Distributional soft actor-critic: Off-policy reinforcement learning for addressing value estimation errors,

    J. Duan, Y . Guan, S. E. Li, Y . Ren, Q. Sun, and B. Cheng, “Distributional soft actor-critic: Off-policy reinforcement learning for addressing value estimation errors,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 11, pp. 6584–6598, 2022

    2022

  6. [6]

    Integrated decision and control: Toward interpretable and computationally efficient driving intelligence,

    Y . Guan, Y . Ren, Q. Sun, S. E. Li, H. Ma, J. Duan, Y . Dai, and B. Cheng, “Integrated decision and control: Toward interpretable and computationally efficient driving intelligence,” IEEE Transactions on Cybernetics, vol. 53, no. 2, pp. 859–873, 2023

    work page 2023

  7. [7]

    Encoding distributional soft actor-critic for autonomous driving in multi-lane scenarios [research frontier],

    J. Duan, Y . Ren, F. Zhang, J. Li, S. E. Li, Y . Guan, and K. Li, “Encoding distributional soft actor-critic for autonomous driving in multi-lane scenarios [research frontier],” IEEE Computational Intelligence Magazine, vol. 19, no. 2, pp. 96–112, 2024

    work page 2024

  8. [8]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

Show all 38 references
  1. [9]

    On the optimization landscape of dynamic output feedback linear quadratic control,

    J. Duan, W. Cao, Y . Zheng, and L. Zhao, “On the optimization landscape of dynamic output feedback linear quadratic control,” IEEE Transactions on Automatic Control , vol. 69, no. 2, pp. 920–935, 2024

    2024

  2. [10]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” in 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Tr...

    2016

  3. [11]

    Deep reinforcement learning with double q-learning,

    H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, 2016

    2016

  4. [12]

    Soft actor-critic algorithms and applications,

    T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V . Kumar, H. Zhu, A. Gupta, P. Abbeel, and S. Levine, “Soft actor-critic algorithms and applications,” CoRR, vol. abs/1812.05905, 2018

    2018 arXiv

  5. [13]

    Trust region policy optimization,

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International conference on machine learning. PMLR, 2015, pp. 1889–1897

    2015

  6. [14]

    Comparison of the stochastic gradient descent based optimization techniques,

    E. Yazan and M. F. Talu, “Comparison of the stochastic gradient descent based optimization techniques,” in 2017 International Artificial Intelligence and Data Processing Symposium (IDAP) . IEEE, 2017, pp. 1–5

    2017

  7. [15]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, and A. Courville, Deep learning. MIT press, 2016

    2016

  8. [16]

    Optimization methods for large-scale machine learning,

    L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for large-scale machine learning,” SIAM review, vol. 60, no. 2, pp. 223–311, 2018

    2018

  9. [17]

    Adaptive subgradient methods for online learning and stochastic optimization,

    J. Duchi, E. Hazan, and Y . Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” Journal of Machine Learning Research, vol. 12, no. Jul, pp. 2121–2159, 2011

    2011

  10. [18]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR, San Diego, CA, USA , 2015

    2015

  11. [19]

    A variational perspective on accelerated methods in optimization,

    A. Wibisono, A. C. Wilson, and M. I. Jordan, “A variational perspective on accelerated methods in optimization,” proceedings of the National Academy of Sciences , vol. 113, no. 47, pp. E7351–E7358, 2016

    2016

  12. [20]

    Gradient flows and prox- imal splitting methods: A unified view on accelerated and stochastic optimization,

    G. Fran c ¸a, D. P. Robinson, and R. Vidal, “Gradient flows and prox- imal splitting methods: A unified view on accelerated and stochastic optimization,” Physical Review E , vol. 103, no. 5, p. 053304, 2021

    2021

  13. [21]

    Dynamical, symplectic and stochastic perspectives on gradient-based optimization,

    M. I. Jordan, “Dynamical, symplectic and stochastic perspectives on gradient-based optimization,” in Proceedings of the International Congress of Mathematicians: Rio de Janeiro 2018 . World Scientific, 2018, pp. 523–549

    2018

  14. [22]

    On dissipative symplectic integration with applications to gradient-based optimization,

    G. Fran c ¸a, M. I. Jordan, and R. Vidal, “On dissipative symplectic integration with applications to gradient-based optimization,” Journal of Statistical Mechanics: Theory and Experiment , vol. 2021, no. 4, p. 043402, 2021

    2021

  15. [23]

    Optimization with momentum: Dynamical, control-theoretic, and symplectic perspectives,

    M. Muehlebach and M. I. Jordan, “Optimization with momentum: Dynamical, control-theoretic, and symplectic perspectives,” Journal of Machine Learning Research , vol. 22, no. 73, pp. 1–50, 2021

    2021

  16. [24]

    Feng and M

    K. Feng and M. Qin, Symplectic geometric algorithms for Hamiltonian systems. Springer, 2010

    2010

  17. [25]

    Second order conformal symplectic schemes for damped hamiltonian systems,

    A. Bhatt, D. Floyd, and B. E. Moore, “Second order conformal symplectic schemes for damped hamiltonian systems,” Journal of Scientific Computing, vol. 66, no. 3, pp. 1234–1259, 2016

    2016

  18. [26]

    Conformal symplectic and relativistic optimization,

    G. Franc ¸a, J. Sulam, D. Robinson, and R. Vidal, “Conformal symplectic and relativistic optimization,” Advances in Neural Information Processing Systems, vol. 33, pp. 16 916–16 926, 2020

    2020

  19. [27]

    The marginal value of adaptive gradient methods in machine learning,

    A. C. Wilson, R. Roelofs, M. Stern, N. Srebro, and B. Recht, “The marginal value of adaptive gradient methods in machine learning,” in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Lon...

    2017

  20. [28]

    Mujoco: A physics engine for model- based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model- based control,” in 2012 IEEE/RSJ international conference on intelligent robots and systems . IEEE, 2012, pp. 5026–5033. 15

    2012

  21. [29]

    The arcade learning environment: An evaluation platform for general agents,

    M. G. Bellemare, Y . Naddaf, J. Veness, and M. Bowling, “The arcade learning environment: An evaluation platform for general agents,” Journal of Artificial Intelligence Research , vol. 47, pp. 253–279, 2013

    2013

  22. [30]

    Hairer, G

    E. Hairer, G. Wanner, and C. Lubich, Geometric Numerical Integration: Structure-Preserving Algorithms for Ordinary Differential Equations . Springer Science & Business Media, 2006

    2006

  23. [31]

    Adaptive methods for nonconvex optimization,

    M. Zaheer, S. Reddi, D. Sachan, S. Kale, and S. Kumar, “Adaptive methods for nonconvex optimization,” Advances in neural information processing systems, vol. 31, 2018

    2018

  24. [32]

    Improving generalization performance by switching from adam to SGD,

    N. S. Keskar and R. Socher, “Improving generalization performance by switching from adam to SGD,” CoRR, vol. abs/1712.07628, 2017

    2017 arXiv

  25. [33]

    Incorporating nesterov momentum into adam,

    T. Dozat, “Incorporating nesterov momentum into adam,” in Proceedings of the 4th International Conference on Learning Representations , 2016, pp. 1–4

    2016

  26. [34]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 , 2019

    2019

  27. [35]

    Gops: A general optimal control problem solver for autonomous driving and industrial control applications,

    W. Wang, Y . Zhang, J. Gao, Y . Jiang, Y . Yang, Z. Zheng, W. Zou, J. Li, C. Zhang, W. Cao, G. Xie, J. Duan, and S. E. Li, “Gops: A general optimal control problem solver for autonomous driving and industrial control applications,” Communications in Transportation Research, vo...

    2023

  28. [36]

    A reinforcement learning benchmark for autonomous driving in general urban scenarios,

    Y . Jiang, G. Zhan, Z. Lan, C. Liu, B. Cheng, and S. E. Li, “A reinforcement learning benchmark for autonomous driving in general urban scenarios,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 5, pp. 4335–4345, 2024

    2024

  29. [37]

    Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality: Second Edition

    W. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality: Second Edition . Wiley-Blackwell, 2011. Yao Lyu received his B.S. degree in Automotive En- gineering from Tsinghua University, Beijing, China, in 2019. He is currently pursuing his Ph.D. degree i...

    2013

  30. [2018]

    His research interests include computer vision, machine learning, and autonomous driving

    He is currently an Assistant Research Fellow with the School of Vehicle and Mobility, Tsinghua University. His research interests include computer vision, machine learning, and autonomous driving. Dr. He has authored over 40 granted patents and multiple academic papers in inte...

    1985

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.