REVIEW 6 cited by
LQR through the Lens of First Order Methods: Discrete-time Case
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
We consider the Linear-Quadratic-Regulator (LQR) problem in terms of optimizing a real-valued matrix function over the set of feedback gains. Such a setup facilitates examining the implications of a natural initial-state independent formulation of LQR in designing first order algorithms. It is shown that this cost function is smooth and coercive, and provide an alternate means of noting its gradient dominated property. In the process, we provide a number of analytic observations on the LQR cost when directly analyzed in terms of the feedback gain. We then examine three types of well-posed flows for LQR: gradient flow, natural gradient flow and the quasi-Newton flow. The coercive property suggests that these flows admit unique solutions while gradient dominated property indicates that the corresponding Lyapunov functionals decay at an exponential rate; we also prove that these flows are exponentially stable in the sense of Lyapunov. We then discuss the forward Euler discretization of these flows, realized as gradient descent, natural gradient descent and the quasi-Newton iteration. We present stepsize criteria for gradient descent and natural gradient descent, guaranteeing that both algorithms converge linearly to the global optima. An optimal stepsize for the quasi-Newton iteration is also proposed, guaranteeing a $Q$-quadratic convergence rate--and in the meantime--recovering the Hewer algorithm.
Forward citations
Cited by 6 Pith papers
-
Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches
Online policy-gradient updates for unknown LQR systems are shown to be sequentially stable and convergent to the optimal gain, for indirect, direct, natural-gradient, Gauss-Newton and regularized versions.
-
Neural Policy Gradient Methods: Global Optimality and Rates of Convergence
Under strong regularity assumptions, neural natural policy gradient converges to a global optimum at rate O(1/sqrt(T)), and neural vanilla policy gradient converges to a stationary point at the same rate.
-
Hidden Star-Convexity in Policy Optimization for Gain-Scheduled LQR: Extended Version
For gain-scheduled LQR, the cost is exactly star-convex about the optimum under a covariance-substituted gradient, and a single mismatch ratio below one certifies linear convergence of gradient descent.
-
Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence
PI-FT, a policy-iteration algorithm for KL-regularized diffusion fine-tuning, converges linearly to the globally optimal control under Lipschitz smoothness assumptions.
-
Data-Driven LQR with Finite-Time Experiments via Extremum-Seeking Policy Iteration
EXP-LQR tunes LQR feedback gains from finite-time cost measurements alone, using sinusoidal perturbations and averaging theory to converge near the optimal gain.
-
Minimizing Smooth Kurdyka-{\L}ojasiewicz Functions via Generalized Descent Methods: Convergence Rate and Complexity
Descent methods obeying f(x_{k+1}) ≤ f(x_k) − ρ‖∇f(x_k)‖^θ converge linearly when θ equals the inverse KL exponent, with a unified rate/complexity analysis.
Discussion (0). Continue with ORCID to comment.