Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Online gradient steps can learn the optimal LQR gain while never losing closed-loop stability.

desk verdict The indirect vanilla-gradient PGAC proof is credible and the framework is new, but the paper's 'all approaches' guarantee outruns the proofs actually present; still worth a serious referee. read the letter →

arxiv 2505.03706 v2 pith:VZXMOOWE submitted 2025-05-06 math.OC cs.SYeess.SY

Feiran Zhao , Alessandro Chiuso , Florian Dörfler This is my paper · ORCID
classification math.OCcs.SYeess.SY MSC 93C4093E3549N10
keywords adaptivecontrolpolicygradientlinearquadraticregulatorclosed-loopstabilitysignal-to-noiseratiodata-drivencertaintyequivalenceonlinelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a feedback controller can learn the optimal linear-quadratic regulator gain online by taking small gradient steps on the LQR cost using only closed-loop data, while staying inside the set of stabilizing gains. This matters because optimality-seeking adaptive controllers, including reinforcement-learning-style policy updates, have generally lacked stability guarantees. The central result is a per-time bound: the optimality gap decays exponentially from its initial value plus a bias that scales with the inverse signal-to-noise ratio of the data, and the state trajectory stays bounded by an exponentially decaying term plus a noise-dependent bias. The same template is shown to cover indirect and direct formulations, vanilla and natural gradients, a Gauss-Newton update that becomes an adaptive Hewer's algorithm, and a variance-based regularizer.

What carries the argument

The load-bearing object is the policy-gradient structure of the LQR cost $C(K)$ together with the signal-to-noise ratio $\mathrm{SNR}_t=\gamma_t/\delta_t$ of the collected data. The non-convex cost is gradient dominated, so exact gradient descent on the true plant converges linearly, and the paper's perturbation lemmas show that replacing the true model with the least-squares estimate perturbs the per-step cost decrease only by a term proportional to $1/\mathrm{SNR}_t$. Sequential stability of the switched closed-loop system is then enforced by choosing the stepsize small enough that consecutive gains change slowly, in the precise sense of strong stability. A further identity links the two families: the direct projected gradient update is exactly the indirect update premultiplied by a data-dependent positive definite matrix $M_t$, which is why both methods share the same stability-and-convergence template.

What would settle it

On the benchmark of Section VI with i.i.d. Gaussian noise and constant probing excitation, run Algorithm 1 and record $C(K_t)-C^*$ together with $\mathrm{SNR}_t$; the theorem predicts the optimality gap is bounded by an exponentially decaying term plus a term proportional to $\sum(1-\eta/2\mu)^{t-i}/\mathrm{SNR}_i$, which is $O(1/\sqrt{t})$. If the empirical gap decays slower than $O(1/\sqrt{t})$, or diverges while $\mathrm{SNR}_t$ stays above the threshold and $\eta$ below its bound, the certified bound is false.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that policy gradient adaptive control solves a two-part problem: starting from a stabilizing gain and offline persistently exciting data, alternate between applying a feedback-plus-probing input and taking one gradient step on a certainty-equivalence LQR cost. In the indirect version the gradient is computed through a recursive least-squares model estimate; in the direct version it is computed through a sample-covariance parameterization of the policy. For every variant the authors prove sequential stability of the switched closed-loop system and non-asymptotic convergence of the gain to the optimal LQR gain. Theorem 1 states the result for the indirect vanilla-gradient case: if the data signal-to-noise ratio stays above a threshold and the stepsize is small enough, the state obeys the bound in (22) and the optimality gap obeys (23), an exponential contraction plus an SNR-weighted bias. The signal-to-noise ratio is the information metric that sets the convergence rate, so the guarantees degrade smoothly as data become noisier relative to excitation, in line with the monotonicity principles of adaptive control.

Load-bearing premise

The proven stability and convergence hold only when the data are informative enough and the policy is updated slowly enough, and the constants defining those thresholds depend on the unknown system and the initial controller, so a user cannot verify the condition from measured data alone.

Editorial extensions

If this is right

  • Indirect PGAC with the vanilla gradient converges at the certified rate $O(1/\sqrt{t})$ when noise is i.i.d. Gaussian and excitation is constant, improving on the earlier sublinear rate for the direct method.
  • Natural-gradient PGAC costs less per step, since it solves one Lyapunov equation instead of two, and provably bridges the indirect and direct updates.
  • Setting the Gauss-Newton stepsize to $1/2$ recovers an adaptive version of Hewer's algorithm, which coincides with online identification-based policy iteration and inherits local quadratic convergence near the optimal gain.
  • Regularizing with a coefficient $\lambda_t \le O(\delta_t)$ preserves all stability and convergence guarantees while compensating uncertainty in the closed-loop covariance; in the paper's simulations it raises the fraction of convergent trials from roughly 81-83 percent to 98-99 percent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the SNR-weighted bias term suggests a general recipe for adaptive reinforcement learning: whenever an estimated gradient is used, stepsizes inversely proportional to the current SNR yield both a stability certificate and a rate, with the LQR serving as the case where gradient dominance is available.
  • The equivalence between direct and indirect updates through the matrix $M_t$ indicates that the conditioning of the data matrix, not just its minimum singular value, controls how close a model-free update is to a model-based one; shaping probing noise to improve the spectrum of $\Phi_t$ is a plausible accelerator for direct PGAC.
  • The simulations show the optimality gap decreasing as $O(1/t)$ even though the certified bound is $O(1/\sqrt{t})$, so the analysis is plausibly loose; a sharper Lyapunov or bias-decay argument might match the known one-shot adaptive LQR rate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes policy gradient adaptive control (PGAC) for the linear quadratic regulator, unifying indirect (model-based identification followed by gradient computation) and direct (sample-covariance parameterization) approaches. The algorithm alternates between applying a state-feedback policy plus probing noise and performing one gradient step on the LQR cost, with variants using vanilla gradient, natural gradient, Gauss-Newton updates, and a variance-based regularizer. The central claim is that, under a sufficiently large data signal-to-noise ratio and a sufficiently small stepsize, the policy sequence is sequentially stable and the optimality gap converges to a bias that scales inversely with the SNR. The main theoretical result with a complete proof chain is Theorem 1 for indirect vanilla PGAC, proved in Appendices A-B. Theorems 2-7 extend the guarantee to direct PGAC, natural gradient, Gauss-Newton, and regularized variants, but several load-bearing proofs are omitted or only sketched.

Significance. If all the stated theorems are correct, this is a significant contribution: it provides stability and optimality certificates for online gradient-based LQR adaptation, bridges indirect and direct data-driven formulations, connects Gauss-Newton PGAC to an adaptive Hewer's algorithm, and improves the convergence rate over the authors' earlier DeePO result. The proof of Theorem 1 is detailed and makes explicit use of gradient dominance, local smoothness, and Lyapunov perturbation bounds, which lends credibility to the core idea. The simulation study is small but informative, and the authors provide code. The main weakness is that the headline 'for all considered PGAC approaches' claim rests on several omitted proofs, notably Lemma 22, Lemmas 23-24, Lemma 27, and the proofs of Theorems 4, 6, and 7.

major comments (5)
  1. [Appendix A, Lemma 22] The proof of Lemma 22, which certifies sequential stability and the state bound for direct PGAC, is omitted with the statement 'follows the same vein of that of Lemmas 17 and 18'. This is load-bearing for Theorem 2: direct PGAC updates K by K' = K - eta M grad, where M is data-dependent, so one cannot simply reuse the indirect proof without additional bounds on M and its variation. The stability of Algorithm 2 is therefore not currently verified. A full proof is required.
  2. [Appendix B, Lemmas 23-24 and Theorem 3] The natural-gradient PGAC guarantee in Theorem 3 depends on Lemmas 23 and 24, which bound ||E_hat - E|| and the resulting cost difference; both proofs are omitted. Since the natural-gradient update has a different gradient error structure than the vanilla gradient, the claimed generalization of Theorem 1 does not follow automatically. These proofs must be supplied before Theorem 3 can be considered established.
  3. [Section IV-B, Theorem 4 and Lemma 27] Theorem 4, which gives convergence and stability for indirect Gauss-Newton PGAC, is stated without proof, and Lemma 27, the perturbation bound for the inverse term (R + B^T P B)^{-1} E, is stated as 'proof follows from that of Lemma 12 and is omitted'. The Gauss-Newton update is not a small perturbation of the vanilla gradient; the inverse term introduces additional nonlinear dependence on the estimate. The stated convergence certificate (35) is therefore unverified.
  4. [Section V, Theorems 6-7] The regularization results are not proven. Lemma 8 and Lemma 9 (gradient expressions for the regularized costs) are omitted, Lemma 31 is given but the rest of the proof of Theorem 6 is omitted, and Theorem 7 has no proof at all. Since the decay condition lambda_t <= nu_6 delta_t is essential for the claimed convergence to the unregularized optimal gain, the proofs of these statements must be included.
  5. [Theorems 1-7, conditions on nu_i] The constants nu_i and the polynomial bounds p_i in all theorems depend on the unknown plant (A,B,Q,R) and the initial gain K_{t0}. Consequently, the stepsize condition eta <= min{nu_2, 2 mu} and the SNR condition SNR_t >= nu_1 cannot be verified from data alone. This is a practical limitation that should be stated clearly, and the paper would be stronger if it gave even conservative explicit estimates for the constants or a procedure to certify them online.
minor comments (5)
  1. [Abstract and Section I] The abstract states 'For all the considered PGAC approaches, we show closed-loop stability and convergence', but the manuscript currently omits proofs for several of these approaches. The wording should be softened or the missing proofs added before publication.
  2. [Section III-C, Lemma 6] In the statement of Lemma 6, the lower bound is written as 'sigma(M_t) >= gamma_t^2' but the surrounding text and proof suggest 'sigma(M_t) >= gamma_t^2' should be 'sigma(M_t) >= gamma_t^2' with a superscript; please clarify the notation.
  3. [Theorem 5, discussion after (36)] The sentence 'the condition on the initial policy K_{t0} can be replaced with a condition on the SNR' is asserted without proof. If this claim is used, it should be proven or explicitly marked as a conjecture.
  4. [Section VI-A, Figures 2-3] The figures do not show the natural-gradient and Gauss-Newton curves in Fig. 2, and the text says they 'lie between those of the one-shot-based method and indirect PGAC with the vanilla gradient'. It would be clearer to include them or state explicitly why they are omitted.
  5. [Section III-C, Theorem 2] The condition SNR_t >= max{nu_1, nu_2 ||M_t||/sigma(M_t)} combines the information metric with the conditioning of M_t. Since M_t depends on online data, the paper should clarify whether this condition can be monitored online and what happens when it is violated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the Theorem 1 proof chain is self-contained, self-citations provide auxiliary lemmas but not the target claim, and the omitted-proof passages create a verifiability gap rather than a circular reduction.

full rationale

The central derivation of the indirect vanilla PGAC (Theorem 1) is self-contained relative to the stated assumptions: Lemma 1 bounds the least-squares model error by 1/SNR_t directly from the definitions of gamma_t and delta_t; Lemmas 12-18 then propagate this bound through Lyapunov perturbation, gradient dominance and smoothness (quoted from the external Fazel et al. result [18]), and the sequential-stability argument to obtain the state bound (22) and the optimality-gap bound (23). The direct variant is reduced to the indirect update via Lemma 6, which is an algebraic chain-rule identity, and the natural-gradient and Gauss-Newton variants are handled through the same perturbation estimates. Citations to the authors' own DeePO work ([21], [29], [34]) supply gradient formulas and Lyapunov perturbation lemmas, but none of those cited results assumes or contains the present stability/convergence certificate, so the self-citations are not load-bearing in a circular sense. The convergence rate involving 1/SNR_i is a theorem consequence of the gradient-error bound, not a fitted parameter renamed as a prediction. The manuscript does contain a completeness gap: Lemma 22, Theorem 4, Lemmas 23-24, and parts of the regularization theorems are stated with "proof follows same vein" or without proof, so the abstract's "for all considered PGAC approaches" claim is not fully verified as written; however, an omitted proof is a verifiability issue, not a circularity. No circular step satisfies the quote-and-reduction standard, so the score is 0.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claims rest on standard LQR theory, two common domain assumptions (PE and stabilizing initial gain), and several external results from [18] and [21]. Two tuning parameters (eta and lambda0) are hand-chosen; their certified ranges involve existential constants that depend on the unknown system.

free parameters (2)
  • Step-size sequence eta (or eta_t) = Simulation: eta=0.02 (indirect vanilla), 0.2 (natural), 0.5 (Gauss-Newton), eta_t=0.2/||M_t|| (direct)
    Theorems 1-7 require eta <= min{nu2,2mu} or similar, with nu_i existential functions of unknown (A,B,Q,R,K_t0). The user must hand-tune eta; the certified bound does not give an explicit constructive rule.
  • Regularization coefficient lambda0 (schedule lambda_t = lambda0 * delta_t) = Simulation uses lambda_t = 1/(10*sqrt(t-t0)) (i.e., lambda0 with delta_t ~ 1/sqrt(t)); Table II compares lambda=0 vs…
    Regularized PGAC requires lambda_t <= nu6 * delta_t, but the scale lambda0 is tuned by hand for performance and stability; the paper states 'we still need to tune the coefficient lambda0'.
assumptions (6)
  • domain assumption Assumption 1: Persistency of excitation: sigma(Phi_t) >= gamma_t > 0 for all t.
    Section III-A, used to define SNR, invert Phi_t in (9)/(12), and prove Lemma 1; without it the identification and covariance parameterization fail.
  • domain assumption Assumption 2: Initial gain K_t0 is stabilizing, K_t0 in S.
    Section III-B; required for the algorithm to start and for the sublevel set analysis to remain meaningful.
  • domain assumption (A,B) controllable and Q,R positive definite.
    Section II-A; guarantees the LQR has a unique optimal gain and the Riccati/Hewer convergence results apply.
  • standard math Gradient dominance and local smoothness of C(K) from Fazel et al. [18, Lemmas 1, 2, Theorem 7].
    Used as black boxes in Lemmas 3-4 and throughout the convergence proofs (Appendix A); no proof is reproduced.
  • standard math Perturbation bounds for Lyapunov equations (Lemma 11) from [21, Lemma 15].
    Assumed to bound ||Sigma - Sigma_hat|| and ||P - P_hat||; taken from prior work without proof.
  • domain assumption For the O(1/sqrt(t)) rate, noise is i.i.d. Gaussian with constant excitation so delta_t ~ O(1/sqrt(t)) and gamma_t ~ O(1).
    Remark 1 gives the SNR scaling used in the numerical examples; the general theorems only need a given SNR sequence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches." pith.science (2026). https://pith.science/paper/VZXMOOWE

@misc{pith2026250503706,
  author       = {Pith},
  title        = {Pith review of: Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VZXMOOWE}},
  note         = {Machine review of arXiv:2505.03706}
}
read the original abstract

Motivated by recent advances of reinforcement learning and direct data-driven control, we propose policy gradient adaptive control (PGAC) for the linear quadratic regulator (LQR), which uses online closed-loop data to improve the control policy while maintaining stability. Our method adaptively updates the policy in feedback by descending the gradient of the LQR cost and is categorized as indirect, when gradients are computed via an estimated model, versus direct, when gradients are derived from data using sample covariance parameterization. Beyond the vanilla gradient, we also showcase the merits of the natural gradient and Gauss-Newton methods for the policy update. Notably, natural gradient descent bridges the indirect and direct PGAC, and the Gauss-Newton method of the indirect PGAC leads to an adaptive version of the celebrated Hewer's algorithm. To account for the uncertainty from noise, we propose a regularization method for both indirect and direct PGAC. For all the considered PGAC approaches, we show closed-loop stability and convergence of the policy to the optimal LQR gain. Simulations validate our theoretical findings and demonstrate the robustness and computational efficiency of PGAC.

Figures

Figures reproduced from arXiv: 2505.03706 by the authors.

Figure 1
Figure 1. Classes of policy gradient adaptive control. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Finite-horizon cost of one-shot-based and PGAC methods. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adaptive Linear Quadratic Control of Unknown Linear Time-Varying Systems via Policy Gradient Methods

    math.OC 2026-07 accept novelty 6.0 of 10

    One-step policy-gradient LQR updates with normalized sliding-window least-squares stabilize unknown slowly varying and piecewise-constant linear systems and track frozen-time optima on average.

  2. Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression

    eess.SY 2025-12 unverdicted novelty 6.0 of 10

    Primal-dual robust linear regression enables O(1/epsilon) sample complexity for model-free policy gradient methods on stochastic LQR.

Reference graph

Works this paper leans on

41 extracted references · 23 canonical work pages · cited by 2 Pith papers

  1. [1]

    A historical perspective of adaptive control and learning,

    A. M. Annaswamy and A. L. Fradkov, “A historical perspective of adaptive control and learning,” Annual Reviews in Control , vol. 52, pp. 18–41, 2021

    work page 2021

  2. [2]

    Adaptive control and intersections with reinforce- ment learning,

    A. M. Annaswamy, “Adaptive control and intersections with reinforce- ment learning,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 6, pp. 65–93, 2023

    work page 2023

  3. [3]

    Adaptive servomechanisms,

    R. Drenick and R. Shahbender, “Adaptive servomechanisms,” Trans- actions of the American Institute of Electrical Engineers, Part II: Applications and Industry , vol. 76, no. 5, pp. 286–292, 1957

    work page 1957

  4. [4]

    P. A. Ioannou and J. Sun, Robust adaptive control. PTR Prentice-Hall Upper Saddle River, NJ, 1996, vol. 1

    work page 1996

  5. [5]

    On model-free adaptive control and its stability analysis,

    Z. Hou and S. Xiong, “On model-free adaptive control and its stability analysis,” IEEE Transactions on Automatic Control, vol. 64, no. 11, pp. 4555–4569, 2019

    2019

  6. [6]

    Adaptive control: Towards a complexity-based general theory,

    G. Zames, “Adaptive control: Towards a complexity-based general theory,” Automatica, vol. 34, no. 10, pp. 1161–1167, 1998

    work page 1998

  7. [7]

    Reinforcement learning and adaptive dynamic programming for feedback control,

    F. L. Lewis and D. Vrabie, “Reinforcement learning and adaptive dynamic programming for feedback control,” IEEE Circuits and Systems Magazine, vol. 9, no. 3, pp. 32–50, 2009

    2009

  8. [8]

    Value iteration and adaptive dynamic pro- gramming for data-driven adaptive optimal control design,

    T. Bian and Z.-P. Jiang, “Value iteration and adaptive dynamic pro- gramming for data-driven adaptive optimal control design,” Automatica, vol. 71, pp. 348–360, 2016

    2016

Show all 41 references
  1. [9]

    Certainty equivalence is efficient for linear quadratic control,

    H. Mania, S. Tu, and B. Recht, “Certainty equivalence is efficient for linear quadratic control,” in Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc., 2019

    2019

  2. [10]

    Almost surely √ T regret bound for adaptive LQR,

    Y . Lu and Y . Mo, “Almost surely √ T regret bound for adaptive LQR,” IEEE Transactions on Automatic Control (early access) , 2025

    2025

  3. [11]

    Naive exploration is optimal for online LQR,

    M. Simchowitz and D. Foster, “Naive exploration is optimal for online LQR,” in International Conference on Machine Learning. PMLR, 2020, pp. 8937–8948

    2020

  4. [12]

    Learning linear-quadratic regu- lators efficiently with only √ T regret,

    A. Cohen, T. Koren, and Y . Mansour, “Learning linear-quadratic regu- lators efficiently with only √ T regret,” in International Conference on Machine Learning. PMLR, 2019, pp. 1300–1309. 16

    2019

  5. [13]

    Fully adaptive regret-guaranteed algorithm for control of linear quadratic systems,

    J. A. Chekan and C. Langbort, “Fully adaptive regret-guaranteed algorithm for control of linear quadratic systems,” arXiv preprint arXiv:2406.07746, 2024

    2024 arXiv

  6. [14]

    Adaptive control by regulation-triggered batch least squares,

    I. Karafyllis, M. Kontorinaki, and M. Krstic, “Adaptive control by regulation-triggered batch least squares,” IEEE Transactions on Auto- matic Control, vol. 65, no. 7, pp. 2842–2855, 2019

    2019

  7. [15]

    Robustness of online identification-based policy iteration to noisy data,

    B. Song and A. Iannelli, “Robustness of online identification-based policy iteration to noisy data,” arXiv preprint arXiv:2504.07627 , 2025

    2025 arXiv

  8. [16]

    Failures of adaptive control theory and their resolu- tion,

    B. D. Anderson, “Failures of adaptive control theory and their resolu- tion,” Commun. Inf. Syst. , vol. 5, no. 1, pp. 1–20, 2005

    2005

  9. [17]

    Toward a theoretical foundation of policy optimization for learning control policies,

    B. Hu, K. Zhang, N. Li, M. Mesbahi, M. Fazel, and T. Bas ¸ar, “Toward a theoretical foundation of policy optimization for learning control policies,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 6, pp. 123–158, 2023

    2023

  10. [18]

    Global convergence of policy gradient methods for the linear quadratic regulator,

    M. Fazel, R. Ge, S. Kakade, and M. Mesbahi, “Global convergence of policy gradient methods for the linear quadratic regulator,” in Interna- tional Conference on Machine Learning , 2018, pp. 1467–1476

    2018

  11. [19]

    Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem,

    H. Mohammadi, A. Zare, M. Soltanolkotabi, and M. R. Jovanovi ´c, “Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem,” IEEE Transactions on Automatic Control, vol. 67, no. 5, pp. 2435–2450, 2022

    2022

  12. [20]

    Global convergence of policy gradient primal-dual methods for risk-constrained LQRs,

    F. Zhao, K. You, and T. Bas ¸ar, “Global convergence of policy gradient primal-dual methods for risk-constrained LQRs,” IEEE Transactions on Automatic Control, vol. 68, no. 5, pp. 2934–2949, 2023

    2023

  13. [21]

    Data-enabled policy optimization for direct adaptive learning of the LQR,

    F. Zhao, F. D ¨orfler, A. Chiuso, and K. You, “Data-enabled policy optimization for direct adaptive learning of the LQR,” arXiv preprint arXiv:2401.14871, 2024

    2024 arXiv

  14. [22]

    Integration of adaptive control and reinforcement learning for real-time control and learning,

    A. M. Annaswamy, A. Guha, Y . Cui, S. Tang, P. A. Fisher, and J. E. Gaudio, “Integration of adaptive control and reinforcement learning for real-time control and learning,” IEEE Transactions on Automatic Control, vol. 68, no. 12, pp. 7740–7755, 2023

    2023

  15. [23]

    Behavioral systems theory in data-driven analysis, signal processing, and control,

    I. Markovsky and F. D ¨orfler, “Behavioral systems theory in data-driven analysis, signal processing, and control,” Annual Reviews in Control , vol. 52, pp. 42–64, 2021

    2021

  16. [24]

    Data-enabled predictive control: In the shallows of the DeePC,

    J. Coulson, J. Lygeros, and F. D ¨orfler, “Data-enabled predictive control: In the shallows of the DeePC,” in 18th European Control Conference (ECC), 2019, pp. 307–312

    2019

  17. [25]

    Formulas for data-driven control: Stabilization, optimality, and robustness,

    C. De Persis and P. Tesi, “Formulas for data-driven control: Stabilization, optimality, and robustness,” IEEE Transactions on Automatic Control , vol. 65, no. 3, pp. 909–924, 2019

    2019

  18. [26]

    On the certainty-equivalence ap- proach to direct data-driven lqr design,

    F. D ¨orfler, P. Tesi, and C. De Persis, “On the certainty-equivalence ap- proach to direct data-driven lqr design,”IEEE Transactions on Automatic Control, vol. 68, no. 12, pp. 7989–7996, 2023

    2023

  19. [27]

    On the role of regularization in direct data-driven LQR control,

    ——, “On the role of regularization in direct data-driven LQR control,” in 61st IEEE Conference on Decision and Control (CDC) , 2022, pp. 1091–1098

    2022

  20. [28]

    Harnessing uncertainty for a separation principle in direct data-driven predictive control,

    A. Chiuso, M. Fabris, V . Breschi, and S. Formentin, “Harnessing uncertainty for a separation principle in direct data-driven predictive control,” Automatica, vol. 173, p. 112070, 2025

    2025

  21. [29]

    Data-enabled policy optimization for the linear quadratic regulator,

    F. Zhao, F. D ¨orfler, and K. You, “Data-enabled policy optimization for the linear quadratic regulator,” in 62nd IEEE Conference on Decision and Control (CDC) , 2023, pp. 6160–6165

    2023

  22. [30]

    Direct adaptive control of grid-connected power converters via output-feedback data-enabled policy optimization,

    F. Zhao, R. Leng, L. Huang, H. Xin, K. You, and F. D ¨orfler, “Direct adaptive control of grid-connected power converters via output-feedback data-enabled policy optimization,” arXiv preprint arXiv:2411.03909 , 2024

    2024 arXiv

  23. [31]

    Unified aeroelastic flutter and loads control via data-enabled policy optimiza- tion,

    X. Wang, F. Zhao, A. Jurisson, F. D ¨orfler, and R. S. Smith, “Unified aeroelastic flutter and loads control via data-enabled policy optimiza- tion,” IEEE Transactions on Aerospace and Electronic Systems , pp. 1– 12, 2025 (early access)

    2025

  24. [32]

    An adaptive data-enabled policy optimization approach for autonomous bicycle control,

    N. Persson, F. Zhao, M. Kaheni, F. D ¨orfler, and A. V . Papadopoulos, “An adaptive data-enabled policy optimization approach for autonomous bicycle control,” arXiv preprint arXiv:2502.13676 , 2025

    2025 arXiv

  25. [33]

    An iterative technique for the computation of the steady state gains for the discrete optimal regulator,

    G. Hewer, “An iterative technique for the computation of the steady state gains for the discrete optimal regulator,” IEEE Transactions on Automatic Control, vol. 16, no. 4, pp. 382–384, 1971

    1971

  26. [34]

    Regularization for covariance parameterization of direct data-driven LQR control,

    F. Zhao, A. Chiuso, and F. D ¨orfler, “Regularization for covariance parameterization of direct data-driven LQR control,” arXiv preprint arXiv:2503.02985, 2025

    2025 arXiv

  27. [35]

    On the sample com- plexity of the linear quadratic regulator,

    S. Dean, H. Mania, N. Matni, B. Recht, and S. Tu, “On the sample com- plexity of the linear quadratic regulator,” Foundations of Computational Mathematics, vol. 20, no. 4, pp. 633–679, 2020

    2020

  28. [36]

    B. D. Anderson and J. B. Moore, Optimal control: linear quadratic methods. Courier Corporation, 2007

    2007

  29. [37]

    A note on persistency of excitation,

    J. C. Willems, P. Rapisarda, I. Markovsky, and B. L. De Moor, “A note on persistency of excitation,” Systems & Control Letters , vol. 54, no. 4, pp. 325–329, 2005

    2005

  30. [38]

    The informativity approach: To data-driven analysis and control,

    H. J. Van Waarde, J. Eising, M. K. Camlibel, and H. L. Trentelman, “The informativity approach: To data-driven analysis and control,” IEEE Control Systems Magazine , vol. 43, no. 6, pp. 32–66, 2023

    2023

  31. [39]

    A quantitative notion of persistency of excitation and the robust fundamental lemma,

    J. Coulson, H. J. Van Waarde, J. Lygeros, and F. D ¨orfler, “A quantitative notion of persistency of excitation and the robust fundamental lemma,” IEEE Control Systems Letters , vol. 7, pp. 1243–1248, 2022

    2022

  32. [40]

    LQR through the lens of first order methods: Discrete-time case,

    J. Bu, A. Mesbahi, M. Fazel, and M. Mesbahi, “LQR through the lens of first order methods: Discrete-time case,” arXiv preprint arXiv:1907.08921, 2019

    1907 arXiv

  33. [41]

    Noise sensitivity of the semidefinite programs for direct data-driven lqr,

    X. Zeng, L. Bako, and N. Ozay, “Noise sensitivity of the semidefinite programs for direct data-driven lqr,” arXiv preprint arXiv:2412.19705 , 2024. Feiran Zhao received the B.S. degree in Control Science and Engineering from the Harbin Institute of Technology, China, in 2018, ...

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.