Pith. sign in

REVIEW 3 major objections 4 minor 28 references

Direct Data Driven Natural Gradient Descent for Control

T0 review · 3 major / 4 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Direct data-driven natural gradient control forces closed-loop states to follow an uncertainty-aware descent path without identifying a model.

desk verdict Solid data-driven extension of the authors' NGD idea with clean LMIs and real robot runs; the linearization gap is real but does not erase the contribution. read the letter →

arxiv 2607.03393 v1 pith:ZVBFICTS submitted 2026-07-03 eess.SY cs.SY

Ramin Esmzad , Farnaz Adib Yaghmaie , Bahare Kiumarsi , Hamidreza Modares This is my paper · ORCID
classification eess.SYcs.SY
keywords data-drivencontrolnaturalgradientdescentFisherInformationMatrixlinearinequalitiesclosed-loopcovariancedirectparameterizationrobotics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a linear feedback gain can be synthesized directly from input-state data so that the closed-loop mean state evolves exactly as a natural-gradient step. The Fisher Information Matrix of the Gaussian state is the inverse covariance; using it as a preconditioner makes the update large in uncertain directions and small in well-known ones. Two data-based representations of the closed-loop map (raw snapshots and sample covariances) are turned into linear matrix inequalities that enforce this geometry and certify contraction. A single scalar step-size then trades speed against smoothness, replacing the usual multi-matrix LQR tuning. Hardware trials on a Mecanum robot show that the resulting policies are stable, intuitive, and competitive with classical and data-driven LQR baselines under limited data.

What carries the argument

The Fisher Information Matrix of a Gaussian state (G=Σ^{-1}) together with the two data-parameterizations X_1 G = I−2αΣP (or the covariance analogue); these identities force the closed-loop mean to follow natural-gradient flow while the accompanying LMIs certify contraction and recover the gain K from data alone.

What would settle it

Collect a data set that deliberately violates full row rank or drive the robot with large heading angles that leave the linear regime; if the LMI-synthesized gain still produces the predicted natural-gradient contraction and matches hardware trajectories, the claim is false.

Watch

Extended reading notes

Core claim

Given sufficiently rich input-state data, a feasible solution of the stated LMIs yields a linear gain K such that the data-based closed-loop mean dynamics equal the natural-gradient recursion μ_{k+1}=(I−2αΣP)μ_k and are therefore λ-contractive in expectation; the same LMIs also upper-bound the stationary covariance so that the Fisher Information Matrix used for preconditioning remains consistent with the closed-loop uncertainty.

Load-bearing premise

The collected data matrix must have full row rank and the plant must stay inside the linear small-heading model used to gather that data; if either fails, the LMI certificates no longer apply to the physical system.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper develops a direct data-driven natural-gradient-descent (NGD) control framework for unknown stochastic LTI systems. Using two data-based closed-loop parameterizations (raw input-state and sample-covariance), it embeds the Fisher information matrix (inverse closed-loop covariance) as a preconditioner so that the mean dynamics exactly reproduce the NGD recursion μ_{k+1}=(I-2αΣP)μ_k. Theorems 1 and 2 supply LMI/SDP conditions that certify λ-contractiveness of this mean dynamics (hence stability) and recover the linear gain K; supporting sample-complexity and iteration-complexity lemmas are given. The approach is demonstrated in Monte-Carlo SNR studies, (N,α) sweeps, Gazebo simulation, and hardware experiments on a ROSbot XL Mecanum platform, with comparisons to model-based LQR and existing data-driven LQR baselines that emphasize single-parameter (α) interpretability.

Significance. If the claims hold, the work supplies a geometrically motivated, single-scalar-tuned alternative to classical Q/R shaping for data-driven LQR-like design, together with explicit uncertainty-aware covariance recursions (eqs. 13, 21) and SDP certificates. The hardware demonstration on a real mobile robot and the systematic (N,α) trade-off tables are concrete strengths that go beyond purely theoretical data-driven LMI papers. The contribution is therefore of genuine interest to the data-driven and learning-based control communities, provided the linearization gap and the conservatism of the convex relaxations are clarified.

major comments (3)
  1. [Theorems 1–2, Section V, Appendix A, eqs. (48)–(51)] Theorems 1–2 certify λ-contractiveness only for the linear data-based maps (11) and (19) that rest on the constant-A=I linearization (50)–(51) and Assumption 3. All hardware, Gazebo, and nonlinear SNR results (Table II, Section V, Appendix A) are obtained on the true Mecanum kinematics (48)–(49). No residual bound, maximum heading excursion, or empirical validation that the realized closed-loop remains inside the certified linear regime is supplied; consequently the abstract claim of “stability-guaranteed policy synthesis … on a ROSbot XL platform” is not rigorously supported by the theory. Either restrict the claims, quantify the linearization error, or add a supporting nonlinear argument.
  2. [Theorems 1–2, eqs. (28c)–(28e), (36)–(37)] The convex relaxations M ≻ GΣGᵀ and Z ≻ YΣ^{-1}Y (eqs. 28c–e and the analogous set in Theorem 2) are only sufficient. The manuscript never checks tightness, reports the duality gap, or verifies a posteriori that the recovered K satisfies the original stationary-covariance equality rather than merely the relaxed upper bound. Without such evidence the certificates may be arbitrarily conservative, especially for the small data sets (N=24) used on hardware.
  3. [Lemma 4, Section V.F, Tables III–IV, Appendix B] Lemma 4 supplies a high-probability sample-size bound under Gaussian noise and the designed closed-loop, yet the hardware experiments use only N=24 samples for a 7-dimensional regressor and never report the realized condition number of Φ or D_0, nor verify the BMSB constants. Given that Theorem 2 is already shown to be highly sensitive to small N and tiny α (Tables III–IV), the practical reliability of the rank and positive-definiteness assumptions under the collected excitation remains unquantified.
minor comments (4)
  1. [Algorithm 1, Theorems 1–2] Notation for the two parameterizations (G versus H, X_1 versus X-bar_1) is introduced cleanly but then occasionally mixed in the algorithm box and the recovery formulas; a short consistency pass would help.
  2. [Section V, Appendix] Figures 1 and 5–14 would benefit from explicit legends that identify which curve belongs to which controller/α; several captions simply say “various α” without listing the values.
  3. [Assumption 2, Section V.D] The free parameters α, λ and W are acknowledged, yet the text never states how W is chosen for the hardware runs (estimated or hand-tuned). A one-sentence clarification would remove ambiguity.
  4. [Throughout] A few typographical inconsistencies appear (e.g., “Linköping” vs. “Link ¨oping”, missing spaces around “λ-contractive”). None affect readability but should be cleaned.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-citation of the authors' prior model-based NGD form; data-driven LMI certificates and noise terms are independently derived and do not reduce by construction.

  1. self citation load bearing [Section III.B, equations (24)–(25) and the paragraph preceding Theorem 1]
    "Reference [9] introduced a closed-loop control framework that combines gradient descent with a linear control policy. … A+BK=I−2ΓP, … This formulation ensures that the state evolution mimics a preconditioned gradient descent x_{k+1}=x_k−Γ∇V(x_k). … In the stochastic control setting … the NGD update for the state mean then becomes μ_{k+1}=μ_k−αG(μ_k)^{−1}∇_{μ_k}E[V(x_k)]."

    The precise algebraic target that Theorems 1–2 enforce (X_1G=I−2αΣP) is taken verbatim from the authors' own prior model-based construction rather than re-derived from first principles inside the present paper. While the subsequent LMI certificates are new, the claim that the resulting closed-loop is 'natural-gradient' rests on that self-citation.

full rationale

The paper's central results (Theorems 1–2) construct LMIs that force a data-based closed-loop map to equal the NGD recursion μ_{k+1}=(I−2αΣP)μ_k and then certify λ-contractiveness of that map. The target recursion itself is imported from the authors' earlier model-based papers [9],[10], but once that design choice is made the subsequent algebra (change of variables F=GY, Schur complements for the Lyapunov inequality, convex relaxations M≽GΣG⊤ and Z≽YΣ^{-1}Y, and the extra Tr(·)W noise-propagation terms) is self-contained and does not collapse to a fitted quantity or to an unverified uniqueness claim. α and λ remain free design parameters; no prediction is forced by a data fit. Hardware validation and Monte-Carlo studies are external to the derivation chain. Consequently the only circularity is a non-load-bearing self-citation of the geometric ansatz, scoring 2.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central certificates rest on three classical domain assumptions (controllability, full-rank data, Gaussian process noise), two free design scalars (α, λ), and the optional noise covariance W that may be estimated or hand-tuned. No new physical entities are postulated; the ‘invented’ objects are purely algorithmic (the two data-based NGD parameterizations and their LMI relaxations).

free parameters (3)
  • α (NGD step size)
    Scalar free design parameter that directly sets the contraction rate; chosen by the user and swept in all experiments. Not identified from data.
  • λ (contraction factor)
    User-chosen target in (0,1) that appears in the LMI (28b)/(37); trades feasibility against speed.
  • W (process-noise covariance)
    Assumption 2 explicitly allows W to be either estimated or used as a tuning knob; it enters every covariance LMI and therefore affects the returned gain.
assumptions (5)
  • domain assumption Pair (A,B) is controllable (Assumption 1).
    Standard LTI controllability; required for existence of a stabilizing K but never verified from data.
  • domain assumption Data matrix D_0 has full row rank m+n (Assumption 3).
    Enables the two data-based parameterizations; Lemma 4 only gives a high-probability sample bound under additional Gaussian and closed-loop stability hypotheses.
  • domain assumption Process noise is i.i.d. Gaussian with covariance W (system model (1)).
    Used to obtain the exact FIM G(μ)=Σ^{-1} and the closed-loop covariance recursions (13),(21).
  • ad hoc to paper Convex relaxations M ⪰ GΣGᵀ and Z ⪰ YΣ^{-1}Y are sufficient for the true stationary covariance inequality.
    Introduced in the proof of Theorem 1 to obtain an LMI; tightness is not proved.
  • domain assumption Robot kinematics may be replaced by the constant-A=I linearization about ϕ≈0 (eqs. 50–51).
    All hardware and Gazebo claims rest on this approximation; large heading errors invalidate the certificates.
invented entities (1)
  • Uncertainty-aware data-based closed-loop covariance (eqs. 13 and 21)
    purpose: Propagates the unknown noise matrix Ω_0 through the data map so that the FIM used by NGD reflects both process and data uncertainty.
    Extends earlier data-driven covariance formulae by retaining the Tr(GΣGᵀ)W term; purely algebraic, no new physical object.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Direct Data Driven Natural Gradient Descent for Control." pith.science (2026). https://pith.science/paper/ZVBFICTS

@misc{pith2026260703393,
  author       = {Pith},
  title        = {Pith review of: Direct Data Driven Natural Gradient Descent for Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZVBFICTS}},
  note         = {Machine review of arXiv:2607.03393}
}
read the original abstract

This paper introduces a novel direct data-driven control framework based on Natural Gradient Descent (NGD) to design interpretable and robust closed-loop policies without requiring explicit model identification. We propose two data-driven NGD formulations that incorporate the closed-loop covariance matrix through the Fisher Information Matrix (FIM), allowing gradient updates to be preconditioned according to the system's intrinsic uncertainty. Leveraging two distinct data-based parameterizations of the closed-loop system, our method enables stability-guaranteed policy synthesis directly from data. We provide theoretical guarantees for contraction and convergence using semidefinite programs (SDPs) and validate our framework in both simulations and on hardware on a ROSbot XL platform. The results demonstrate intuitive features compared to linear-quadratic regulator (LQR) and standard data-driven baselines, particularly in terms of convergence speed, robustness, and control interpretability. This work bridges the gap between trajectory-oriented natural gradient methods and practical data-driven control design.

Figures

Figures reproduced from arXiv: 2607.03393 by the authors.

Figure 1
Figure 1. Comparison of controller behavior on the real robot [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. The physical (left) and simulated ROSbot XL platform [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Data collected from the Gazebo simulation. Left: robot [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Control inputs (wheel speeds) ω1 to ω4 applied to the robot in Gazebo during data collection. The inputs were se￾lected from structured translation-only (with slight rotational) patterns to preserve the validity of the linear model. • Mean: The sample mean at time step…
Figure 7
Figure 7. Figure 7: Monte Carlo x-y trajectory and heading for α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 10
Figure 10. Figure 10: Wheel speeds obtained for various α in Theorem 1 [7] R. Padmanabhan and P. Seiler, “Analysis of gradient descent with vary￾ing step sizes using integral quadratic constraints,” IEEE Transactions on Automatic Control, 2024. [8] S. S. Nayyer, G. Revati, S. Wagh, and N. …
Figure 13
Figure 13. Figure 13: Robot trajectory obtained for various Q and R in DDLQR [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 14
Figure 14. Figure 14: Wheel speeds obtained for various Q and R in DDLQR Farnaz Adib Yaghmaie is an Assistant Pro￾fessor in the Department of Electrical Engineering at Linkoping University, Sweden. She earned her ¨ Ph.D. in Electrical and Electronic Engineering from Nanyang Technological U…
Figure 15
Figure 15. Figure 15: Data collected from the real robot. Left: robot tra [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: Control inputs (wheel speeds) ω1 to ω4 applied to the robot in real-world experiment during data collection. The inputs were selected from structured translation-only (with slight rotational) patterns to preserve the validity of the linear model. Hamidreza Modares rec…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 3 linked inside Pith

  1. [1]

    An overview of gradient descent optimization algorithms,

    S. Ruder, “An overview of gradient descent optimization algorithms,” 2017

    2017

  2. [2]

    A lyapunov analysis for accelerated gra- dient methods: From deterministic to stochastic case,

    M. Laborde and A. Oberman, “A lyapunov analysis for accelerated gra- dient methods: From deterministic to stochastic case,” inInternational Conference on Artificial Intelligence and Statistics, pp. 602–612, PMLR, 2020

    2020

  3. [3]

    New insights and perspectives on the natural gradient method,

    J. Martens, “New insights and perspectives on the natural gradient method,”Journal of Machine Learning Research, vol. 21, no. 146, pp. 1– 76, 2020

    2020

  4. [4]

    Defining and characterizing reward gaming,

    J. Skalse, N. Howe, D. Krasheninnikov, and D. Krueger, “Defining and characterizing reward gaming,”Advances in Neural Information Processing Systems, vol. 35, pp. 9460–9471, 2022

    2022

  5. [5]

    Opti- mization algorithms as robust feedback controllers,

    A. Hauswirth, Z. He, S. Bolognani, G. Hug, and F. D ¨orfler, “Opti- mization algorithms as robust feedback controllers,”Annual Reviews in Control, vol. 57, p. 100941, 2024

    2024

  6. [6]

    Analysis and design of opti- mization algorithms via integral quadratic constraints,

    L. Lessard, B. Recht, and A. Packard, “Analysis and design of opti- mization algorithms via integral quadratic constraints,”SIAM Journal on Optimization, vol. 26, no. 1, pp. 57–95, 2016. Fig. 9: Robot trajectory obtained for variousαin Theorem 1 Fig. 10: Wheel speeds obtained for variousαin Theorem 1

    2016

  7. [7]

    Analysis of gradient descent with vary- ing step sizes using integral quadratic constraints,

    R. Padmanabhan and P. Seiler, “Analysis of gradient descent with vary- ing step sizes using integral quadratic constraints,”IEEE Transactions on Automatic Control, 2024

    2024

  8. [8]

    Passivity and immersion based-modified gradient estimator: A control perspective in parameter estimation,

    S. S. Nayyer, G. Revati, S. Wagh, and N. M. Singh, “Passivity and immersion based-modified gradient estimator: A control perspective in parameter estimation,”arXiv preprint arXiv:2211.10674, 2022

    arXiv 2022

Show all 28 references
  1. [9]

    Trajectory-oriented control using gradient descent: An unconventional approach,

    R. Esmzad and H. Modares, “Trajectory-oriented control using gradient descent: An unconventional approach,” 2024

    2024

  2. [10]

    Natural gradient descent for control,

    R. Esmzad, F. Adib Yaghmaie, and H. Modares, “Natural gradient descent for control,”ASME Letters in Dynamic Systems and Control, vol. 6, no. 1, p. 011006, 2026

    2026

  3. [11]

    Low-complexity learning of linear quadratic regulators from noisy data,

    C. De Persis and P. Tesi, “Low-complexity learning of linear quadratic regulators from noisy data,”Automatica, vol. 128, p. 109548, 2021

    2021

  4. [12]

    Bridging direct and indirect data-driven control formulations via regularizations and relaxations,

    F. D ¨orfler, J. Coulson, and I. Markovsky, “Bridging direct and indirect data-driven control formulations via regularizations and relaxations,” IEEE Transactions on Automatic Control, vol. 68, no. 2, pp. 883–897, 2022

    2022

  5. [13]

    Direct data-driven discounted infinite hori- zon linear quadratic regulator with robustness guarantees,

    R. Esmzad and H. Modares, “Direct data-driven discounted infinite hori- zon linear quadratic regulator with robustness guarantees,”Automatica, vol. 175, p. 112197, 2025

    2025

  6. [14]

    Natural gradient works efficiently in learning,

    S.-i. Amari, “Natural gradient works efficiently in learning,”Neural Computation, vol. 10, no. 2, pp. 251–276, 1998

    1998

  7. [15]

    Data-enabled policy optimization for direct adaptive learning of the lqr,

    F. Zhao, F. D ¨orfler, A. Chiuso, and K. You, “Data-enabled policy optimization for direct adaptive learning of the lqr,”arXiv preprint arXiv:2401.14871, 2024

    2024 arXiv

  8. [16]

    A new approach to linear filtering and prediction problems,

    R. E. Kalman, “A new approach to linear filtering and prediction problems,”Journal of Basic Engineering, vol. 82, pp. 35–45, 03 1960

    1960

  9. [17]

    Blanchini and S

    F. Blanchini and S. Miani,Set-theoretic methods in control, vol. 78. Springer, 2008

    2008

  10. [18]

    Covariance control theory,

    A. Hotz and R. E. Skelton, “Covariance control theory,”International Journal of Control, vol. 46, no. 1, pp. 13–32, 1987

    1987

  11. [19]

    A data-driven riccati equation,

    A. Rantzer, “A data-driven riccati equation,” in6th Annual Learning for Dynamics & Control Conference, pp. 504–513, PMLR, 2024

    2024

  12. [20]

    Linear quadratic dual control,

    A. Rantzer, “Linear quadratic dual control,” 2024

    2024

  13. [21]

    Finite-time identification of stable linear systems: Optimality of the least-squares estimator,

    Y . Jedra and A. Prouti `ere, “Finite-time identification of stable linear systems: Optimality of the least-squares estimator,”arXiv:2003.07937, 2020

    2003 arXiv

  14. [22]

    Learning without mixing: Towards a sharp analysis of linear system identifi- cation,

    M. Simchowitz, H. Mania, S. Tu, M. I. Jordan, and B. Recht, “Learning without mixing: Towards a sharp analysis of linear system identifi- cation,” inProceedings of the 31st Conference On Learning Theory (S. Bubeck, V . Perchet, and P. Rigollet, eds.), vol. 75 ofProceedings of ...

    2018

  15. [23]

    CVXPY: A Python-embedded modeling lan- guage for convex optimization,

    S. Diamond and S. Boyd, “CVXPY: A Python-embedded modeling lan- guage for convex optimization,”Journal of Machine Learning Research, vol. 17, no. 83, pp. 1–5, 2016

    2016

  16. [24]

    A rewriting system for convex optimization problems,

    A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd, “A rewriting system for convex optimization problems,”Journal of Control and Decision, vol. 5, no. 1, pp. 42–60, 2018

    2018

  17. [25]

    ApS,MOSEK Optimizer API for Python 9.3.22, 2022

    M. ApS,MOSEK Optimizer API for Python 9.3.22, 2022

    2022

  18. [26]

    Finite-time pure pursuit guidance control of a four mecanum wheeled mobile robot with active disturbance rejection,

    N. Banik, J. Ghommam, and M. H. Rahman, “Finite-time pure pursuit guidance control of a four mecanum wheeled mobile robot with active disturbance rejection,”IEEE Access, vol. 13, pp. 39214–39234, 2025

    2025

  19. [27]

    Path-following control of mecanum-wheels omnidirectional mobile robots using nonsingular terminal sliding mode,

    Z. Sun, H. Xie, J. Zheng, Z. Man, and D. He, “Path-following control of mecanum-wheels omnidirectional mobile robots using nonsingular terminal sliding mode,”Mechanical Systems and Signal Processing, vol. 147, p. 107128, 2021

    2021

  20. [28]

    Kinematic modeling for feedback control of an omnidirectional wheeled mobile robot,

    P. Muir and C. Neuman, “Kinematic modeling for feedback control of an omnidirectional wheeled mobile robot,” inProceedings. 1987 IEEE International Conference on Robotics and Automation, vol. 4, pp. 1772– 1778, 1987. Ramin Esmzadreceived his Ph.D. in Mechanical Engineering fro...

    1987

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.