Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a deep-learning reduced-order feedback controller, trained offline on adjoint-generated optimal trajectories, retrieves near-optimal controls in real time from any observed state and scenario, and can keep…

desk verdict A useful extension of the authors' open-loop DL-ROM controller to feedback and latent-loop settings, with solid numerical evidence but no closure guarantee or artifact release. read the letter →

arxiv 2412.09942 v1 pith:QE5WIGRR submitted 2024-12-13 math.OC cs.LGcs.NAmath.NA

Matteo Tomasetto , Francesco Braghin , Andrea Manzoni This is my paper · ORCID
classification math.OCcs.LGcs.NAmath.NA MSC 49K2093B5268T0765M60
keywords feedbackcontrolPDE-constrainedoptimaldeeplearning-basedreducedordermodelsparametricsystemsproperorthogonaldecompositionautoencoderslatentlooptransport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that optimal feedback control of high-dimensional distributed systems governed by parametrized PDEs can be made real-time by moving the expensive computation offline. The proposed pipeline generates optimal state-control trajectories with an adjoint-based solver, compresses states and distributed controls into small latent coordinates using POD combined with a deep autoencoder, and trains a feedforward network to approximate the map from current state and scenario to optimal control. Once trained, the control for a new state and scenario is a few forward passes through small networks, with no online optimization. The same latent space also contains a learned forward model, so if full-order measurements are missing the controller can keep issuing controls from predicted latent states. On two two-dimensional optimal-transport problems the resulting feedback laws reach the target with control errors around seven percent and run roughly one thousand to tens of thousands of times faster than full-order solves.

What carries the argument

The machinery is a two-stage reduction combined with joint training. State and control snapshots are first projected onto POD modes and then encoded and decoded by deep autoencoders, yielding latent coordinates $y_N=\varphi^y_E(V_y^\top y_h)$ and $u_N=\varphi^u_E(V_u^\top u_h)$; the surrogate policy $\pi_N$ is a feedforward network mapping latent state and scenario parameters to latent control. When measurements are missing, a second feedforward network $\varphi_N$ maps $(y_N,u_N,\mu_s)$ to the next latent state. All networks are trained together by minimizing one cumulative loss that combines state and control reconstruction errors, the policy's latent-space and after-decoding prediction errors, and, in the latent-loop variant, one-step forward prediction errors computed from both data and policy outputs. This joint objective is what makes the latent coordinates meaningful for control rather than for reconstruction alone, and it is the mechanism that lets a policy in ten to eighteen dimensions replace a high-dimensional optimal-control solve.

What would settle it

One concrete check: deploy the trained full-order feedback controller on a test scenario not used in training, apply each predicted control to the high-fidelity PDE, and compute the probability of arrival $P(Y(T)\in B_{0.5}(\mu^d_1,\mu^d_2))$; if this probability falls well below the noise-free test levels for both the full-order loop and the latent loop, or if the latent-loop trajectory diverges from the full-order-feedback trajectory while the one-step forward errors remain at the paper's reported few-percent levels, the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that the full-order policy $\pi_h: \mathbb{R}^{N^y_h}\times\mathcal{P}\to\mathbb{R}^{N^u_h}$ can be replaced, for online purposes, by a latent policy $\pi_N: \mathbb{R}^{N_y}\times\mathcal{P}\to\mathbb{R}^{N_u}$ trained on optimal state-control pairs produced by the adjoint method. The paper shows that after generating $N_s$ optimal trajectories offline, a POD-plus-autoencoder reduction to latent dimensions around $N_y\approx 10$–$14$ and $N_u\approx 14$–$18$, followed by a small feedforward network, predicts the distributed velocity-field control for unseen initial states and scenarios with full-order relative errors around $7\%$. It further claims that a concurrently trained forward map $\varphi_N$ at the latent level can close the feedback loop when full-order measurements are unavailable, so control continues from predicted latent states rather than measured ones. Together these pieces make real-time feedback control of high-dimensional parametrized transport-dominated systems a matter of a few forward passes through light neural networks.

Load-bearing premise

The load-bearing premise is that the online closed loop stays inside the training distribution: states reached under the policy's own controls must resemble the optimal trajectories on which $\pi_N$ and $\varphi_N$ were trained, and the forward model's one-step errors must not compound over the control horizon.

Editorial extensions

If this is right

  • If the central claim is correct, optimal closed-loop control for these systems no longer requires solving an optimization problem online; each control step is a forward pass through small encoder, policy, and decoder networks.
  • The latent feedback loop would let a controller keep issuing controls after sensor loss or during measurement latency, since the forward model supplies the next state from the control it just issued.
  • A single trained architecture covers multiple scenarios encoded in $\mu_s$—different target locations and, in the fluid test, different inflow speeds and angles—so new cases are handled without retraining.
  • Because the state and control autoencoders discard high-frequency components, the policy should remain usable with noisy state measurements, as demonstrated by the near-target arrival probabilities in the noise experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to monitor the latent forward model's error online and switch back to full-order feedback when its prediction drifts; the paper does not provide such an error-bounding mechanism.
  • Because the policy is trained on optimal trajectories sampled from chosen initial-state and scenario distributions, active or adaptive sampling in the offline phase would be the direct test of whether the method resists distribution shift when the closed-loop states differ from the training states.
  • The encoder could be retrained on partial observations $z_h = O_h y_h$, turning the same architecture into a sensor-based controller for camera or sparse-sensor data, which the paper lists as a future direction.
  • The training-data generation remains expensive, so combining a few high-fidelity trajectories with many cheap surrogate-generated ones—an idea the paper mentions for future work—would determine whether the offline cost can be amortized to practical levels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes an offline-online reduced-order feedback control strategy for PDE-constrained optimal control problems with distributed state and control variables. During the offline phase, optimal state-control trajectories are generated for sampled initial conditions and scenario parameters using the adjoint method; the snapshots are reduced via POD followed by an autoencoder (POD+AE); and a feedforward policy is trained to map low-dimensional state and scenario parameters to low-dimensional controls. An optional latent forward model is trained and used to close the loop when full-order state measurements are unavailable. Two Fokker-Planck optimal transport problems (in vacuum and in a fluid with an obstacle) are used to evaluate reconstruction, one-step policy prediction, forward-model prediction, noise robustness, and online speedups, with reported per-snapshot errors around 4–7% and speedups of 1000×–32000×.

Significance. The main potential contribution is a non-intrusive, real-time feedback control pipeline for high-dimensional parametrized PDEs, combining POD+AE reduction with a learned policy and a latent forward model. The numerical experiments are nontrivial and include a noise robustness study over 100 scenarios. If the closed-loop performance were demonstrated against full-order optimal feedback benchmarks, this would be a practical and useful advance over MPC and DRL baselines. However, the current evidence is primarily one-step supervised errors on test snapshots and a few qualitative closed-loop trajectories; the claims of 'optimal' control and of preservation of stability and performance exceed what is demonstrated.

major comments (3)
  1. [Section 3.2 and 3.3, Eqs. (5), (7)] The trained policy πN and forward model φN are evaluated only through one-step supervised losses on test snapshots, while the online phase in Section 3.2 and the latent loop in Section 3.3 apply them recursively to states produced by the policy itself. No analysis or experiment addresses the resulting distribution shift or the compounding of forward-model errors over Nt steps; the reported 4–7% errors are therefore not evidence for the abstract's claim that 'optimal control actions are retrieved in real-time for any observed state', nor for the conclusion's claim that the latent loop avoids 'losses of optimality, performance or stability'. Please provide closed-loop rollouts for multiple unseen scenarios, quantify the distance from the full-order optimal closed-loop trajectory or cost, and report multi-step latent prediction error (e.g., error after k steps without resetting).
  2. [Section 4.1 and 4.2 (speed-up claims)] The speed-up comparison is not against a full-order feedback controller. The online times are compared with the time to solve a single open-loop OCP (15 or 22 minutes per scenario), whereas a true full-order feedback loop would require repeated OCP solves at every time step. Please clarify the baseline and, ideally, compare the closed-loop performance and cost with a receding-horizon FOM controller for at least one scenario.
  3. [Section 4.1 and 4.2 (statistical evaluation)] All reported errors (e.g., εu_rel = 7.08% in Test 1.2) come from a single train-test split and a single initialization. Since the data generation and network initialization are random, the paper should report variance over several independent runs or data splits; otherwise the claimed accuracy levels and the noise-robustness conclusions cannot be assessed for statistical significance.
minor comments (5)
  1. [Section 4.2] The phrase 'Fokker-Plank' should be corrected to 'Fokker-Planck'.
  2. [Section 4 (mean relative errors)] The definitions of εy_rel and εu_rel do not specify whether the norm is the Euclidean norm and which time instants are included; please clarify the notation.
  3. [Eq. (5)] The after-decoding error term uses φu_D on both the target and the prediction; consider writing the prediction more explicitly as φu_D(πN(y_N, µ_s)) to avoid ambiguity.
  4. [Figures 9 and 14] The closed-loop results are presented only qualitatively; please add quantitative metrics (e.g., ||y(T)-y_d|| or arrival probability) for these specific test cases.
  5. [Section 3.3] The index set I'_train is introduced but not fully characterized; please state whether it includes all intermediate time steps or only a subset.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the policy and latent forward model are supervised surrogates trained on adjoint-generated optimal pairs and evaluated on held-out data; the main validity gap is distribution shift, not circularity.

full rationale

The paper's derivation chain is not circular. In Section 3.1, optimal state-control pairs are generated by solving the full-order OCP (Eq. 1) through the KKT/adjoint system (Eq. 3); the policy πN is then trained by minimizing the supervised loss JπN (Eq. 5) on those pairs, with state/control autoencoders fitted via reconstruction losses. This is a standard supervised approximation of the policy map in Eq. (2), not an equation that assumes the conclusion. The reported errors (e.g., 4.28% latent / 7.09% decoded policy error in Test 1.1; 3.62%/7.08% in Test 1.2) are computed on held-out test snapshots, so they are out-of-sample predictions rather than fitted data renamed as predictions. The latent forward model φN is trained with one-step prediction losses (Eq. 7), including prediction-from-data and prediction-from-policy terms; the online latent loop (Section 3.3) iterates this map, but that is an extrapolation whose accuracy is not guaranteed by the training loss, not a circular reduction. Self-citations ([25,26,27,72]) are used to position the method as an extension of prior ROM/control work, not as a uniqueness theorem or as the sole justification for the central approximation; no load-bearing claim reduces to a self-citation. The skeptic concern about policy-induced state distribution shift and lack of closed-loop stability is a real correctness/robustness limitation, but it is not a case where the prediction is equivalent to its input by construction. Therefore the appropriate finding is no significant circularity (score 0).

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The ledger shows that the method's performance depends on a substantial number of manually chosen hyperparameters, POD mode counts, and network sizes, as well as on assumptions about the sufficiency of the latent representation and the generalization of the trained policy. These choices are not derived from theory, which is typical for deep-learning ROM methods but should be disclosed when judging the strength of the numerical claims.

free parameters (6)
  • POD mode counts = 150 state / 160 control (Test 1), 200 state / 160 control (Test 2)
    Chosen by inspecting singular value decays and reconstruction errors; not derived from first principles.
  • Latent dimensions after autoencoder = Ny=10, Nu=18 (Test 1); Ny=14, Nu=14 (Test 2)
    Set by hand as a trade-off between reconstruction accuracy and network size; central to the claimed speedups.
  • Loss weights λ1, λ2, λ3 = 0.01 (Test 1), 0.001 (Test 2)
    Chosen to balance reconstruction and policy losses; no automatic tuning procedure is described.
  • Latent-loop loss weights λ4, λ5, λ6 = λ4=λ5=1, λ6=0.001 (Test 1); λ4=λ5=0.01, λ6=0.0001 (Test 2)
    Selected manually to weight forward-model errors; the forward model accuracy depends on these values.
  • Cost functional weights β, βg = 0.2 for both test cases
    Set to balance state tracking against control effort and gradient regularization; influences all generated optimal trajectories.
  • Network architectures = Policy: 3 hidden layers x 50 neurons; AEs: 1-2 hidden layers x 100 neurons
    Chosen by the authors without a systematic architecture search; the reported errors are specific to these sizes.
assumptions (5)
  • standard math Finite element discretization and the adjoint-based KKT system yield accurate optimal trajectories for the Fokker-Planck OCPs.
    The paper relies on FEniCS/dolfin-adjoint to generate all training data; correctness of the optimal trajectories is inherited from these solvers (Section 4.1, 4.2).
  • domain assumption POD plus autoencoder latent coordinates preserve enough information for the policy to compute near-optimal controls.
    The approach assumes that compressing state and control to dimensions 10-18 does not discard control-relevant features; this is validated only empirically on the two test problems.
  • domain assumption A feedforward neural network can approximate the policy and the forward latent dynamics to within a few percent error.
    The method is built on universal-approximation-style assumptions; no approximation-theoretic or data-dependent bound is given for the policy or forward map.
  • domain assumption The offline sampling distribution over initial states and scenario parameters covers the online deployment distribution.
    The paper samples random endpoints and initial positions in fixed intervals; unseen scenarios are assumed to be drawn from the same distribution.
  • domain assumption One-step-ahead training of the latent forward model is sufficient for multi-step closed-loop use.
    The forward model φN is trained to predict y_N(t_j) from y_N(t_{j-1}); the paper does not analyze error accumulation over multiple latent-loop iterations though this is the mechanism that keeps the loop running.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models." pith.science (2026). https://pith.science/paper/QE5WIGRR

@misc{pith2026241209942,
  author       = {Pith},
  title        = {Pith review of: Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QE5WIGRR}},
  note         = {Machine review of arXiv:2412.09942}
}
read the original abstract

Continuous monitoring and real-time control of high-dimensional distributed systems are often crucial in applications to ensure a desired physical behavior, without degrading stability and system performances. Traditional feedback control design that relies on full-order models, such as high-dimensional state-space representations or partial differential equations, fails to meet these requirements due to the delay in the control computation, which requires multiple expensive simulations of the physical system. The computational bottleneck is even more severe when considering parametrized systems, as new strategies have to be determined for every new scenario. To address these challenges, we propose a real-time closed-loop control strategy enhanced by nonlinear non-intrusive Deep Learning-based Reduced Order Models (DL-ROMs). Specifically, in the offline phase, (i) full-order state-control pairs are generated for different scenarios through the adjoint method, (ii) the essential features relevant for control design are extracted from the snapshots through a combination of Proper Orthogonal Decomposition (POD) and deep autoencoders, and (iii) the low-dimensional policy bridging latent control and state spaces is approximated with a feedforward neural network. After data generation and neural networks training, the optimal control actions are retrieved in real-time for any observed state and scenario. In addition, the dynamics may be approximated through a cheap surrogate model in order to close the loop at the latent level, thus continuously controlling the system in real-time even when full-order state measurements are missing. The effectiveness of the proposed method, in terms of computational speed, accuracy, and robustness against noisy data, is finally assessed on two different high-dimensional optimal transport problems, one of which also involving an underlying fluid flow.

Figures

Figures reproduced from arXiv: 2412.09942 by the authors.

Figure 1
Figure 1. Feedback control scheme considering multiple optimal control problem resolutions and a high-fidelity full-order [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Offline phase of the deep learning-based reduced order feedback controller. After generating optimal state and control [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Online phase of the deep learning-based reduced order feedback controller. The optimal full-order control action [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Online phase of the deep learning-based reduced order feedback controller with latent feedback loop. The optimal [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Test 1.1. Optimal transport in a vacuum. Top left: representation of an optimal state trajectory in a vacuum within the domain Ω, where y0 stands for the initial density centered at (µ 0 1 , µ0 2 ) = (−0.45, 0.21), while yd(µs ) represents the target configuration cent…
Figure 6
Figure 6. Figure 6: Test 1.1. Optimal transport in a vacuum. Singular values decay in log-log scale along with the two most energetic POD modes related to the state (top), x1 component (center) and x2 component (bottom) of the control. The state and control POD coefficients can be further…
Figure 7
Figure 7. Figure 7: Test 1.1. Optimal transport in a vacuum. High-fidelity optimal control trajectory (first row), POD+AE reconstructions (second row), policy predictions (third row), POD+AE reconstruction errors (fourth row) and policy prediction errors (fifth row) at t = 0, 0.25, 0.5, 0…
Figure 8
Figure 8. Figure 8: Test 1.1. Optimal transport in a vacuum. High-fidelity optimal state trajectory (first row), POD+AE reconstructions (second row), forward model predictions (third row), POD+AE reconstruction errors (fourth row) and forward model prediction errors (fifth row) at t = 0.2…
Figure 9
Figure 9. Figure 9: Test 1.1. Optimal transport in a vacuum. First row: discrepancy between the target configuration yd centered at (µ d 1 , µd 2 ) = (0.48, −0.03) and the state yh(t) considering the deep learning-based reduced order feedback controller (left) and the latent feedback loop…
Figure 10
Figure 10. Figure 10: Test 1.2. Optimal transport in a fluid. Top: representation of an optimal state trajectory in a fluid within the domain Ω, where y0 stands for the initial density centered at (µ 0 1 , µ0 2 ) = (−0.5, −0.25), yd(µs ) represents the target configuration centered at (µ d…
Figure 11
Figure 11. Figure 11: Test 1.2. Optimal transport in a fluid. Singular values decay in log-log scale along with the two most energetic POD modes related to the state (top), x1 component (center) and x2 component (bottom) of the control. with 100 neurons each. The low-dimensional policy πN …
Figure 12
Figure 12. Figure 12: Test 1.2. Optimal transport in a fluid. High-fidelity optimal control trajectory (first row), POD+AE reconstructions (second row), policy predictions (third row), POD+AE reconstruction errors (fourth row) and policy prediction errors (fifth row) at t = 0, 0.25, 0.5, 0…
Figure 13
Figure 13. Figure 13: Test 1.2. Optimal transport in a fluid. High-fidelity optimal state trajectory (first row), POD+AE reconstructions (second row), forward model predictions (third row), POD+AE reconstruction errors (fourth row) and forward model prediction errors (fifth row) at t = 0.2…
Figure 14
Figure 14. Figure 14: Test 1.2. Optimal transport in a fluid. First and second rows: system evolution driven by policy-based controls at t = 0, 0.25, 0.75, 1.0, 1.5 related to an initial state centered at (µ 0 1 , µ0 2 ) = (−0.5, 0.0) and a vector of scenario parameters µs = (0.0, 0.5, 0.5…
Figure 15
Figure 15. Figure 15: Test 1.2. Optimal transport in a fluid. First row: boxplots of the probabilities of arrival in 100 random scenarios for different noise levels when considering the deep learning-based reduced order feedback controller at full-order level (left) and the latent feedback…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Toward Goal-Agnostic Joint-Embedding Predictive Control of Partial Differential Equations

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A goal-agnostic latent-dynamics controller for 2D Navier-Stokes improves tracking by planning against a learned kinetic-energy probe rather than raw latent-space distance.

Reference graph

Works this paper leans on

81 extracted references · 71 canonical work pages · cited by 1 Pith paper

  1. [1]

    Aggelogiannaki and H

    E. Aggelogiannaki and H. Sarimveis. Nonlinear model predictive control for distributed parameter systems using data driven artificial neural network models. Computers and Chemical Engineering, 32(6):1225–1237, 2008

    2008

  2. [2]

    G. Albi, S. Bicego, and D. Kalise. Control of high-dimensional collective dynamics by deep neural feedback laws and kinetic modelling. arXiv:2404.02825, 2024

    work page Pith review arXiv 2024

  3. [3]

    A. Alla, B. Haasdonk, and A. Schmidt. Feedback control of parametrized PDEs via model order reduction and dynamic programming principle. Advances in Computational Mathematics, 46(9), 2020

    work page 2020

  4. [4]

    Alla and M

    A. Alla and M. Hinze. HJB-POD feedback control for navier-stokes equations. In Progress in Industrial Mathematics at ECMI 2014, pages 861–868, Cham, 2016. Springer International Publishing

    work page 2014

  5. [5]

    Alla and S

    A. Alla and S. V olkwein. Asymptotic stability of POD based model predictive control for a semilinear parabolic PDE. Advances in Computational Mathematics, 41(5):1073–1102, 2015

    work page 2015

  6. [6]

    Alnæs, J

    M. Alnæs, J. Blechta, J. Hake, A. Johansson, B. Kehlet, A. Logg, C. Richardson, J. Ring, M. Rognes, and G. Wells. The fenics project version 1.5. Archive of Numerical Software, 3(100):9–23, 2015

    work page 2015

  7. [7]

    Amsallem, M

    D. Amsallem, M. Zahr, Y . Choi, and C. Farhat. Design optimization using hyper-reduced-order models.Structural and Multidisciplinary Optimization, 51(4):919–940, 2015

    work page 2015

  8. [8]

    E. A. Antonelo, E. Camponogara, L. O. Seman, J. P. Jordanou, E. R. de Souza, and J. F. Hübner. Physics-informed neural nets for control of dynamical systems. Neurocomputing, 579:127419, 2024

    work page 2024

Show all 81 references
  1. [9]

    Bardi and I

    M. Bardi and I. Capuzzo-Dolcetta. Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations . Birkhäuser Boston, MA, 2009

    2009

  2. [10]

    R. Bellman. Dynamic Programming. Princeton University Press, Princeton, NJ, USA, 1 edition, 1957

    1957

  3. [11]

    Benner, E

    P. Benner, E. Sachs, and S. V olkwein. Model order reduction for PDE constrained optimization.International Series of Numerical Mathematics, 165:303–326, 2014

    2014

  4. [12]

    Bieker, S

    K. Bieker, S. Peitz, S. L. Brunton, J. N. Kutz, and M. Dellnitz. Deep model predictive flow control with limited sensor data and online learning. Theoretical and Computational Fluid Dynamics, 34(4):577–591, 2020. 26 Latent feedback control of distributed systems in multiple sc...

    2020

  5. [13]

    Biferale, F

    L. Biferale, F. Bonaccorso, M. Buzzicotti, P. Clark Di Leoni, and K. Gustavsson. Zermelo’s problem: Optimal point-to-point navigation in 2D turbulent flows using reinforcement learning. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(10):103138, 10 2019

    2019

  6. [14]

    Botteghi, K

    N. Botteghi, K. Alaa, M. Poel, B. Sirmacek, C. Brune, A. Mersha, and S. Stramigioli. Low dimensional state representation learning with robotics priors in continuous action spaces. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 190–197, 2021

    2021

  7. [15]

    Botteghi and U

    N. Botteghi and U. Fasel. Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies. arXiv:2403.15267, 2024

    2024 arXiv

  8. [16]

    Brunton and J

    S. Brunton and J. N. Kutz. Data-driven science and engineering: machine learning, dynamical systems, and control . Cambridge University Press, 2019

    2019

  9. [17]

    S. L. Brunton, M. Budiši´c, E. Kaiser, and J. N. Kutz. Modern Koopman Theory for Dynamical Systems. SIAM Review, 64(2):229–340, 2022

    2022

  10. [18]

    S. L. Brunton, J. L. Proctor, and J. N. Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences of the United States of America, 113(15):3932–3937, 2016

    2016

  11. [19]

    S. L. Brunton, J. L. Proctor, and J. N. Kutz. Sparse Identification of Nonlinear Dynamics with Control (SINDYc). IFAC-PapersOnLine, 49(18):710–715, 2016

    2016

  12. [20]

    M. A. Bucci, O. Semeraro, A. Allauzen, G. Wisniewski, L. Cordier, and L. Mathelin. Control of chaotic systems by deep reinforcement learning. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , 475(2231):20190351, 2019

    2019

  13. [21]

    Busoniu, R

    L. Busoniu, R. Babuska, and B. De Schutter. A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(2):156–172, 2008

    2008

  14. [22]

    Camacho and C

    E. Camacho and C. Bordons. Model Predictive Control. Springer London, 2004

    2004

  15. [23]

    Y . Chen, Y . Shi, and B. Zhang. Optimal Control Via Neural Networks: A Convex Approach. arXiv:1805.11835, 2018

    2018 arXiv

  16. [24]

    Draeger, S

    A. Draeger, S. Engell, and H. Ranke. Model predictive control using neural networks. IEEE Control Systems Magazine, 15(5):61–66, 1995

    1995

  17. [25]

    N. R. Franco, A. Manzoni, and P. Zunino. A Deep Learning approach to Reduced Order Modeling of parameter dependent Partial Differential Equations. Mathematics of Computation, 92(340):483–524, 2023

    2023

  18. [26]

    Fresca, L

    S. Fresca, L. Dede’, and A. Manzoni. A Comprehensive Deep Learning-Based Approach to Reduced Order Modeling of Nonlinear Time-Dependent Parametrized PDEs. Journal of Scientific Computing, 87(2):1–36, 2021

    2021

  19. [27]

    Fresca and A

    S. Fresca and A. Manzoni. POD-DL-ROM: Enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition. Computer Methods in Applied Mechanics and Engineering, 388:114181, 2022

    2022

  20. [28]

    Garnier, J

    P. Garnier, J. Viquerat, J. Rabault, A. Larcher, A. Kuhnle, and E. Hachem. A review on deep reinforcement learning for fluid mechanics. Computers and Fluids, 225, 2021

    2021

  21. [29]

    Geuzaine and J.-F

    C. Geuzaine and J.-F. Remacle. Gmsh: A 3-D finite element mesh generator with built-in pre- and post-processing facilities. International Journal for Numerical Methods in Engineering, 79(11):1309–1331, 2009

    2009

  22. [30]

    Ghiglieri and S

    J. Ghiglieri and S. Ulbrich. Optimal flow control based on POD and MPC and an application to the cancellation of Tollmien–Schlichting waves. Optimization Methods and Software, 29(5):1042–1074, 2014

    2014

  23. [31]

    Guastoni, J

    L. Guastoni, J. Rabault, P. Schlatter, H. Azizpour, and R. Vinuesa. Deep reinforcement learning for turbulent drag reduction in channel flows. The European Physical Journal E, 46(4):27, 2023

    2023

  24. [32]

    K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 1026–1034, 2015

    2015

  25. [33]

    Hesthaven and S

    J. Hesthaven and S. Ubbiali. Non-intrusive reduced order modeling of nonlinear problems using neural networks. Journal of Computational Physics, 363, 2018

    2018

  26. [34]

    J. S. Hesthaven, G. Rozza, and B. Stamm. Certified Reduced Basis Methods for Parametrized Partial Differential Equations. Certified Reduced Basis Methods for Parametrized Partial Differential Equations, pages 1–131, 2015

    2015

  27. [35]

    M. K. Hickner, U. Fasel, A. G. Nair, B. W. Brunton, and S. L. Brunton. Data-Driven Unsteady Aeroelastic Modeling for Control. AIAA Journal, 61(2):780–792, 2023

    2023

  28. [36]

    Hüttenrauch, A

    M. Hüttenrauch, A. Šoši ´c, and G. Neumann. Deep reinforcement learning for swarm systems. J. Mach. Learn. Res., 20(1):1966–1996, jan 2019. 27 Latent feedback control of distributed systems in multiple scenarios through DL-ROMs

    1966

  29. [37]

    Ishize, H

    T. Ishize, H. Omichi, and K. Fukagata. Flow control by a hybrid use of machine learning and control theory. arXiv:2311.08624, 2023

    2023 arXiv

  30. [38]

    J. Jeon, J. Rabault, J. Vasanth, F. Alcántara-Ávila, S. Baral, and R. Vinuesa. Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality. arXiv:2407.17822, 2024

    2024 arXiv

  31. [39]

    Kaiser, J

    E. Kaiser, J. N. Kutz, and S. L. Brunton. Sparse identification of nonlinear dynamics for model predictive control in the low-data limit. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 474(2219), 2018

    2018

  32. [40]

    S. Klus, F. Nüske, S. Peitz, J.-H. Niemann, C. Clementi, and C. Schütte. Data-driven approximation of the Koopman generator: Model reduction, system identification, and control. Physica D: Nonlinear Phenomena, 406:132416, 2020

    2020

  33. [41]

    Korda and I

    M. Korda and I. Mezi´c. Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control. Automatica, 93:149–160, 2018

    2018

  34. [42]

    Kunisch and S

    K. Kunisch and S. V olkwein. Control of the Burgers Equation by a Reduced-Order Approach Using Proper Orthogonal Decomposition. Journal of Optimization Theory and Applications, 102(2):345–371, 1999

    1999

  35. [43]

    Kunisch and S

    K. Kunisch and S. V olkwein. Proper orthogonal decomposition for optimality systems.ESAIM: Modélisation mathématique et analyse numérique, 42(1):1–23, 2008

    2008

  36. [44]

    Kunisch, S

    K. Kunisch, S. V olkwein, and L. Xie. HJB-POD-Based Feedback Design for the Optimal Control of Evolution Problems. SIAM Journal on Applied Dynamical Systems, 3(4):701–722, 2004

    2004

  37. [45]

    Leibfritz and S

    F. Leibfritz and S. V olkwein. Reduced order output feedback control design for PDE systems using proper orthogonal decomposition and nonlinear semidefinite programming. Linear Algebra and its Applications , 415(2):542–575, 2006. Special Issue on Order Reduction of Large-Scale Systems

    2006

  38. [46]

    Levine, A

    S. Levine, A. Kumar, G. Tucker, and F. Fu. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv:2005.01643, 2020

    2005 arXiv

  39. [47]

    D. Luo, T. O’Leary-Roseberry, P. Chen, and O. Ghattas. Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators. arXiv:2305.20053, 2023

    2023 arXiv

  40. [48]

    P. Ma, Y . Tian, Z. Pan, B. Ren, and D. Manocha. Fluid directed rigid body control using deep reinforcement learning.ACM Transactions on Graphics, 37(4), 2018

    2018

  41. [49]

    Manzoni and S

    A. Manzoni and S. Pagani. A certified RB method for PDE-constrained parametric optimization problems.Communications in Applied and Industrial Mathematics, 10(1):123–152, 2019

    2019

  42. [50]

    Manzoni, A

    A. Manzoni, A. Quarteroni, and S. Salsa. Optimal Control of Partial Differential Equations. Springer Cham, 2021

    2021

  43. [51]

    Mirzakhanloo, S

    M. Mirzakhanloo, S. Esmaeilzadeh, and M.-r. Alam. Active cloaking in Stokes flows via reinforcement learning. Journal of Fluid Mechanics, 903(A34), 2020

    2020

  44. [52]

    S. K. Mitusch, S. W. Funke, and J. S. Dokken. dolfin-adjoint 2018.1: automated adjoints for fenics and firedrake. Journal of Open Source Software, 4(38):1292, 2019

    2018

  45. [53]

    Negri, G

    F. Negri, G. Rozza, A. Manzoni, and A. Quarteroni. Reduced basis method for parametrized elliptic optimal control problems. SIAM Journal on Scientific Computing, 35(5):A2316–A2340, 2013

    2013

  46. [54]

    Peitz and K

    S. Peitz and K. Bieker. On the universal transformation of data-driven models to control systems. Automatica, 149:110840, 2023

    2023

  47. [55]

    Peitz and S

    S. Peitz and S. Klus. Koopman operator-based model reduction for switched-system control of PDEs. Automatica, 106:184–191, 2019

    2019

  48. [56]

    Peitz and S

    S. Peitz and S. Klus. Feedback Control of Nonlinear PDEs Using Data-Efficient Reduced Order Models Based on the Koopman Operator, pages 257–282. Springer International Publishing, Cham, 2020

    2020

  49. [57]

    Peitz, J

    S. Peitz, J. Stenner, V . Chidananda, O. Wallscheid, S. L. Brunton, and K. Taira. Distributed control of partial differential equations using convolutional reinforcement learning. Physica D: Nonlinear Phenomena, 461:134096, 2024

    2024

  50. [58]

    Quarteroni

    A. Quarteroni. Numerical Models for Differential Problems. Springer Cham, 2017

    2017

  51. [59]

    Quarteroni, A

    A. Quarteroni, A. Manzoni, and F. Negri. Reduced basis methods for partial differential equations: An introduction . Springer Cham, 2015

    2015

  52. [60]

    Rabault, F

    J. Rabault, F. Ren, W. Zhang, H. Tang, and H. Xu. Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization. Journal of Hydrodynamics, 32(2):234–246, 2020

    2020

  53. [61]

    F. Ren, C. Wang, and H. Tang. Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth. Physics of Fluids, 33(9):093602, 2021. 28 Latent feedback control of distributed systems in multiple scenarios through DL-ROMs

    2021

  54. [62]

    Rosafalco, J

    L. Rosafalco, J. M. De Ponti, L. Iorio, R. V . Craster, R. Ardito, and A. Corigliano. Reinforcement learning optimisation for graded metamaterial design using a physical-based constraint on the state representation and action space. Scientific Reports, 13(1):21836, 2023

    2023

  55. [63]

    P. J. Schmid. Dynamic mode decomposition of numerical and experimental data. Journal of Fluid Mechanics, 656:5–28, 2010

    2010

  56. [64]

    Schmidt and B

    A. Schmidt and B. Haasdonk. Data-driven surrogates of value functions and applications to feedback control for dynamical systems. IFAC-PapersOnLine, 51(2):307–312, 2018. 9th Vienna International Conference on Mathematical Modelling

    2018

  57. [65]

    T. Shah, L. Zhuo, P. Lai, A. De La Rosa-Moreno, F. Amirkulova, and P. Gerstoft. Reinforcement learning applied to metamaterial design. The Journal of the Acoustical Society of America, 150(1):321–338, 2021

    2021

  58. [66]

    Sinigaglia, F

    C. Sinigaglia, F. Braghin, and S. Berman. Optimal Control of Velocity and Nonlocal Interactions in the Mean-Field Kuramoto Model. Proceedings of the American Control Conference, 2022-June:290–295, 2022

    2022

  59. [67]

    Sinigaglia, A

    C. Sinigaglia, A. Manzoni, and F. Braghin. Density Control of Large-Scale Particles Swarm Through PDE-Constrained Optimization. IEEE Transactions on Robotics, 38(6):3530–3549, 2022

    2022

  60. [68]

    Sinigaglia, A

    C. Sinigaglia, A. Manzoni, F. Braghin, and S. Berman. Robust optimal density control of robotic swarms. arXiv:2205.12592, 2022

    2022 arXiv

  61. [69]

    Sinigaglia, D

    C. Sinigaglia, D. E. Quadrelli, A. Manzoni, and F. Braghin. Fast active thermal cloaking through PDE-constrained optimization and reduced-order modelling. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 478(2258), 2022

    2022

  62. [70]

    Sirovich

    L. Sirovich. Turbulence and the dynamics of coherent structures part i: Coherent structures. Quarterly of Applied Mathematics, 45(3):561–571, 1987

    1987

  63. [71]

    R. S. Sutton and A. G. Barto. Reinforcement Learning: an introduction. The MIT Press, 2018

    2018

  64. [72]

    Tomasetto, A

    M. Tomasetto, A. Manzoni, and F. Braghin. Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models. arXiv:2409.05709, 2024

    2024 arXiv

  65. [73]

    Tonti, J

    F. Tonti, J. Rabault, and R. Vinuesa. Navigation in a simplified urban flow through deep reinforcement learning. arXiv:2409.17922, 2024

    2024 arXiv

  66. [74]

    Varela, P

    P. Varela, P. Suárez, F. Alcántara-Ávila, A. Miró, J. Rabault, B. Font, L. M. García-Cuevas, O. Lehmkuhl, and R. Vinuesa. Deep reinforcement learning for flow control exploits different physics for increasing reynolds number regimes. Actuators, 11(12), 2022

    2022

  67. [75]

    Vasanth, J

    J. Vasanth, J. Rabault, F. Alcántara-Ávila, M. Mortensen, and R. Vinuesa. Multi-agent reinforcement learning for the control of three-dimensional rayleigh-bénard convection. arXiv:2407.21565, 2024

    2024 arXiv

  68. [76]

    Verma, G

    S. Verma, G. Novati, and P. Koumoutsakos. Efficient collective swimming by harnessing vortices through deep reinforcement learning. Proceedings of the National Academy of Sciences of the United States of America, 115(23):5849–5854, 2018

    2018

  69. [77]

    Vignon, J

    C. Vignon, J. Rabault, J. Vasanth, F. Alcántara-Ávila, M. Mortensen, and R. Vinuesa. Effective control of two-dimensional Rayleigh–Bénard convection: Invariant multi-agent reinforcement learning is all you need. Physics of Fluids, 35(6):065146, 2023

    2023

  70. [78]

    Vignon, J

    C. Vignon, J. Rabault, and R. Vinuesa. Recent advances in applying deep reinforcement learning for flow control: Perspectives and future directions. Physics of Fluids, 35(3), 2023

    2023

  71. [79]

    Vinuesa, O

    R. Vinuesa, O. Lehmkuhl, A. Lozano-Durán, and J. Rabault. Flow control in wings and discovery of novel approaches via deep reinforcement learning. Fluids, 7(2), 2022

    2022

  72. [80]

    Z. Zhou, S. Kearnes, L. Li, R. N. Zare, and P. Riley. Optimization of Molecules via Deep Reinforcement Learning. Scientific Reports, 9(1), 2019

    2019

  73. [81]

    Zolman, U

    N. Zolman, U. Fasel, J. N. Kutz, and S. L. Brunton. SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning. arXiv:2403.09110, 2024. 29

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.