REVIEW 4 major objections 5 minor 1 cited by
DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper introduces DeePC-Hunt, a backpropagation method that tunes the regularization hyperparameters of Data-enabled Predictive Control on an approximate model so that they work on the true closed-loop system.
desk verdict Good idea and clean implementation, but the missing untuned-DeePC baseline means the paper never shows the tuning step is what helps. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the DeePC solution map $\pi_\lambda$, which maps the regularization vector $\lambda=(\lambda_0,\lambda_1,\lambda_2)$ and the recent input/output history to the next control action; because the underlying problem is a disciplined parameterized convex program, this map can be differentiated with respect to $\lambda$ by differentiating the KKT conditions. The mechanism is the bilevel optimization problem that minimizes the expected closed-loop cost of $\pi_\lambda$ simulated on the surrogate model, solved with a projected resilient backpropagation algorithm that uses only gradient signs and per-coordinate adaptive step sizes. The specific object being tuned is the regularization function $\lambda_0|(I-\Pi)g|_2^2 + \lambda_1|g|_1 + \lambda_2|\sigma_y|_1$.
What would settle it
Take the VTVL benchmark, keep everything fixed, and sweep surrogate mismatch continuously by inflating $l_1$, shrinking $l_2$, and changing the mass parameter by increasing amounts; record the true-system success rate of the resulting tuned policies. If success-rate degrades sharply to the MPC (B) level for a mismatch well within the range a user might reasonably call approximate, the transfer claim is falsified; the paper currently reports only one moderately mismatched model.
Extended reading notes
Core claim
The paper's central claim is that hyperparameters minimizing the expected closed-loop cost of a DeePC policy on an approximate surrogate model also deliver strong closed-loop performance on the true system. To make this computable, DeePC is recast as a convex-optimization policy whose solution map is differentiable with respect to the regularization vector via the implicit function theorem, so the gradient of the closed-loop cost can be obtained by backpropagation. A constrained variant of resilient backpropagation then updates the three regularization weights, including projections to keep them non-negative. In the VTVL landing task, both an accurate and a deliberately mis-estimated surrogate produced near-identical DeePC-Hunt landing trajectories, and the policy tuned through the bad model outperformed MPC using the same bad model on success rate.
Load-bearing premise
The load-bearing premise is that regularization parameters that minimize the closed-loop cost on the approximate surrogate model are also good for the true system, an assumption the paper motivates only by the heuristic that similar systems have similar optimal parameters.
Editorial extensions
If this is right
- DeePC users can tune regularization parameters entirely offline, without risky or expensive experiments on the real system, as long as an approximate model is available.
- Under model mismatch, the DeePC-Hunt policy can remain effective where a linearized MPC using the same mismatched model fails, because receding-horizon replanning absorbs much of the model error.
- The differentiable-policy viewpoint opens DeePC to gradient-based optimization of not just regularization weights but any problem parameter that enters the convex program.
- The constrained resilient backpropagation update is applicable to any hyperparameter search with box or cone constraints, not only to DeePC.
Reading between the lines
- A natural extension the authors leave implicit is to tune the cost matrices $Q$ and $R$, the horizons $T_{ini}$ and $T_f$, or the reference trajectory distribution through the same pipeline, since all of these enter the same differentiable solution map.
- The transfer claim could be stress-tested by plotting tuned-$\lambda$ performance against a continuous measure of surrogate mismatch; the paper reports two snapshots (accurate and moderately wrong models) rather than a failure boundary.
- If the surrogate is chosen so that its closed-loop cost is an upper bound on the true cost, then minimization of the surrogate cost would carry a formal transfer guarantee; the paper relies instead on empirical similarity without proving such a bound.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DeePC-Hunt, an offline hyperparameter-tuning method for Data-enabled Predictive Control (DeePC). The method interprets the DeePC optimization as a differentiable policy, backpropagates the closed-loop cost on an approximate surrogate model through the policy, and optimizes the regularization parameters λ using a projected resilient backpropagation (Rprop) update. The authors validate the approach on a VTVL rocket landing task in a Gym environment, comparing DeePC policies tuned with two approximate models against MPC policies with exact and mismatched models. The central claim is that, given a reasonably accurate surrogate, DeePC-Hunt finds performant regularization parameters for the true system and that the resulting DeePC policy is more robust to model mismatch than model-based MPC.
Significance. If the central claim is substantiated, DeePC-Hunt would be a practically useful tool: it moves hyperparameter tuning for DeePC offline, avoids unsafe real-world guess-and-check, and only requires an approximate model rather than exact dynamics. The paper has clear strengths: the formulation as a bilevel optimization problem (Eq. 13) is clean, the use of differentiable convex optimization layers is appropriate, the projected Rprop variant (Eq. 10) is a sensible algorithmic contribution, and the authors provide a public GitHub repository for reproducibility. However, the empirical validation as presented does not currently isolate the effect of the tuning step, and several statements about local optimality and robustness go beyond what the experiments support. The mathematical components (KKT differentiation, Rprop updates) are standard and correctly described, so the main risk is in the interpretation of the numerical results.
major comments (4)
- [§4.4, Table 1] The central claim that DeePC-Hunt 'yields performant hyperparameters' is not supported without a comparison against untuned or baseline-tuned DeePC policies. Table 1 reports success rates and costs only for MPC(A), MPC(B), πλA, and πλB; there is no evaluation of the DeePC policy at the initial λ0=(50,50,1000) or at any other fixed λ. The tuned values (λA=(49.84,8.36,1000.05), λB=(27.475,2.128,946.05)) are not far from the initialization, and some components barely move, so the reported 66% vs 22% success-rate gap could be due to the inherent robustness of the DeePC policy over a broad range of λ rather than to the optimization in (13). Please add an ablation that evaluates πλ0 on the true environment, together with a simple grid or random-search baseline over λ, and report the resulting success rates and costs. Without this, the paper does not demonstrate that the tuning step is responsible for the observed performance.
- [§3, Eq. (13), text after (10)] The statement that the projected Rprop method 'can be used to compute a locally optimal solution λ∗ of (13)' is an overclaim. Rprop is a heuristic that uses only the sign of the gradient with adaptive step sizes; it has no convergence guarantee to a local minimum of a non-convex objective. In addition, the DeePC solution map πλ in (7) is not differentiable everywhere because the regularizer in (8) contains |g|_1 and |σ_y|_1 terms; the implicit-function-theorem differentiation used by CvxpyLayers is only valid on the set where the active set and strict complementarity hold. The paper should either soften the claim to something like 'a stationary point in the sense of the Rprop update' or state the regularity conditions under which a local optimum can be certified.
- [§4.4, Table 1] The empirical evaluation lacks error bars and independent repetitions. Success rates are computed over 50 initial states, but the DeePC-Hunt training procedure itself is stochastic: Algorithm 1 uses Monte Carlo sampling with B=1 and random initial conditions wini sampled from D(Wp), and the data collected via a PRBS input sequence is a single realization. With only one training run and one data set, it is impossible to assess whether the 66% vs 22% success-rate difference is statistically significant or an artifact of a particular seed or data draw. Please report means and standard deviations over multiple independent training runs and multiple data sets, or at least multiple Monte Carlo seeds within a fixed data set.
- [§3, §4.2-4.3] The surrogate-to-real transfer assumption is introduced in Section 3 as 'the intuition that optimal regularization parameters for similar systems are typically close,' but the paper does not test the limits of this assumption. Only two surrogate models are considered (model A and model B), and both are linearizations of the same true nonlinear dynamics. There is no systematic variation of the surrogate quality (e.g., increasing parameter error, adding unmodeled dynamics, or using a reduced-order model) to show where the proposed tuning starts to fail. Moreover, the comparison to MPC is not a controlled comparison between tuning methods: MPC uses the approximate model as the predictive model, whereas DeePC-Hunt uses it only to tune a data-driven policy, so the comparison primarily demonstrates the robustness of data-driven control under model mismatch. While this is a legitimate observation, it does not establish that DeePC-Hunt's closed-loop backpropagation is superior to existing DeePC tuning heuristics (such as the Final Control Error method cited in the introduction). Adding a comparison to an open-loop tuning baseline would help isolate the contribution of the closed-loop objective in (13).
minor comments (5)
- [§4.1, Eq. (14)] In the equations of motion, the term FE sin(φ+θ)l1 in the first two equations appears to have units of torque rather than force, since m¨x and m¨y are forces. Please check whether l1 should multiply only the angular equation or whether the term should be FE sin(φ+θ) without l1.
- [§2.2, Eq. (8)] The definition of the projection matrix Π=(Wp/Yf)†(Wp/Yf) is unclear: the notation Wp/Yf is not standard, and the use of Yf in the projection is suspicious because the elastic-net regularization should project onto the row space of the past data matrix Wp. Please clarify the intended definition.
- [§2.1] There is a typo in the second paragraph: 'defininng' should be 'defining'.
- [References] The reference for OpenAI Gym (Brockman et al., 2016) lists arXiv number arXiv:1904.09043, which is the same as the reference for Agrawal et al. (2019b) 'Differentiating through a cone program.' Please correct the arXiv identifier for the Gym paper.
- [Figure 3] The caption states the performance of MPC and DeePC-Hunt policies for models A and B, but does not state whether the plotted trajectories correspond to successful landings. Since MPC(B) has a success rate of only 22%, the reader cannot tell if the trajectory shown is a successful run or a representative failure. Please clarify.
Circularity Check
No circularity: DeePC-Hunt tunes hyperparameters on a separate surrogate model and evaluates them on the true Gym environment, so the reported results are not forced by the training objective.
full rationale
The paper's central claim is that hyperparameters optimized on the approximate closed-loop cost (12)-(13) transfer to the true system (5). The surrogate model (11) is an input to the tuning procedure, not an output of it, and the true-system closed-loop cost and success rate used in Table 1 are never folded into the optimization objective (13). Thus the evaluation metric is not statistically forced by the fitting procedure, and no equation in the paper reduces to its own input by construction. The claimed transfer is explicitly stated as an assumption, 'leveraging the intuition that optimal regularization parameters for similar systems are typically close'; an unproved assumption is a correctness or robustness risk, not circularity. The self-citations (Coulson et al. for DeePC, Dorfler et al. for regularization sensitivity and the elastic-net regularizer) supply background and design choices but are not load-bearing for the DeePC-Hunt derivation: the method would remain well-defined with any differentiable regularizer and any approximate model. The skeptic concern that no comparison is made to the untuned initialization lambda0 or to a lambda grid is an evidentiary gap about whether the tuning step itself matters, not a circularity in the derivation chain. Accordingly, no circular step can be exhibited, and the paper is self-contained against the external Gym benchmark evaluation.
Assumptions & free parameters
free parameters (4)
- Initial regularization parameters λ0 =
(50, 50, 1000)
- Rprop step-size schedule =
ηmax=1e2, ηmin=1e-3, β=1.2, α=0.5 (initial η0 not reported)
- Monte Carlo batch size B =
1
- Simulation horizon N and iterations Niter =
N=20, Niter=100
assumptions (4)
- domain assumption The approximate model (11) is sufficiently accurate that optimal DeePC hyperparameters for it are also effective on the true system.
- domain assumption The distribution D(Wp), uniform over columns of the past data matrix Wp, is representative of closed-loop initial conditions.
- standard math The DeePC solution map is differentiable along the optimization trajectory, allowing backpropagation through CvxpyLayers.
- domain assumption The persistent excitation condition on the offline data holds so that the Hankel matrix represents the system behavior.
Cite this review
Pith. "Pith review of DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization." pith.science (2026). https://pith.science/paper/J7NSJFBV
@misc{pith2026241206481,
author = {Pith},
title = {Pith review of: DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/J7NSJFBV}},
note = {Machine review of arXiv:2412.06481}
}
read the original abstract
This paper introduces Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization (DeePC-Hunt), a backpropagation-based method for automatic hyperparameter tuning of the DeePC algorithm. The necessity for such a method arises from the importance of hyperparameter selection to achieve satisfactory closed-loop DeePC performance. The standard methods for hyperparameter selection are to either optimize the open-loop performance, or use manual guess-and-check. Optimizing the open-loop performance can result in unacceptable closed-loop behavior, while manual guess-and-check can pose safety challenges. DeePC-Hunt provides an alternative method for hyperparameter tuning which uses an approximate model of the system dynamics and backpropagation to directly optimize hyperparameters for the closed-loop DeePC performance. Numerical simulations demonstrate the effectiveness of DeePC in combination with DeePC-Hunt in a complex stabilization task for a nonlinear system and its superiority over model-based control strategies in terms of robustness to model misspecifications.
Figures
Forward citations
Cited by 1 Pith paper
-
Fine-tuning for Data-enabled Predictive Control of Noisy Systems by Reinforcement Learning
A SARSA reinforcement learning agent is trained offline and deployed online to adaptively set the DeePC regularization hyperparameter, with simulations showing competitive or better tracking under Gaussian and uniform noise.
Reference graph
Works this paper leans on
-
[1]
A rewriting system for convex optimization problems
Akshay Agrawal, Robin Verschueren, Steven Diamond, and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5 0 (1): 0 42--60, 2018
2018
-
[2]
Differentiable convex optimization layers
Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J Zico Kolter. Differentiable convex optimization layers. Advances in neural information processing systems, 32, 2019 a
work page 2019
-
[4]
Learning convex optimization control policies
Akshay Agrawal, Shane Barratt, Stephen Boyd, and Bartolomeo Stellato. Learning convex optimization control policies. In Learning for Dynamics and Control, pages 361--373. PMLR, 2020
work page 2020
-
[5]
Optnet: Differentiable optimization as a layer in neural networks
Brandon Amos and J Zico Kolter. Optnet: Differentiable optimization as a layer in neural networks. In International conference on machine learning, pages 136--145. PMLR, 2017
2017
-
[6]
Differentiable MPC for end-to-end planning and control
Brandon Amos, Ivan Jimenez, Jacob Sacks, Byron Boots, and J Zico Kolter. Differentiable MPC for end-to-end planning and control. Advances in neural information processing systems, 31, 2018
work page 2018
-
[7]
Approximation of large-scale dynamical systems
Athanasios C Antoulas. Approximation of large-scale dynamical systems. SIAM, 2005
work page 2005
-
[8]
Predictive Control for Linear and Hybrid Systems
Francesco Borrelli, Alberto Bemporad, and Manfred Morari. Predictive Control for Linear and Hybrid Systems. Cambridge University Press, 2017
work page 2017
-
[9]
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. OpenAI Gym . arXiv preprint arXiv:1904.09043, 2016
arXiv 1904
Show all 29 references
-
[10]
Harnessing uncertainty for a separation principle in direct data-driven predictive control
Alessandro Chiuso, Marco Fabris, Valentina Breschi, and Simone Formentin. Harnessing uncertainty for a separation principle in direct data-driven predictive control. arXiv preprint arXiv:2312.14788, 2023
2023 arXiv
-
[11]
Data-enabled predictive control: In the shallows of the deepc
Jeremy Coulson, John Lygeros, and Florian D \"o rfler. Data-enabled predictive control: In the shallows of the deepc. European Control Conference (ECC), 2019 a
2019
-
[12]
Regularized and distributionally robust data-enabled predictive control
Jeremy Coulson, John Lygeros, and Florian D \"o rfler. Regularized and distributionally robust data-enabled predictive control. Conference on Decision and Control (CDC), 2019 b
2019
-
[13]
Distributionally robust chance constrained data-enabled predictive control
Jeremy Coulson, John Lygeros, and Florian D \"o rfler. Distributionally robust chance constrained data-enabled predictive control. IEEE Transactions on Automatic Control, 67 0 (7): 0 3289--3304, 2021
2021
-
[14]
CVXPY : A P ython-embedded modeling language for convex optimization
Steven Diamond and Stephen Boyd. CVXPY : A P ython-embedded modeling language for convex optimization. Journal of Machine Learning Research, 17 0 (83): 0 1--5, 2016
2016
-
[15]
Bridging direct and indirect data-driven control formulations via regularizations and relaxations
Florian Dörfler, Jeremy Coulson, and Ivan Markovsky. Bridging direct and indirect data-driven control formulations via regularizations and relaxations. IEEE Transactions on Automatic Control, 68 0 (2): 0 883--897, 2023
2023
-
[16]
S P C : Subspace predictive control
Wouter Favoreel and Bart De Moor. S P C : Subspace predictive control. IFAC Proceedings Volumes, 32, 1999
1999
-
[17]
A robust control approach for rocket landing, 2017
Reuben Ferrante. A robust control approach for rocket landing, 2017
2017
-
[18]
Toward a theoretical foundation of policy optimization for learning control policies
Bin Hu, Kaiqing Zhang, Na Li, Mehran Mesbahi, Maryam Fazel, and Tamer Ba s ar. Toward a theoretical foundation of policy optimization for learning control policies. Annual Review of Control, Robotics, and Autonomous Systems, 6 0 (1): 0 123--158, 2023
2023
-
[19]
Proximal algorithms
Neal Parikh and Stephen Boyd. Proximal algorithms. Foundations and trendsin Optimization, 1 0 (3): 0 127--239, 2014
2014
-
[20]
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...
2019
-
[21]
Model predictive control: theory, computation, and design, volume 2
James Blake Rawlings, David Q Mayne, Moritz Diehl, et al. Model predictive control: theory, computation, and design, volume 2. Nob Hill Publishing Madison, WI, 2017
2017
-
[22]
A direct adaptive method for faster backpropagation learning: the rprop algorithm
Martin Riedmiller and Heinrich Braun. A direct adaptive method for faster backpropagation learning: the rprop algorithm. International Conference on Neural Networks, 1993
1993
-
[23]
Learning representations by back-propagating errors
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature, 323 0 (6088): 0 533--536, 1986
1986
-
[24]
Ryu and Wotao Yin
Ernest K. Ryu and Wotao Yin. Large-Scale Convex Optimization: Algorithms & Analyses via Monotone Operators. Cambridge University Press, 2022
2022
-
[25]
Learning quadruped locomotion using differentiable simulation
Yunlong Song, Sangbae Kim, and Davide Scaramuzza. Learning quadruped locomotion using differentiable simulation. arXiv preprint arXiv:2403.14864, 2024
2024 arXiv
-
[26]
van Waarde, Claudio De Persis, M
Henk J. van Waarde, Claudio De Persis, M. Kanat Camlibel, and Pietro Tesi. Willems’ fundamental lemma for state-space systems and its extension to multiple datasets. IEEE Control Systems Letters, 4 0 (3): 0 602--607, 2020
2020
-
[27]
Data informativity: A new perspective on data-driven analysis and control
Henk Waarde, Jaap Eising, Harry Trentelman, and Kanat Camlibel. Data informativity: A new perspective on data-driven analysis and control. IEEE Transactions on Automatic Control, pages 1--1, 01 2020
2020
-
[28]
u est, Antonio Loquercio, Matthias M \
Nina Wiedemann, Valentin W \"u est, Antonio Loquercio, Matthias M \"u ller, Dario Floreano, and Davide Scaramuzza. Training efficient controllers via analytic policy gradient. International Conference on Robotics and Automation (ICRA), 2023
2023
-
[29]
Data-driven system level synthesis
Anton Xue and Nikolai Matni. Data-driven system level synthesis. Learning for Dynamics and Control, 2021
2021
-
[30]
Bp-mpc: Optimizing closed-loop performance of mpc using backpropagation
Riccardo Zuliani, Efe C Balta, and John Lygeros. Bp-mpc: Optimizing closed-loop performance of mpc using backpropagation. arXiv preprint arXiv:2312.15521, 2023
2023 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.