{"id":"71ac487c-8723-4472-abb5-f0c4dfa1ecdd","arxiv_id":"2412.06481","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DeePC-Hunt uses backpropagation through an approximate model to automatically tune DeePC regularization hyperparameters for closed-loop performance.","lead":"The paper introduces DeePC-Hunt, an algorithm that automatically tunes the regularization parameters of the data-driven controller DeePC by differentiating through a simulated closed-loop rollout on an approximate model. It provides a practical way to tune controllers without expensive real-world experiments, and shows on a rocket-landing benchmark that the tuned controller remains effective when the model is inaccurate.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No experiment compares DeePC-Hunt's tuned λ against the initial λ0 or a λ grid on the true system, so the reported gains are not shown to come from the tuning step.","rationale":"I read the paper in good faith and acknowledge its concrete contributions: a clean bilevel formulation (13), a projected Rprop variant, an implementation built on CvxpyLayers, and a reproducible benchmark with released code. The reader's identified weakness, surrogate-to-real transfer of tuned hyperparameters, is indeed unproved and deserves testing across model-mismatch levels. However, the single most load-bearing gap in the paper as written is more basic: the experimental section never isolates the effect of the tuning step. Table 1 compares the full DeePC-Hunt pipeline to model-based MPC, but there is no DeePC baseline at the initial λ0 or at other hyperparameter values. Since the tuned λA and λB are close to λ0 (and λA is nearly identical to λ0 for two of three components), the reported 66% success rate may be due to the data-driven DeePC formulation itself, not to the proposed backpropagation algorithm. The method could still be useful, and the code is provided, but the central claim is under-supported by the current experiments. A comparison against untuned DeePC on the same 50 initial conditions would settle this. The reader's CONDITIONAL verdict remains appropriate; my concern adds a concrete, easily testable reason for the condition.","tokens_in":11065,"tokens_out":9970,"duration_ms":107909,"concrete_test":"Run the DeePC policy (6)-(8) on the same VTVL simulator with the initial λ0=(50,50,1000) and with a modest grid or random set of λ (e.g., 20 log-spaced values around λ0), over the same 50 initial states used for Table 1, reporting success rate and realized cost. If λ0 or many grid points achieve success rates at or above the reported 66% for πλB, the tuning step is not demonstrated to matter; if λ0 performs substantially worse, the tuning claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 1 compares the DeePC-Hunt policies (πλA, πλB) only against MPC(A/B). It reports no success rate or closed-loop cost for the DeePC policy at the initialization λ0=(50,50,1000), nor for any other hyperparameter value. The central contribution of the paper is the claim that backpropagation-based tuning selects performant regularization parameters, and the reported 66% success rate is attributed to DeePC-Hunt. But the data are also consistent with a different story: the fixed Hankel data-driven policy may be robust over a wide range of λ, making the tuning step irrelevant or even harmful. This is not a merely aesthetic ablation. The tuned values are close to the initialization (λA=(49.84,8.36,1000.05), λB=(27.475,2.128,946.05)), so the experiments give little evidence that the optimization in (13) is responsible for the closed-loop result. Without a comparison to untuned DeePC, the paper's central claim that DeePC-Hunt yields performant hyperparameters and outperforms mismatched MPC does not land. The surrogate-to-real transfer concern raised by the reader is real, but it is downstream: first one must establish that the tuning step changes performance at all.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DeePC-Hunt, an offline hyperparameter-tuning method for Data-enabled Predictive Control (DeePC). The method interprets the DeePC optimization as a differentiable policy, backpropagates the closed-loop cost on an approximate surrogate model through the policy, and optimizes the regularization parameters λ using a projected resilient backpropagation (Rprop) update. The authors validate the approach on a VTVL rocket landing task in a Gym environment, comparing DeePC policies tuned with two approximate models against MPC policies with exact and mismatched models. The central claim is that, given a reasonably accurate surrogate, DeePC-Hunt finds performant regularization parameters for the true system and that the resulting DeePC policy is more robust to model mismatch than model-based MPC.","tokens_in":11298,"tokens_out":5622,"duration_ms":57455,"significance":"If the central claim is substantiated, DeePC-Hunt would be a practically useful tool: it moves hyperparameter tuning for DeePC offline, avoids unsafe real-world guess-and-check, and only requires an approximate model rather than exact dynamics. The paper has clear strengths: the formulation as a bilevel optimization problem (Eq. 13) is clean, the use of differentiable convex optimization layers is appropriate, the projected Rprop variant (Eq. 10) is a sensible algorithmic contribution, and the authors provide a public GitHub repository for reproducibility. However, the empirical validation as presented does not currently isolate the effect of the tuning step, and several statements about local optimality and robustness go beyond what the experiments support. The mathematical components (KKT differentiation, Rprop updates) are standard and correctly described, so the main risk is in the interpretation of the numerical results.","major_comments":[{"comment":"The central claim that DeePC-Hunt 'yields performant hyperparameters' is not supported without a comparison against untuned or baseline-tuned DeePC policies. Table 1 reports success rates and costs only for MPC(A), MPC(B), πλA, and πλB; there is no evaluation of the DeePC policy at the initial λ0=(50,50,1000) or at any other fixed λ. The tuned values (λA=(49.84,8.36,1000.05), λB=(27.475,2.128,946.05)) are not far from the initialization, and some components barely move, so the reported 66% vs 22% success-rate gap could be due to the inherent robustness of the DeePC policy over a broad range of λ rather than to the optimization in (13). Please add an ablation that evaluates πλ0 on the true environment, together with a simple grid or random-search baseline over λ, and report the resulting success rates and costs. Without this, the paper does not demonstrate that the tuning step is responsible for the observed performance.","section":"§4.4, Table 1"},{"comment":"The statement that the projected Rprop method 'can be used to compute a locally optimal solution λ∗ of (13)' is an overclaim. Rprop is a heuristic that uses only the sign of the gradient with adaptive step sizes; it has no convergence guarantee to a local minimum of a non-convex objective. In addition, the DeePC solution map πλ in (7) is not differentiable everywhere because the regularizer in (8) contains |g|_1 and |σ_y|_1 terms; the implicit-function-theorem differentiation used by CvxpyLayers is only valid on the set where the active set and strict complementarity hold. The paper should either soften the claim to something like 'a stationary point in the sense of the Rprop update' or state the regularity conditions under which a local optimum can be certified.","section":"§3, Eq. (13), text after (10)"},{"comment":"The empirical evaluation lacks error bars and independent repetitions. Success rates are computed over 50 initial states, but the DeePC-Hunt training procedure itself is stochastic: Algorithm 1 uses Monte Carlo sampling with B=1 and random initial conditions wini sampled from D(Wp), and the data collected via a PRBS input sequence is a single realization. With only one training run and one data set, it is impossible to assess whether the 66% vs 22% success-rate difference is statistically significant or an artifact of a particular seed or data draw. Please report means and standard deviations over multiple independent training runs and multiple data sets, or at least multiple Monte Carlo seeds within a fixed data set.","section":"§4.4, Table 1"},{"comment":"The surrogate-to-real transfer assumption is introduced in Section 3 as 'the intuition that optimal regularization parameters for similar systems are typically close,' but the paper does not test the limits of this assumption. Only two surrogate models are considered (model A and model B), and both are linearizations of the same true nonlinear dynamics. There is no systematic variation of the surrogate quality (e.g., increasing parameter error, adding unmodeled dynamics, or using a reduced-order model) to show where the proposed tuning starts to fail. Moreover, the comparison to MPC is not a controlled comparison between tuning methods: MPC uses the approximate model as the predictive model, whereas DeePC-Hunt uses it only to tune a data-driven policy, so the comparison primarily demonstrates the robustness of data-driven control under model mismatch. While this is a legitimate observation, it does not establish that DeePC-Hunt's closed-loop backpropagation is superior to existing DeePC tuning heuristics (such as the Final Control Error method cited in the introduction). Adding a comparison to an open-loop tuning baseline would help isolate the contribution of the closed-loop objective in (13).","section":"§3, §4.2-4.3"}],"minor_comments":[{"comment":"In the equations of motion, the term FE sin(φ+θ)l1 in the first two equations appears to have units of torque rather than force, since m¨x and m¨y are forces. Please check whether l1 should multiply only the angular equation or whether the term should be FE sin(φ+θ) without l1.","section":"§4.1, Eq. (14)"},{"comment":"The definition of the projection matrix Π=(Wp/Yf)†(Wp/Yf) is unclear: the notation Wp/Yf is not standard, and the use of Yf in the projection is suspicious because the elastic-net regularization should project onto the row space of the past data matrix Wp. Please clarify the intended definition.","section":"§2.2, Eq. (8)"},{"comment":"There is a typo in the second paragraph: 'defininng' should be 'defining'.","section":"§2.1"},{"comment":"The reference for OpenAI Gym (Brockman et al., 2016) lists arXiv number arXiv:1904.09043, which is the same as the reference for Agrawal et al. (2019b) 'Differentiating through a cone program.' Please correct the arXiv identifier for the Gym paper.","section":"References"},{"comment":"The caption states the performance of MPC and DeePC-Hunt policies for models A and B, but does not state whether the plotted trajectories correspond to successful landings. Since MPC(B) has a success rate of only 22%, the reader cannot tell if the trajectory shown is a successful run or a representative failure. Please clarify.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically sound in its algorithmic components, but the empirical evidence is not yet sufficient to support the central claim that the tuning step itself improves performance. The missing ablation against untuned DeePC is the key issue. I would not reject the paper, as the method is plausible and the code is available, but the authors need to add the missing baseline and report variance before the claim can be accepted. The overstatement of 'local optimality' should also be corrected. A comparison to an existing DeePC tuning heuristic would strengthen the paper further."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague, quick take on 2412.06481. The core idea is genuinely useful: treat DeePC's regularization weights as policy parameters, instantiate the policy with differentiable convex optimization layers, and backprop the closed-loop cost on an approximate surrogate model to tune them. That is new relative to the MPC tuning work of Amos et al. and Zuliani et al., which need a state-space model for the tuned policy itself. DeePC-Hunt keeps the data-driven policy and just uses the approximate model as a surrogate for tuning, which is a sensible division. The implementation is careful, the projected Rprop variant is a modest but real extension, and the paper is honest that surrogate-to-real transfer is an assumption rather than a theorem. The VTVL experiments show the whole pipeline runs and the comparison against badly mismatched MPC (66% vs 22% success) is suggestive.\n\nThe soft spot is load-bearing: there is no ablation against the initial regularization values or a grid. The tuned lambda values sit very close to the starting point (lambda_A = (49.84, 8.36, 1000.05) vs (50, 50, 1000); lambda_B = (27.475, 2.128, 946.05) vs (50, 50, 1000)). So the reported success rates could come from DeePC's baseline robustness rather than from the tuning step. Without a DeePC-at-lambda0 column in Table 1, the central claim that DeePC-Hunt \"yields performant hyperparameters\" does not land. The other concerns are minor: one benchmark, no error bars over the 50 initial states, no comparison to existing tuning heuristics, and the \"local optimality\" phrasing overstates what a sign-based Rprop heuristic can guarantee. The self-citations to Dorfler et al. are relevant and not a problem.\n\nBottom line: the method is plausible and clearly presented, the code is released, but the evidence as presented does not isolate the contribution. That is fixable with an honest baseline table and ideally a second system. I would send this to peer review; a referee should ask for the ablation first. It is worth reading for anyone working on DeePC or differentiable predictive control, and it would make a good reading-group discussion about evaluation practice.","headline":"Good idea and clean implementation, but the missing untuned-DeePC baseline means the paper never shows the tuning step is what helps.","tokens_in":11859,"tokens_out":2190,"would_cite":true,"duration_ms":22778,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C25","93B40","93C55"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces DeePC-Hunt, a backpropagation method that tunes the regularization hyperparameters of Data-enabled Predictive Control on an approximate model so that they work on the true closed-loop system.","keywords":["DeePC","data-enabled predictive control","hyperparameter tuning","differentiable optimization","backpropagation","resilient backpropagation","model mismatch","predictive control"],"falsifier":"Take the VTVL benchmark, keep everything fixed, and sweep surrogate mismatch continuously by inflating $l_1$, shrinking $l_2$, and changing the mass parameter by increasing amounts; record the true-system success rate of the resulting tuned policies. If success-rate degrades sharply to the MPC (B) level for a mismatch well within the range a user might reasonably call approximate, the transfer claim is falsified; the paper currently reports only one moderately mismatched model.","tokens_in":10854,"feed_emoji":"🚀","tokens_out":9376,"duration_ms":90787,"temperature":0.7,"pith_summary":"DeePC's closed-loop performance depends heavily on three regularization weights, but tuning them is usually done either by optimizing open-loop predictions, which can ignore replanning, or by trial and error on the real system, which can be unsafe or expensive. DeePC-Hunt treats the DeePC optimizer as a differentiable control policy and backpropagates the simulated closed-loop cost through an approximate model to adjust the weights with a projected resilient backpropagation step. The paper claims that for a reasonably approximated model this produces regularization parameters that transfer to the true system, and reports that on a vertical-takeoff-and-landing benchmark DeePC-Hunt succeeded 66% of the time while an MPC policy built on the same mismatched model succeeded 22% of the time. The contribution is a practical, fully offline hyperparameter-tuning loop for a popular direct data-driven controller.","feed_headline":"Backprop tunes data-driven control without risky real-system trials","feed_subtitle":"From an approximate model, DeePC-Hunt tunes closed-loop hyperparameters and beats mismatched model-based control on rocket landing.","key_machinery":"The central object is the DeePC solution map $\\pi_\\lambda$, which maps the regularization vector $\\lambda=(\\lambda_0,\\lambda_1,\\lambda_2)$ and the recent input/output history to the next control action; because the underlying problem is a disciplined parameterized convex program, this map can be differentiated with respect to $\\lambda$ by differentiating the KKT conditions. The mechanism is the bilevel optimization problem that minimizes the expected closed-loop cost of $\\pi_\\lambda$ simulated on the surrogate model, solved with a projected resilient backpropagation algorithm that uses only gradient signs and per-coordinate adaptive step sizes. The specific object being tuned is the regularization function $\\lambda_0|(I-\\Pi)g|_2^2 + \\lambda_1|g|_1 + \\lambda_2|\\sigma_y|_1$.","core_discovery":"The paper's central claim is that hyperparameters minimizing the expected closed-loop cost of a DeePC policy on an approximate surrogate model also deliver strong closed-loop performance on the true system. To make this computable, DeePC is recast as a convex-optimization policy whose solution map is differentiable with respect to the regularization vector via the implicit function theorem, so the gradient of the closed-loop cost can be obtained by backpropagation. A constrained variant of resilient backpropagation then updates the three regularization weights, including projections to keep them non-negative. In the VTVL landing task, both an accurate and a deliberately mis-estimated surrogate produced near-identical DeePC-Hunt landing trajectories, and the policy tuned through the bad model outperformed MPC using the same bad model on success rate.","pith_inferences":["A natural extension the authors leave implicit is to tune the cost matrices $Q$ and $R$, the horizons $T_{ini}$ and $T_f$, or the reference trajectory distribution through the same pipeline, since all of these enter the same differentiable solution map.","The transfer claim could be stress-tested by plotting tuned-$\\lambda$ performance against a continuous measure of surrogate mismatch; the paper reports two snapshots (accurate and moderately wrong models) rather than a failure boundary.","If the surrogate is chosen so that its closed-loop cost is an upper bound on the true cost, then minimization of the surrogate cost would carry a formal transfer guarantee; the paper relies instead on empirical similarity without proving such a bound."],"forward_implications":["DeePC users can tune regularization parameters entirely offline, without risky or expensive experiments on the real system, as long as an approximate model is available.","Under model mismatch, the DeePC-Hunt policy can remain effective where a linearized MPC using the same mismatched model fails, because receding-horizon replanning absorbs much of the model error.","The differentiable-policy viewpoint opens DeePC to gradient-based optimization of not just regularization weights but any problem parameter that enters the convex program.","The constrained resilient backpropagation update is applicable to any hyperparameter search with box or cone constraints, not only to DeePC."],"supporting_citations":[{"why":"introduces the DeePC algorithm and the receding-horizon optimization that DeePC-Hunt differentiates as a policy","marker":"Coulson et al., 2019a"},{"why":"supplies the elastic-net regularization form and the observation that DeePC performance is sensitive to the regularization parameters","marker":"Dörfler et al., 2023"},{"why":"provides differentiable convex optimization layers that make the DeePC solution map differentiable with respect to hyperparameters","marker":"Agrawal et al., 2019a"},{"why":"establishes the KKT-based implicit differentiation technique that the gradient computation relies on","marker":"Amos and Kolter, 2017"},{"why":"shows that differentiable optimization can tune general convex optimization control policies, the broader setting DeePC-Hunt enters","marker":"Agrawal et al., 2020"},{"why":"provides the closest prior method, backprop-based closed-loop MPC tuning, which DeePC-Hunt extends from state-space models to data-driven policies","marker":"Zuliani et al., 2023"},{"why":"contributes the resilient backpropagation algorithm whose projected variant is the workhorse optimizer in DeePC-Hunt","marker":"Riedmiller and Braun, 1993"},{"why":"defines the VTVL rocket landing benchmark used for the numerical evaluation","marker":"Ferrante, 2017"},{"why":"supplies the standardized simulation environment in which the landing task is implemented","marker":"Brockman et al., 2016"}],"fun_headline_variants":["Backprop tunes DeePC hyperparameters for closed-loop performance","Tune DeePC without risky real-system trials","Gradient-based tuning for robust data-driven control","DeePC-Hunt: backprop tunes regularization for robust control","Automatic DeePC tuning beats mismatched model-based control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that regularization parameters that minimize the closed-loop cost on the approximate surrogate model are also good for the true system, an assumption the paper motivates only by the heuristic that similar systems have similar optimal parameters.","fun_headline_variants_meta":{"raw":{"variants":["Backprop tunes DeePC hyperparameters for closed-loop performance","Tune DeePC without risky real-system trials","Gradient-based tuning for robust data-driven control","DeePC-Hunt: backprop tunes regularization for robust control","Automatic DeePC tuning beats mismatched model-based control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001336,"raw_usage":{"total_tokens":5392,"prompt_tokens":866,"completion_tokens":4526,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":4447}},"tokens_in":482,"tokens_out":4526,"duration_ms":33893,"temperature":1.0,"reasoning_tokens":4447,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:36:21.104945+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the VTVL benchmark, keep everything fixed, and sweep surrogate mismatch continuously by inflating $l_1$, shrinking $l_2$, and changing the mass parameter by increasing amounts; record the true-system success rate of the resulting tuned policies. If success-rate degrades sharply to the MPC (B) level for a mismatch well within the range a user might reasonably call approximate, the transfer claim is falsified; the paper currently reports only one moderately mismatched model.","supporting_citations":[{"cited_title":"Learning convex optimization control policies","cited_arxiv_id":null,"evidence_quote":"shows that differentiable optimization can tune general convex optimization control policies, the broader setting DeePC-Hunt enters"},{"cited_title":"BP-MPC: Optimizing the Closed-Loop Performance of MPC using BackPropagation","cited_arxiv_id":"2312.15521","evidence_quote":"provides the closest prior method, backprop-based closed-loop MPC tuning, which DeePC-Hunt extends from state-space models to data-driven policies"},{"cited_title":"A direct adaptive method for faster backpropagation learning: the rprop algorithm","cited_arxiv_id":null,"evidence_quote":"contributes the resilient backpropagation algorithm whose projected variant is the workhorse optimizer in DeePC-Hunt"},{"cited_title":"A robust control approach for rocket landing, 2017","cited_arxiv_id":null,"evidence_quote":"defines the VTVL rocket landing benchmark used for the numerical evaluation"}],"review_version":1}