{"id":"4cbd0e28-6082-4f7a-b0db-2350136f27b4","arxiv_id":"2412.02033","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Using linearized game values as supervision for neural Hamilton-Jacobi solvers improves the speed and accuracy of learned controllers for high-dimensional nonlinear systems.","lead":"This paper proposes two ways to combine fast solutions of linearized Hamilton-Jacobi equations with deep learning to train value functions for high-dimensional nonlinear control and differential games. The methods report speed and accuracy gains on a 50-dimensional game and a 10-dimensional quadrotor collision-avoidance problem.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing gap is that supervision is justified only by a value-closeness bound; LLS also matches gradients, and no result bounds gradient error, so in strongly nonlinear regimes the linear supervisor can bias the learned policy.","rationale":"The reader's weakest assumption identifies the reliance on V_ℓ being close to V; my concern extends this to the gradient-matching term, which is equally load-bearing because the control law is derived from ∇V. The empirical results in Fig. 2 and Table 1 are internally consistent and show substantial improvements in the tested scenarios, including the 50-D game and the 10-D quadrotor, so the central claim has genuine support. However, the theoretical justification is local: Theorem 1 requires a min-over-time equality and trajectory confinement, and Corollary 1 only establishes closeness in the limit as the operating point is approached. The paper itself flags the unproven optimization-path assumption in the decayed scheme. These gaps do not invalidate the reported results, but they make the general claim conditional on V_ℓ being a faithful surrogate over the training distribution. A concrete large-δ* experiment would settle whether the method fails gracefully or actively biases learning when the linearization is poor. This is an addressable weakness, not a fatal flaw, so keeping the conditional verdict is appropriate.","tokens_in":15170,"tokens_out":7059,"duration_ms":79582,"concrete_test":"In the 50-D publisher-subscriber benchmark, run the baseline, LSS-Decay, and Vλ-LSS programs with a nonlinearity setting where δ*=max_{S̄×U×D×T}∥f−ℓ∥ is at least 5–10× its largest value among the four reported variations (e.g., α=β=50), fixing all training hyperparameters and reporting 5 seeds with error bars; also record ∥∇V−∇V_ℓ∥ on the test set. If the supervised schemes do not beat the baseline in IOU/MSE, or if gradient error is large while value error is small, the supervision term is not a safe general accelerator and the paper should restrict its claim to near-linear regimes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Decayed and augmented programs both add LLS(θ)=ρ∥V_θ−V_ℓ∥+ρ_g∥∇V_θ−∇V_ℓ∥ to the PDE loss. For this to accelerate rather than bias learning, V_ℓ and ∇V_ℓ must be good surrogates on the training region. The only theoretical support is Theorem 1 and Corollary 1. Theorem 1 bounds |V−V_ℓ| under two restrictive assumptions: the min-over-time equality V = min_τ sup_d inf_u J_T(x(τ)), and confinement of trajectories to S̄(τ); Corollary 1 gives a bound only as the operating point is approached. The training distributions in Sec. 4 are not localized near the operating point, and the 50-D benchmark includes nonlinearity parameters up to (α,β)=(20,−10). Moreover, no result in the paper bounds ∥∇V−∇V_ℓ∥, yet LLS explicitly fits the gradient. In reachability, the policy is u*=argmin max ⟨∇_x V,f⟩, so an incorrect supervised gradient can yield incorrect controls even when the value itself is close. The paper concedes in Sec. 3.2 that there is no guarantee of a globally minimizing path between LLS and L_PDE for λ0→λK. Thus, for a strongly nonlinear system where V_ℓ is not close, the supervision term can actively conflict with the PDE loss; the reported gains are evidence for the tested regimes, not for the general claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to accelerate and improve neural solutions of high-dimensional, nonlinear Hamilton-Jacobi reachability problems by supervising a neural value network with V_ell, the value of a linearized (or Hopf-formula-solvable) game. Two loss programs are introduced: a decayed linear semi-supervision loss (Definition 2) that anneals from the supervision loss to the PDE residual loss, and a nonlinear-spectrum augmented loss (Definition 4) that trains a network on an augmented state (x, lambda) with supervision on the lambda=0 slice and PDE loss everywhere. The methods are evaluated on a 50-D publisher-subscriber differential game with dynamic-programming ground truth and on a 10-D quadrotor collision-avoidance problem, reporting improved IoU, MSE, recovered safe volume, and reduced false positives. The paper also provides Theorem 1 and Corollary 1 to justify closeness of V and V_ell, and Theorem 2 for the augmented-game construction.","tokens_in":15548,"tokens_out":7224,"duration_ms":76124,"significance":"If substantiated, the central idea is valuable: cheap linear/Hopf solutions are used as inductive bias for learned nonlinear HJ solvers, potentially removing the time-curriculum bottleneck and improving accuracy in high dimensions. The 50-D benchmark scored against a dynamic-programming ground truth is a genuine strength, and the augmented-game construction in Theorem 2 is a clean and useful addition. The paper is also honest about some of its limitations, explicitly noting in Section 3.2 that there is no guarantee of a globally minimizing path and in Section 4.2 that the quadrotor solution is not particularly nonlinear. However, the theoretical support covers only value closeness under assumptions that are not verified, no result bounds the gradient error that the supervision loss explicitly fits, and the empirical claims rest on single runs with per-problem hyperparameter tuning. The idea is promising, but the current evidence does not yet establish the full scope claimed in the abstract and conclusion.","major_comments":[{"comment":"The bound |V - V_ell| <= epsilon* in Eq. (10) depends on two assumptions stated only in the proof: the min-over-time equality V(x,t) = min_tau sup_d inf_u J_T(x(tau)) for the states considered, and the confinement of all relevant trajectories to the set S_bar(tau). Neither assumption is verified for the 50-D or 10-D experiments, and Corollary 1 gives closeness only in the limit m -> m0. Since the training distributions in Section 4 cover states far from the operating point, Theorem 1 does not justify the use of V_ell on the actual training region.","section":"Theorem 1 / Appendix 6.1"},{"comment":"The linear supervision loss explicitly fits the gradient, LLS = rho ||V_theta - V_ell|| + rho_g ||grad V_theta - grad V_ell||, but no theorem or experiment bounds ||grad V - grad V_ell||. In the reachability formulation of Section 2 the optimal control is u* = argmin max <grad V, f>, so an incorrect supervised gradient can yield incorrect controls even when the value itself is close. The paper needs either a gradient-error bound under the assumptions of Theorem 1 or an empirical evaluation of control/policy error, ideally in a strongly nonlinear regime.","section":"Definition 1 / Eq. (13)"},{"comment":"The decayed program is described as transitioning from the linear solution to the nonlinear solution as lambda_k increases, but Section 6.5 reports lambda_K = 0.6 for the LSS Decay method, so the final loss still contains 40% linear supervision and never reaches the pure PDE objective. In addition, Section 3.2 states that there is no guarantee of a globally minimizing path between LLS and L_PDE for lambda_0 -> lambda_K. Together these points mean that the mechanism claimed for the decayed method is asserted rather than established.","section":"Definition 2 / Section 6.5"},{"comment":"The empirical claims rest on point estimates from single runs. The training details state that experiments were chosen because they 'performed best' and that a coarse parameter search was used to select lambda_K, rho, and rho_g, but no seeds, no error bars, and no ablations are reported. The headline numbers (2.4x IoU, 23.7x MSE, 20x speedup) are therefore not yet established as significant or robust to hyperparameter choice.","section":"Section 6.5 / Tables 1 and Fig. 2"},{"comment":"The paper's own Section 4.2 says the quadrotor solution 'is not particularly nonlinear,' and the 50-D benchmark is a decomposable publisher-subscriber game whose value has the special additive structure of Remark 1. No experiment is run in a regime where V_ell is known to be a poor approximation of V. The conclusion that the augmented method should be preferred 'if the problem is very nonlinear' is therefore an extrapolation beyond the demonstrated evidence.","section":"Section 4.2 / Conclusion"}],"minor_comments":[{"comment":"The notation H^+-_{ell+epsilon} with nested plus/minus signs is ambiguous; H+ and H- should be defined explicitly as H_ell + max_epsilon <p,epsilon> and H_ell - max_epsilon <p,epsilon> (or with min, as appropriate).","section":"Eq. (9)"},{"comment":"The norms in the supervision loss are not specified; please state whether these are L2 norms, weighted norms, or something else.","section":"Definition 1 / Eq. (13)"},{"comment":"The integral notation 'Z s t' is malformed and should be written as an integral from t to s; there is also an inconsistent use of s as both the integration variable and the final time.","section":"Corollary 1 proof"},{"comment":"The text says 'our fork of the existing DeepReach software may be found here,' but no URL is provided; a link or repository identifier is needed for reproducibility.","section":"Section 6.5"},{"comment":"There are several typos and formatting inconsistencies, including 'Aknowledgements', 'IOU' versus 'IoU', and 'assm.'; these should be cleaned up in revision.","section":"Throughout"},{"comment":"The runtime panel should state units and clarify that the reported runtimes include the time to generate the linear supervisor, as described in Section 6.5.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The core idea is not circular: V_ell is an external benchmark or a separately trained linear network, and the 50-D scoring uses a dynamic-programming ground truth. The main weakness is that the theory and experiments do not yet support the full generality of the claims. I would be willing to accept a revision that adds a genuinely strongly nonlinear test case, reports multiple seeds with error bars, and either proves or empirically checks gradient-error behavior; otherwise the claims should be narrowed to 'mildly nonlinear regimes'."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper proposes something worth knowing about: train a network to solve the nonlinear HJ PDE by supervising it toward the linearized solution, either by decaying the supervision (LSS-D) or by embedding the system in a lambda-augmented game and supervising at lambda=0 (LSS-NS). The augmented-game construction is genuinely fresh -- I haven't seen that interpolation trick for reconciling a linear loss with a PDE loss. The empirical wins are consistent across two quite different problems, and the 50-D benchmark is a clever use of the decomposable structure to get ground truth at a dimension where you normally can't verify anything. The speedup in the decayed program (about 20x on the 50-D) is substantial.\n\nWhat the paper does well: the authors are honest about the main limitation in Sec. 3.2 -- there is no guarantee of a globally minimizing path between LLS and LPDE -- and they buy it back with the augmented program, where the losses share a minimizer at lambda=0. The theory in Theorem 1 is a real generalization of earlier Hopf-based bounds to time-varying endpoints and error-assisting games. That is a legitimate contribution.\n\nNow the soft spots, in proportion. The weakest link is the reliance on V_ell being close to V over the training region. Theorem 1 gives this only under restrictive assumptions -- the min-over-time equality and trajectory confinement to S_bar -- and Corollary 1 only in the limit as the operating point is approached. The 50-D benchmark includes strongly nonlinear parameters like (alpha,beta)=(20,-10), so it is not clear the training distribution is in the regime where the bound applies. More importantly, LLS also fits the gradient, and no result bounds ||grad V - grad V_ell||. Since the control law depends on the gradient of V, a biased gradient can mislead the policy even when the value itself is close. The paper would be stronger if it addressed this gap directly.\n\nThe experimental reporting is thin: no seeds, no error bars, and hyperparameters (lambda_K, rho, rho_g) are tuned on the test problems. The decay program's 20x speedup partly comes from using 10k iterations versus 300k for baseline -- that is a legitimate finding but needs sensitivity analysis to show it holds without cherry-picking.\n\nBottom line: the central idea is sound and the evidence is suggestive, but the claim that supervision 'accelerates and improves' needs a tighter theoretical handle on the gradient error and better empirical calibration before I'd lean on it for safety-critical use. The augmented-game perspective alone is worth the read.\n\nI'd send this to reviewers -- the idea is novel and the demonstrations are on hard problems. A serious referee should engage with it, with the expectation of major revision on the reporting and theory side.","headline":"A well-posed idea with real empirical gains, but the theory underpinning the linear supervisor is narrower than the claims, and the experimental reporting needs more rigor.","tokens_in":16055,"tokens_out":2012,"would_cite":true,"duration_ms":18789,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["49L25","49N70","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Training a neural value function with an extra pull toward the linearized game solution makes high-dimensional nonlinear control and differential games both faster and more accurate to learn.","keywords":["Hamilton-Jacobi equations","differential games","reachability","value function approximation","neural networks","Hopf formula","high-dimensional control","linearization"],"falsifier":"Train both semi-supervision programs on a high-dimensional nonlinear system where the linearization error is known to be large, for example a 50-D publisher-subscriber game with large $\\alpha$ and $\\beta$ so that $\\delta^*$ from Theorem 1 is comparable to the value range, and compare intersection-over-union and mean-squared error against the PDE-only baseline; if the supervised programs do not beat the baseline, the claim that linear supervision reliably improves learned value functions is refuted.","tokens_in":14987,"feed_emoji":"🎯","tokens_out":17157,"duration_ms":120706,"temperature":0.7,"pith_summary":"The paper proposes that a neural network trained to solve a nonlinear Hamilton-Jacobi PDE can be made faster and more accurate by adding a supervision term that pulls the network toward the solution of the corresponding linearized game, computed cheaply via the Hopf formula. Two training programs are defined: one anneals from the linear solution into the nonlinear PDE loss, and one learns the value of an augmented game whose dynamics interpolate between linear and nonlinear, with the linear value acting as a boundary condition. In a 50-dimensional publisher-subscriber differential game, the augmented program raises intersection-over-union by 2.4 times and lowers mean-squared error by 23.7 times relative to PDE-only training, while the decayed program runs about 20 times faster. In a 10-dimensional quadrotor avoidance problem, the decayed schemes recover about 95% of the conformally expanded safe volume versus 81% for the baseline, and cut false positives from 1.86% to as low as 0.23%.","feed_headline":"Accelerate neural control learning with linear game solutions","feed_subtitle":"Augmented scheme cuts error 23.7x in 50-D games; decayed scheme recovers 95% safe volume in quadrotor control.","key_machinery":"The central object is the linear supervision loss $L_{LS}(\\theta) = \\rho\\|V_\\theta - V_\\ell\\| + \\rho_g\\|\\nabla_x V_\\theta - \\nabla_x V_\\ell\\|$, which is added to the Hamilton-Jacobi PDE residual loss $L_{PDE}$. The linear value $V_\\ell$ is the value of the linearized game computed by the Hopf formula, giving a cheap and globally smooth proxy for the true value. The two programs differ in how the losses are combined: the decayed program multiplies $L_{LS}$ by $(1-\\lambda_k)$ and $L_{PDE}$ by $\\lambda_k$ with $\\lambda_k$ increasing from 0 to 1, while the augmented program learns the value $V_\\lambda$ of a game whose dynamics are $(1-\\lambda)\\ell + \\lambda f$, with $V_\\lambda = V_\\ell$ at $\\lambda=0$ and $V_\\lambda = V$ at $\\lambda=1$. The augmentation makes the linear and PDE losses share a set of global minimizers, so the network's task is refinement rather than search from scratch.","core_discovery":"The paper's central claim is that the value function of a nonlinear differential game can be learned more reliably by using the value function of a linearly approximated game as a structured training signal rather than training only against the nonlinear PDE residual. The linear value $V_\\ell$, obtained with the Hopf formula at orders-of-magnitude lower cost than grid-based dynamic programming, is close enough to the true value $V$ over the region of interest that it can serve as a proxy target: the network first approximates $V_\\ell$, then refines toward $V$. The paper proves a bound on $|V - V_\\ell|$ in terms of the maximum difference between the nonlinear and linear dynamics along relevant trajectories, and shows the bound vanishes at the linearization operating point. Empirically, both proposed programs beat the PDE-loss baseline on a 50-dimensional benchmark and a 10-dimensional quadrotor problem, with the augmented game giving the largest accuracy gains and the decayed scheme the largest speed gains.","pith_inferences":["The same supervision idea should transfer to other cheap approximate value functions, such as reduced-order models or coarser-grid solutions; the decayed schedule would likely retain most of its acceleration for any structured prior.","A practical sanity check emerges from the augmented construction: after training, the network's prediction on the $\\lambda=0$ slice should match $V_\\ell$; a significant mismatch would indicate the linear boundary condition was not internalized.","The theoretical bound's assumptions (min-over-time equality and confinement to $\\bar S$) are restrictive; if they fail for a given system, the empirical benefit could persist even where the proof does not apply, so practitioners may need to treat the bound as qualitative guidance rather than a certificate.","One testable extension is to make the supervision weight $\\rho$ depend on a local estimate of the linearization error $\\delta^*$, so the network trusts $V_\\ell$ only in regions where it is known to be accurate."],"forward_implications":["The learning problem shifts from generating a value function from scratch to refining a partially correct one, so the time-curriculum used by prior learned Hamilton-Jacobi solvers can be dropped; in the 50-D benchmark the decayed program completes in about one twentieth of the baseline time.","The augmented program's accuracy gains (2.4x intersection-over-union, 23.7x lower mean-squared error) show that adding the $\\lambda=0$ linear boundary condition supplies structure that helps the network approximate the $\\lambda=1$ nonlinear solution.","The drop in false positives from 1.86% to 0.23% in the quadrotor task means linear supervision counteracts the optimistic bias that makes high-dimensional learned value functions mark unsafe states as safe.","When the true solution is mildly nonlinear, the decayed scheme performs best because it polishes the linear solution cheaply; when the nonlinearity is strong, the augmented scheme should be more robust."],"supporting_citations":[{"why":"Provides the baseline PDE-loss training program and the sinusoidal-network architecture that the new losses augment.","marker":"Bansal and Tomlin (2021)"},{"why":"Introduces the Hopf formula used to generate the linear supervisor $V_\\ell$ without spatial discretization.","marker":"Darbon and Osher (2016)"},{"why":"Proves that the game value is the viscosity solution of the Hamilton-Jacobi PDE, justifying the PDE residual loss.","marker":"Evans and Souganidis (1984c)"},{"why":"Supplies the sinusoidal periodic activation functions used in the value-network architecture.","marker":"Sitzmann et al. (2020)"},{"why":"Provides the conservative linear-envelope bound used in the proof of Theorem 1 to control $|V - V_\\ell|$.","marker":"Sharpless et al. (2024a)"},{"why":"Certifies the generalized Hopf representation for the nonautonomous Hamilton-Jacobi equations that enter the bound.","marker":"Rublev (2000)"},{"why":"Gives the conformal-prediction expansion used to measure the probabilistic safe volume in the quadrotor experiments.","marker":"Lin and Bansal (2023)"},{"why":"Shows the least-squares supervision loss satisfies the local Polyak-Łojasiewicz condition, supporting convergence guarantees.","marker":"Liu et al. (2022)"},{"why":"Defines the backward reachable tube and the non-anticipative strategy formalism in which the value function is stated.","marker":"Mitchell et al. (2005b)"}],"fun_headline_variants":["Linear supervision accelerates high-dimensional neural control","Speed up neural differential games with linear solutions","Use linear value functions to learn nonlinear control faster","Accelerate neural control using linear PDE solutions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The methods depend on the linearized value $V_\\ell$ being a good approximation of the true nonlinear value $V$ over the region used for training; if the nonlinearity is strong enough to make the linearization error large, the supervision can pull the network away from the true solution instead of toward it.","fun_headline_variants_meta":{"raw":{"variants":["Linear supervision accelerates high-dimensional neural control","Speed up neural differential games with linear solutions","Use linear value functions to learn nonlinear control faster","Accelerate neural control using linear PDE solutions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000785,"raw_usage":{"total_tokens":3428,"prompt_tokens":871,"completion_tokens":2557,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":2501}},"tokens_in":487,"tokens_out":2557,"duration_ms":214624,"temperature":1.0,"reasoning_tokens":2501,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:54:22.453431+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train both semi-supervision programs on a high-dimensional nonlinear system where the linearization error is known to be large, for example a 50-D publisher-subscriber game with large $\\alpha$ and $\\beta$ so that $\\delta^*$ from Theorem 1 is comparable to the value range, and compare intersection-over-union and mean-squared error against the PDE-only baseline; if the supervised programs do not beat the baseline, the claim that linear supervision reliably improves learned value functions is refuted.","supporting_citations":[{"cited_title":"DeepReach : A deep learning approach to high-dimensional reachability","cited_arxiv_id":null,"evidence_quote":"Provides the baseline PDE-loss training program and the sinusoidal-network architecture that the new losses augment."},{"cited_title":"Algorithms for overcoming the curse of dimensionality for certain Hamilton--Jacobi equations arising in control theory and elsewhere","cited_arxiv_id":null,"evidence_quote":"Introduces the Hopf formula used to generate the linear supervisor $V_\\ell$ without spatial discretization."},{"cited_title":"Generalized Hopf formulas for the nonautonomous Hamilton-Jacobi equation","cited_arxiv_id":null,"evidence_quote":"Certifies the generalized Hopf representation for the nonautonomous Hamilton-Jacobi equations that enter the bound."},{"cited_title":"Verification of Neural Reachable Tubes via Scenario Optimization and Conformal Prediction","cited_arxiv_id":"2312.08604","evidence_quote":"Gives the conformal-prediction expansion used to measure the probabilistic safe volume in the quadrotor experiments."}],"review_version":1}