{"id":"f61bce89-94e5-470f-950e-650b1df52302","arxiv_id":"2507.15455","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A PINN-based policy iteration method for nonconvex Hamilton-Jacobi-Isaacs equations with a convergence analysis and tests up to 10 dimensions.","lead":"Researchers propose a mesh-free algorithm that alternates between solving linear PDEs with neural networks and updating game strategies via automatic differentiation, aimed at high-dimensional stochastic differential games. The authors prove convergence to the viscosity solution under assumptions and report accurate results on two- to ten-dimensional benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's limit passage from the policy-evaluation PDEs to the HJI equation is not justified: equi-Lipschitz convergence of v_n does not imply convergence of ∇v_n, and the invoked viscosity stability property is not applicable as written.","rationale":"The paper combines a useful PINN--policy-iteration scheme with an attempted rigorous convergence theory. The numerical experiments, hard terminal-condition ansatz, and the residual-controlled error bound are genuinely interesting, and the experimental comparison with direct PINN solvers is plausible. However, the central theoretical claim depends on identifying the uniform limit of the policy-iteration value functions with the viscosity solution of the HJI equation. The proof of Theorem 1 does not establish the required consistency of the Hamiltonians: it passes from L(t,x,∇v_n)(π_n) to H(t,x,∇v) without proving convergence of the policies or gradients. Since Lemma 1 only gives Lipschitz dependence of the feedback selector on the gradient, the defect is controlled by |∇v_n−∇v_{n−1}|, for which no decay is shown. This is a genuine gap in the argument, not merely a disagreement with existing consensus. Proposition 1 amplifies the issue because its exponential-rate conclusion presupposes the limit v and depends on an unstated gradient bound; Theorem 2 then inherits both difficulties in its use of Proposition 1. The reader's weakest assumption (Assumption 1 does not cover the high-dimensional experiments) is real and security-relevant, but the proof gap in Theorem 1 is more fundamental because it threatens the theoretical guarantee even in cases where Assumption 1 holds. These problems are likely addressable by adding a suitable consistency or monotonicity argument, so a conditional verdict is appropriate rather than outright rejection.","tokens_in":19626,"tokens_out":9265,"duration_ms":96686,"concrete_test":"Independently re-derive the viscosity limit in Theorem 1: for a smooth test function φ, write the weak form of the policy-evaluation PDE for v_n and isolate the defect D_n = ∫ (L(t,x,∇v_n)(π_n) − H(t,x,∇v_n)) φ dx. Show, using only Assumption 1 and the established uniform convergence of v_n, that limsup/liminf D_n vanishes. If this requires assuming ∇v_n→∇v locally uniformly, which the proof does not establish, Theorem 1 is incomplete. As a numerical check on a one-dimensional nonconvex HJI equation satisfying Assumption 1, measure sup |∇v_n−∇v|: if it fails to decay while v_n→v uniformly, the stated proof path collapses.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing gap is the final step of the proof of Theorem 1 (Section 3.1). The argument shows only that {v_n} is uniformly bounded and equi-Lipschitz, so a subsequence converges locally uniformly. But the PDE solved by v_n is ∂_t v_n + L(t,x,∇v_n)(π_n) + 1/2 Tr(σσ^T D²v_n)=0, where π_n is the minimax selector evaluated at ∇v_{n−1}, not at ∇v_n. Consequently the nonlinear term is L(t,x,∇v_n)(π_{n−1}*), not H(t,x,∇v_n). The final sentence \"by the stability property of the viscosity solution [26], v solves (3.1)\" would require convergence of these Hamiltonians, e.g. |L(t,x,∇v_n)(π_n)−H(t,x,∇v_n)|→0 in a suitable viscosity sense. By Lemma 1 this defect is bounded by κ‖f‖_∞|∇v_n−∇v_{n−1}|, and equi-Lipschitz continuity alone does not give uniform gradient convergence. Without this step, the HJI limit is not identified. The same missing ingredient underlies Proposition 1: the contraction F_n ≤ ρF_{n−1} relies on an unproved parabolic gradient bound attributed to [6], and if Theorem 1 is incomplete there is no established limit v to which the exponential L² rate applies.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a mesh-free policy iteration method for nonconvex Hamilton-Jacobi-Isaacs (HJI) equations, in which each iteration solves a linear parabolic PDE with a fixed feedback policy via a PINN and then updates the feedback controls by pointwise minimax optimization using automatic differentiation. The main theoretical claims are Theorem 1 (local uniform convergence of the policy-evaluation value functions to the unique viscosity solution of the HJI equation), Proposition 1 (an exponential L2 convergence rate for the exact policy iteration), and Theorem 2 (a global error bound for the practical PINN implementation in terms of the PDE residual and the iteration error). Numerical experiments are reported for a 2D moving-obstacle path-planning game and for 5D and 10D publisher-subscriber games. The paper is clearly organized and the algorithmic description is concrete, but the central convergence arguments contain several load-bearing gaps, and the high-dimensional experiments are run outside the stated assumptions.","tokens_in":19882,"tokens_out":7691,"duration_ms":90003,"significance":"If the convergence theorems were established, the paper would make a useful contribution: a PINN-based policy iteration with a residual-controlled error bound is attractive for nonconvex HJI equations, and the hard-terminal-condition ansatz in (3.9) plus the pointwise minimax update via automatic differentiation are sensible algorithmic choices. The proof of Lemma 1 is sound under the strong convexity-concavity assumption, and the 2D experiment provides a quantitative finite-difference comparison. However, the significance is conditional, because Theorem 1, Proposition 1, and Theorem 2 all rely on regularity or stability steps that are not proven, and because the 5D/10D experiments violate Assumption 1. The paper should be revised so that the theoretical results are either completed or explicitly restricted to settings where the assumptions hold.","major_comments":[{"comment":"The final step of the proof is not justified. The argument establishes only that a subsequence of {v_n} converges locally uniformly by Arzela-Ascoli. However, the policy-evaluation PDE solved by v_n contains L(t,x,∇v_n)(α_n,β_n), where (α_n,β_n) is the feedback pair computed from ∇v_{n-1} in Algorithm 1, not from ∇v_n. By Lemma 1, |L(t,x,∇v_n)(π_n)-H(t,x,∇v_n)| is bounded by κ||f||∞|∇v_n-∇v_{n-1}|, and equi-Lipschitz convergence of v_n does not imply uniform convergence of ∇v_n. Therefore the cited stability property of viscosity solutions ([26]) cannot be applied as written; one would need to prove, for example, uniform convergence of ∇v_n or a viscosity notion of convergence of the Hamiltonians. Without this step, the limit v is not identified as the solution of the HJI equation (3.1).","section":"§3.1, proof of Theorem 1"},{"comment":"The exponential contraction F_n ≤ ρF_{n-1} depends entirely on the asserted gradient bound ||∇xδ_n(t,·)||_2 ≤ C̃√T (F_n + F_{n-1}), which is attributed to [6] without a statement or proof. The cited reference is a standard PDE textbook and does not contain this specific estimate. Moreover, the bound as written has F_n on both sides, so the displayed derivation only gives (1-C̃√T)F_n ≤ C̃√T F_{n-1}; a valid proof must establish the estimate with a constant that does not presuppose the desired contraction. Since Proposition 1 is also used in Theorem 2, this gap is load-bearing for both the exponential rate and the global error bound.","section":"§3.1, Proposition 1 (Eq. 3.7)"},{"comment":"The parabolic L2 estimate in Proposition 2 is applied to viscosity solutions, but the energy inequality d/dt||v||_2^2 + λ||∇v||_2^2 ≤ ... is derived by formal integration by parts and requires v(t,·) ∈ H^1 (and enough regularity to justify the manipulation). No regularization or approximation argument is given to bridge the gap from viscosity solutions to this energy class. In addition, when σσ^T depends on x, the integration by parts produces extra terms involving ∇·(σσ^T) that are not accounted for in the stated inequality. Theorem 2 uses Proposition 2 to control both A_n and B_n, so the global error estimate (3.10) is not rigorously established.","section":"Appendix A, Proposition 2"},{"comment":"The 5D and 10D publisher-subscriber problems do not satisfy Assumption 1. The control and disturbance enter the drift linearly as Bu and Cd, so the Lagrangian L(t,x,p)(u,d) is not strongly convex-concave in (u,d); moreover, the drift contains ψ(x) with cubic terms, so f is neither globally bounded nor globally Lipschitz on R^N. Consequently Theorems 1 and 2 and Proposition 1 do not apply to the experiments in Section 4.3. The paper does not flag this mismatch, and the claim that the iterative framework yields provable L2 bounds in these high-dimensional nonconvex settings is therefore not supported.","section":"§4.3, publisher-subscriber experiments vs. Assumption 1"}],"minor_comments":[{"comment":"There is a typo 'v′ns' in the equi-Lipschitz estimate paragraph; it should read 'v_n's'.","section":"§3.1, proof of Theorem 1"},{"comment":"The stability property of viscosity solutions is cited to [26], an actor-critic flow paper; a standard reference such as [5] (Crandall-Ishii-Lions) would be more appropriate and would make the intended argument easier to verify.","section":"§3.1, proof of Theorem 1"},{"comment":"The theorem statement says 'where (α_n,β_n) is the exact feedback pair obtained from v_{n-1}', but Algorithm 2 computes policies from the approximate value function ṽ_{n-1}. This notational inconsistency should be clarified, since the proof of the B_n term depends on comparing exact and approximate policies.","section":"§3.2, Theorem 2 statement"},{"comment":"The value-function decomposition proof assumes 'sufficient regularity for the chain rule' and block-diagonal or isotropic σ, but these assumptions are not stated in the main text and are not satisfied by the anisotropic experiments in Section 4.3. The global claim that the decomposition holds should be qualified accordingly.","section":"Appendix B"},{"comment":"The 'Direct PINN epoch' entry for the publisher-subscriber setting is listed as a bare number 2,500,000 without indicating whether it is the total number of epochs or a product; this is confusing and should be written explicitly as E × M.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a timely topic and the algorithmic idea is appealing, but in its current form the main theoretical results are not established because of the missing gradient-convergence/stability argument in Theorem 1, the unproved gradient bound in Proposition 1, and the formal nature of Proposition 2. The high-dimensional experiments do not exercise Assumption 1, so they cannot compensate for the proof gaps. I recommend major revision rather than rejection, because the gaps appear fixable with additional parabolic regularity estimates and a more careful stability passage, or by substantially narrowing the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful paper to know about, but not because the theorems are airtight. The core idea—alternate linear policy evaluation PDEs with pointwise minimax updates, and represent each v_n by a PINN—is sensible and the numerics support that it can work. The 2D obstacle problem matches FDM to ~1e-3 relative L2, and the 5D/10D publisher–subscriber results beat a direct PINN baseline. Hard terminal enforcement via the ansatz is a good practical choice.\n\nWhat's genuinely new: a continuous-time policy iteration convergence statement for uniformly elliptic HJI, an equi-Lipschitz compactness argument, and an L2 error bound that ties the PINN residual to the total error. These are reasonable targets, and the paper's structure is clean.\n\nThe soft spots are real. Theorem 1's final step is not justified. You get a subsequence of {v_n} converging locally uniformly, but the policy-evaluation PDE contains L(t,x,∇v_n)(π_n) where π_n was built from ∇v_{n-1}. Showing the limit solves the HJI equation requires convergence of those Hamiltonians, which needs ∇v_n → ∇v in some viscosity sense. Equi-Lipschitzness doesn't give that, and the cited stability result [26] is not a standard viscosity stability theorem covering this situation. So the limit is not identified as the value function. Proposition 1 inherits the problem: the contraction F_n ≤ ρF_{n-1} depends on an unproved 'gradient bound of δ_n demonstrated in [6]', and [6] is Evans's PDE book, which doesn't contain that estimate for this setting. Theorem 2 builds on Proposition 1, so it also needs patching, plus there are notational mismatches in its proof.\n\nThe experiments also run outside Assumption 1 without comment: in the publisher–subscriber problem the drift has quadratic terms, the controls enter linearly, and the Hamiltonian has l1 terms; there is no strong convexity-concavity or global Lipschitz boundedness. That doesn't make the numerics worthless, but it means the theory as stated doesn't cover the headline experiments. No code or error bars either, though the result tables are clear.\n\nWho is this for? Anyone working on PINNs for HJB/HJI or on neural policy iteration will want to see it. The flaws are addressable, and if the gradient-convergence step can be fixed, the L2 residual bound would be a nice result. I'd send it to a serious referee, but my recommendation would be major revision: fix or clearly restate the convergence proof, add the missing gradient estimate or drop the exponential rate claim, and either adjust the experiments to satisfy the assumptions or state the mismatch honestly.","headline":"The method is sensible and the 2D/5D/10D results are decent, but the main convergence proof is missing the step that identifies the limit equation, Proposition 1 leans on an unproved gradient bound, and the experiments run outside Assumption 1; this deserves a serious referee but not acceptance as-is.","tokens_in":20450,"tokens_out":3364,"would_cite":false,"duration_ms":38231,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["49L25","65M99","35D40","91A23"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proves that a mesh-free policy-iteration scheme built on physics-informed neural networks converges locally uniformly to the unique viscosity solution of nonconvex Hamilton–Jacobi–Isaacs equations, and bounds the neural…","keywords":["Hamilton–Jacobi–Isaacs equations","policy iteration","physics-informed neural networks","viscosity solutions","nonconvex Hamiltonians","zero-sum stochastic differential games","uniform ellipticity","high-dimensional PDEs"],"falsifier":"To test the core convergence claim directly, take a one-dimensional zero-sum game with explicit solution—say $L(t,x,p)(a,b)=\\frac{\\mu_A}{2}a^2-\\frac{\\mu_B}{2}b^2+p(a+b)$, $\\sigma=0.1$, terminal cost $g(x)=x^2$—and verify that idealized policy iteration (Algorithm 1) converges locally uniformly to the viscosity solution for positive $\\mu_A,\\mu_B$. Then shrink $\\mu_A=\\mu_B$ toward zero: Lemma 1's Lipschitz constant grows as $1/\\mu$, so failure of convergence while Assumption 1 still holds would falsify Theorem 1, while continued convergence would show the strong convexity–concavity hypothesis is not necessary.","tokens_in":19382,"feed_emoji":"🎯","tokens_out":10331,"duration_ms":102279,"temperature":0.7,"pith_summary":"Policy iteration, a classical dynamic-programming scheme, is usually analyzed on grids; this paper brings it into the mesh-free PINN world for nonconvex Hamilton–Jacobi–Isaacs (HJI) equations. The authors show that when the game Lagrangian is uniformly strongly convex in one player's control and strongly concave in the other's, the value iterates converge locally uniformly to the unique bounded viscosity solution, with no convexity assumed on the Hamiltonian. For the practical neural-network implementation they prove $\\sup_{t\\in[0,T]}\\|\\tilde v_n(t,\\cdot)-v(t,\\cdot)\\|_2 \\le C(p_n+\\rho^n)$, separating the network residual $p_n$ from an exponentially decaying iteration error $\\rho^n$. If correct, this gives high-dimensional zero-sum differential games a solver with explicit $L^2$ error control and a built-in near-optimal feedback policy, and the 2D/5D/10D experiments support the scalability claim.","feed_headline":"Nonconvex HJI equations yield to PINN policy iteration","feed_subtitle":"A mesh-free loop of linear PDE solves and minimax updates gives provable L2 error bounds in up to 10 dimensions.","key_machinery":"The engine is the policy-iteration loop for the HJI equation. In the policy-evaluation step, the current feedback pair $(\\alpha_n,\\beta_n)$ is frozen and the linear parabolic PDE $\\partial_t v_n + L(t,x,\\nabla v_n)(\\alpha_n,\\beta_n) = -\\tfrac{1}{2}\\operatorname{Tr}(\\sigma\\sigma^\\top D^2_{xx}v_n)$ is solved; the PINN represents $v_n$ with the hard-constraint ansatz $v_n = g + (T-t)N_n(t,x;\\theta_n)$, so the terminal condition is enforced exactly and all derivatives come from automatic differentiation. In the policy-improvement step, the new controls are the pointwise minimax pair of the Lagrangian at $p=\\nabla_x v_n$, whose existence, uniqueness, and global Lipschitz dependence on $p$ are the content of Lemma 1. That Lipschitz feedback selector, together with the Feynman–Kac representation and the parabolic $L^2$ energy estimate of Proposition 2, is what carries the equi-Lipschitz regularity and the contraction rate $\\rho$ in Proposition 1 and Theorem 2.","core_discovery":"The paper's central claim is that policy iteration can solve nonconvex HJI equations without grids and without convexity of the Hamiltonian, provided the dynamics satisfy Assumption 1. Each iteration solves a linear parabolic PDE for the current feedback pair—a policy-evaluation step that inherits Feynman–Kac representation and Schauder regularity—and then updates the controls by the pointwise saddle point of the Lagrangian at the current value gradient; Lemma 1 shows this minimax feedback selector is globally Lipschitz because of the strong convexity–concavity of the Lagrangian. Theorem 1 states the resulting value iterates converge locally uniformly to the unique bounded continuous viscosity solution, and Theorem 2 quantifies the error of the practical algorithm as $C(p_n+\\rho^n)$, where $p_n$ is the $L^2$ norm of the PINN residual and $\\rho<1$. The experiments in two, five, and ten dimensions are offered as evidence that the framework is accurate and scalable, with lower residuals and smoother value functions than direct PINN training.","pith_inferences":["The 5D and 10D benchmarks use linear controls with $\\ell^1$ penalties and polynomial drift, which do not satisfy Assumption 1's strong convexity–concavity or global boundedness; if those results hold up, the convergence theory is sufficient but not necessary, and may be extendable to merely convex–concave Lagrangians with compact controls.","Appendix B shows the publisher–subscriber value function splits into a sum of two-dimensional problems when noise is isotropic; the reported 5D/10D accuracy may therefore chiefly measure the approximation of 2D subproblems, not genuine high-dimensional approximation.","The bound $C(p_n+\\rho^n)$ suggests a natural stopping criterion—iterate until the residual plateaus—and also implies that the neural-network approximation error, not the policy iteration, is the ultimate bottleneck of the practical solver."],"forward_implications":["Under Assumption 1, the practical PINN policy-iteration algorithm has total error at most $C(p_n+\\rho^n)$, so a user can monitor the PDE residual $p_n$ and stop iterating once it plateaus.","Every iteration returns a near-optimal feedback policy by a pointwise minimax formula, so no separate control optimization is needed after the value network is trained.","The hard-constraint ansatz $v_n=g+(T-t)N_n$ removes the terminal loss term from the PINN objective, which the paper links to more stable training and smoother value functions.","The method matches a finite-difference reference in the 2D moving-obstacle game with relative $L^2$ errors around $10^{-3}$, and in the 5D and 10D publisher–subscriber games it outperforms direct PINN training in residual and relative $L^2$ error."],"supporting_citations":[{"why":"Supplies the discrete-time policy-iteration scheme for nonconvex viscous HJI equations that Algorithm 1 adapts to the uniformly elliptic, continuous-time, mesh-free setting.","marker":"[9]"},{"why":"Provides the viscosity solution theory—comparison, uniqueness, and stability—used to identify the limit in Theorem 1.","marker":"[5]"},{"why":"Gives the Feynman–Kac representation for linear parabolic equations, used to bound the value iterates uniformly and prove their equi-Lipschitz regularity.","marker":"[8]"},{"why":"Provides the gradient estimates invoked in Proposition 1 to convert $L^2$ differences of consecutive iterates into a contraction on gradients.","marker":"[6]"},{"why":"Develops the $L^2$ energy-estimate route for policy iteration on deterministic control problems that Proposition 1 adapts to the stochastic HJI setting.","marker":"[24]"},{"why":"Establishes $L^2$ energy estimates for policy iteration on exploratory HJB equations, the technical template for the exponential-rate proof.","marker":"[25]"},{"why":"Cited for the stability property of viscosity solutions used in the final step of Theorem 1's proof.","marker":"[26]"},{"why":"Defines the physics-informed neural network residual-minimization framework used in the policy-evaluation loss.","marker":"[21]"},{"why":"Introduces the hard-constraint ansatz for enforcing boundary or terminal conditions, which the paper uses to eliminate the terminal loss term.","marker":"[17]"}],"fun_headline_variants":["Mesh-free PINN policy iteration solves nonconvex HJI up to 10D","PINN policy iteration provably solves nonconvex HJI","No grids, no convexity: PINN policy iteration handles HJI","Mesh-free PINN iteration converges for nonconvex HJI","Proven PINN policy iteration for nonconvex HJI equations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole argument rests on Assumption 1: the game's payoff must be uniformly strongly convex in the minimizing player's control and uniformly strongly concave in the maximizing player's control at every state, while the drift, cost, and diffusion remain bounded, Lipschitz, and uniformly elliptic—if any of that fails, the Lipschitz feedback selector and the equi-Lipschitz estimates that carry the proof are not available.","fun_headline_variants_meta":{"raw":{"variants":["Mesh-free PINN policy iteration solves nonconvex HJI up to 10D","PINN policy iteration provably solves nonconvex HJI","No grids, no convexity: PINN policy iteration handles HJI","Mesh-free PINN iteration converges for nonconvex HJI","Proven PINN policy iteration for nonconvex HJI equations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000562,"raw_usage":{"total_tokens":2698,"prompt_tokens":1005,"completion_tokens":1693,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":1599}},"tokens_in":621,"tokens_out":1693,"duration_ms":12669,"temperature":1.0,"reasoning_tokens":1599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:34:16.046202+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"To test the core convergence claim directly, take a one-dimensional zero-sum game with explicit solution—say $L(t,x,p)(a,b)=\\frac{\\mu_A}{2}a^2-\\frac{\\mu_B}{2}b^2+p(a+b)$, $\\sigma=0.1$, terminal cost $g(x)=x^2$—and verify that idealized policy iteration (Algorithm 1) converges locally uniformly to the viscosity solution for positive $\\mu_A,\\mu_B$. Then shrink $\\mu_A=\\mu_B$ toward zero: Lemma 1's Lipschitz constant grows as $1/\\mu$, so failure of convergence while Assumption 1 still holds would falsify Theorem 1, while continued convergence would show the strong convexity–concavity hypothesis is not necessary.","supporting_citations":[{"cited_title":"Policy iteration for nonconvex viscous Hamilton--Jacobi equations","cited_arxiv_id":"2503.02159","evidence_quote":"Supplies the discrete-time policy-iteration scheme for nonconvex viscous HJI equations that Algorithm 1 adapts to the uniformly elliptic, continuous-time, mesh-free setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the viscosity solution theory—comparison, uniqueness, and stability—used to identify the limit in Theorem 1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the Feynman–Kac representation for linear parabolic equations, used to bound the value iterates uniformly and prove their equi-Lipschitz regularity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the gradient estimates invoked in Proposition 1 to convert $L^2$ differences of consecutive iterates into a contraction on gradients."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Develops the $L^2$ energy-estimate route for policy iteration on deterministic control problems that Proposition 1 adapts to the stochastic HJI setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes $L^2$ energy estimates for policy iteration on exploratory HJB equations, the technical template for the exponential-rate proof."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the hard-constraint ansatz for enforcing boundary or terminal conditions, which the paper uses to eliminate the terminal loss term."}],"review_version":1}