{"id":"4b254317-5ba3-44fd-982b-dbf8531b4ef4","arxiv_id":"2412.17363","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A gradient projection method with least-squares Monte Carlo discretization is proved to converge with first order for control and multiplier in stochastic optimal control with expected integral state constraints, under linear drift.","lead":"This paper develops a numerical method for stochastic optimal control problems in which the expected average of the state must stay below a given limit. The method combines gradient projection with Monte Carlo regression, and the authors prove first-order accuracy for the control and the constraint multiplier.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"First-order claim for the LSMC scheme hinges on an unproved O(Δt) bound for the empirical regression error in Theorem 4.7.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing gap: the missing O(Δt) bound for the empirical regression error in Theorem 4.7. The paper's own Remark 4.8 admits this analysis is deferred, and the final paragraph of §4.2 asserts, rather than proves, that sufficiently many paths and basis functions make ε_N first order. This is not a minor technicality: Theorems 4.2 and 4.3 both have the form C(Δt + ε_N), so without ε_N = O(Δt) the advertised first-order convergence for the LSMC scheme is unsubstantiated. The grid-method variant is better supported through [11], and the numerical experiments are consistent with first-order rates, but the abstract and conclusion claim first-order convergence for the LSMC method as well. I considered other possible concerns, such as the unverified strong monotonicity assumption on J', but the empirical regression bound is the single most load-bearing issue because it is the one step the paper explicitly leaves open. Since the reader already assigned CONDITIONAL for this reason, my independent review does not change the verdict.","tokens_in":32420,"tokens_out":6433,"duration_ms":64847,"concrete_test":"Analytical check: in the setting of §4.1 with L ≥ cN^2 and a specific basis family (e.g., hypercube indicators with τ = O(Δt^{3/2}), or fixed-degree local polynomials), derive explicit bounds for max_n E[(P̂^K_n − P̂^{K,L}_n)^2] and Σ_n Δt E[(Q̂^{\\tilde K}_n − Q̂^{\\tilde K,L}_n)^2] using the results cited in Remark 4.8 (e.g., Theorem 2 of [27] or the estimates in [28]). If the resulting upper bound is O(Δt^2), the first-order claim stands; if it is only O(K^2/L), O(K log L / L), or otherwise not O(Δt^2) under the stated parameter choices, then ε_N = O(Δt) does not follow and Theorems 4.2–4.3 only deliver a conditional rate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The advertised first-order convergence of the fully discrete LSMC scheme (Theorems 4.2 and 4.3) requires ε_N = sup_i ‖J'(u^{i,N}) − J'_N(u^{i,N})‖ = O(Δt). Theorem 4.7 bounds ε_N^2 by temporal, projection, and empirical regression terms, but the two empirical terms — max_n E[(P̂^K_n − P̂^{K,L}_n)^2] and Σ_{n} Δt E[(Q̂^{\\tilde K}_n − Q̂^{\\tilde K,L}_n)^2] — are not controlled by any theorem in the paper. Remark 4.8 explicitly defers this analysis to [27,28], and the closing assertion of §4.2 that 'enough' paths and basis functions make ε_N first-order is not a proof. Since Theorems 4.2 and 4.3 both contain ε_N additively, the headline claim that the LSMC-based Algorithm 1 admits first-order convergence is not established. The grid-method variant does not share this gap because its ε_N = O(Δt) is cited from [11], but the paper's abstract and conclusion claim first-order convergence for the LSMC method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a gradient projection method for stochastic optimal control problems with an expected integral state constraint under a linear drift assumption. It derives the first-order optimality system with a Lagrange multiplier, constructs an explicit projection by choosing the multiplier to enforce the state constraint, and then discretizes the resulting FBSDEs by the Euler scheme in time and by least-squares Monte Carlo in space. Error estimates for the control and the multiplier are stated, and the authors claim first-order convergence for both the LSMC scheme and the grid-method variant, supported by numerical experiments.","tokens_in":32674,"tokens_out":11744,"duration_ms":115035,"significance":"If the first-order claim were fully established, this would be a useful contribution: the explicit multiplier construction is elegant and self-contained, and extending efficient gradient projection to expected integral state constraints is nontrivial. The grid-method variant inherits a first-order estimate from the cited literature [11], and the numerical experiments are consistent with first-order rates for both the control and the multiplier. However, the genuinely new LSMC convergence claim depends on an empirical-regression error bound that is not proved, so the significance of the paper is currently conditional on closing that gap.","major_comments":[{"comment":"The advertised first-order convergence of the fully discrete LSMC scheme is not established. Theorem 4.7 bounds ε_N^2 by a temporal term, two projection terms, and two empirical regression terms, namely max_n E[(P̂^K_n(y_n) − P̂^{K,L}_n(y_n))^2] and Σ_n Δt E[(Q̂^{K̃}_n(y_n) − Q̂^{K̃,L}_n(y_n))^2]. No theorem in the paper controls these two terms: Lemma 4.5 concerns only the projection error between P̂ and P̂^K, and the sentence after Remark 4.8 merely asserts that enough paths and basis functions make ε_N first-order. Remark 4.8 explicitly defers the analysis of the simulation step to [27,28]. Since Theorems 4.2 and 4.3 contain ε_N additively, the claimed O(Δt) convergence for Algorithm 1 is not proven as stated. The grid-method variant, whose ε_N=O(Δt) is cited from [11], is not affected by this gap.","section":"§4.2, Theorem 4.7 and Remark 4.8"},{"comment":"The claimed Monte Carlo bound O(L^{-1/2}) in (4.37) treats the LSMC approximations P̂^{K,L}_n and Q̂^{K̃,L}_n as fixed functions while the empirical average is taken over the L paths. In the actual scheme these regression functions are estimated from the same L paths, so a valid estimate must include the regression-estimation error, which is exactly one of the terms left uncontrolled in Theorem 4.7. Thus the decomposition in Theorem 4.7 does not reduce the empirical regression error to a standard Monte Carlo error; it only relocates the difficulty. This is a load-bearing point for the paper's central claim, not a presentation issue.","section":"§4.2, Eq. (4.37)"}],"minor_comments":[{"comment":"The notation J′_N and J'_N is used for both the expectation-based estimator and the Monte Carlo estimator; the two objects should be distinguished by different symbols to make the proof readable.","section":"§4.2, Eq. (4.37) and surrounding text"},{"comment":"In Eq. (4.14), the factor (1 + C_b/N)^N should read (1 + C_b T/N)^N since Δt = T/N; the displayed estimate loses T.","section":"§4.1, Theorem 4.1 proof"},{"comment":"After 'i → +∞', terms such as α^{i+1}‖u^{*N} − u^{0,N}‖ vanish; they should not be bounded below by Δt‖u^{*N} − u^{0,N}‖. The intended statement is a limsup estimate, and the current display is confusing.","section":"§4.2, Eq. (4.15)"},{"comment":"The constant C_ψ is used in (4.30) but never defined; it should be introduced as the constant from Theorem 4.1 controlling max_n |ψ(t_n) − ψ_n|.","section":"§4.2, Eq. (4.30)"},{"comment":"The numerical section does not report the number of basis functions K and K̃ used for the HC and VP bases. Since the unproved regression error depends on these parameters, reporting them would help the reader assess the gap between theory and experiment.","section":"§5, numerical experiments"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the paper relies substantially on [11] (co-authored by W. Liu) for the projection lemma and the grid-method error estimate. The genuinely new LSMC component is precisely where the proof gap occurs. If the authors cannot supply a rigorous bound for the empirical regression terms, the first-order statement should be withdrawn for the LSMC variant and the paper reframed as a numerical study plus grid-method convergence. The gap may be fixable by importing a known LSMC regression error bound from [27,28], so I do not see this as grounds for rejection, but the current version overstates the theorem."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing is the explicit multiplier construction for the expected integral state constraint. That construction, Theorem 3.1, is clever and mostly correct, and it turns an implicit constraint into a computable gradient projection update. The paper is also the first numerical analysis for this class of SOCPs; the grid-method variant rests on the established estimate from [11] and looks solid. I would not fault the self-citation there, since the cited result is a published theorem.\n\nThe soft spot is the advertised first-order convergence of the LSMC scheme. Theorem 4.7 decomposes ε_N into temporal, projection, and empirical regression errors, but the two regression terms — the max over n of E[(P̂^K_n − P̂^{K,L}_n)^2] and the sum of Δt E[(Q̂^{K̃}_n − Q̂^{K̃,L}_n)^2] — are not controlled by any theorem in the paper. Remark 4.8 explicitly says the analysis is 'rather involved and technical' and defers to [27,28]. Since Theorems 4.2 and 4.3 both contain ε_N additively, the claim that Algorithm 1 converges at first order for the control and multiplier is not established. The numerical experiments are consistent with first order, but an experiment is not a proof. This is a load-bearing gap for the LSMC part of the paper, though not for the grid-method variant.\n\nTwo smaller points. In the proof of Theorem 4.2, the term α^{i+1}‖u∗,N − u0,N‖ is replaced by Δt times the same norm as i→∞; it should vanish. The conclusion is unaffected, but the displayed inequality is wrong as written. Also, the abstract says the method 'admits first order convergence' without the caveat that the LSMC result depends on an unproved bound; that should be qualified.\n\nMy recommendation is to send it to serious peer review. The core algorithmic idea is sound and the error-anatomy framework is useful; the missing regression bound is real but seems fixable, either by supplying the bound from the cited literature or by restating the LSMC result as first order up to an unquantified regression error. A referee with BSDE/LSMC background can push on this. If the gap is closed, this is a solid paper.","headline":"Useful extension of the gradient-projection framework to expected integral state constraints, but the advertised first-order rate for the LSMC scheme rests on an unproved regression-error bound.","tokens_in":33131,"tokens_out":4511,"would_cite":true,"duration_ms":40618,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60H35","65K10","65C20","93E20"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's central claim is that a gradient projection method with an explicit multiplier achieves first-order convergence for both control and multiplier in stochastic optimal control with an expected integral state constraint.","keywords":["stochastic optimal control","expected integral state constraint","backward stochastic differential equation","gradient projection method","least square Monte Carlo","first-order convergence","Lagrange multiplier"],"falsifier":"Run Example 5.2 with $L=C N^2$ paths and measure $\\varepsilon_N = \\sup_i \\|J'(u^{i,N}) - J'_N(u^{i,N})\\|$ directly; if the measured quantity decays slower than $\\Delta t$, say like $\\Delta t^{1/2}$, then the first-order claim for the LSMC scheme fails because the deferred regression-error bound in Remark 4.8 is not satisfied. Equivalently, compare the convergence slopes with $L=N^2$ and $L=N^3$: if the slope drops below one when $L=N^2$, the missing regression term is the bottleneck.","tokens_in":32173,"feed_emoji":"🎯","tokens_out":10555,"duration_ms":95510,"temperature":0.7,"pith_summary":"This paper tackles stochastic optimal control problems in which the expected time-integral of the state is required to stay below a given level $\\delta$. The authors derive the first-order optimality system through a Lagrange multiplier and a backward stochastic differential equation, then propose an iterative gradient projection method that enforces the state constraint at every step by constructing the multiplier explicitly. Discretizing the forward-backward system with Euler time stepping and approximating conditional expectations by least-squares Monte Carlo yields a fully computable scheme. The main claim is that both the control and the multiplier converge at first order in the time step, provided the Monte Carlo regression error is controlled. Numerical experiments, including a five-dimensional example, support the predicted rates.","feed_headline":"First-order solver for state-constrained stochastic control","feed_subtitle":"An explicit multiplier keeps the expected state within bounds; control and multiplier errors fall at first order.","key_machinery":"The load-bearing object is the projection operator with an explicit multiplier. For linear drift, the projection $P$ onto the convex feasible set $U(\\delta)$ is realized by shifting the unprojected control along the adjoint direction $\\psi(t)b'_u(t)$, where $\\psi$ solves the ODE $-d\\psi=(1+\\psi b'_y(t))dt$, $\\psi(T)=0$. The multiplier $\\mu$ is selected so that the expected integral state exactly hits the constraint when the tentative control violates it, using the companion ODE $\\phi$ whose integral $\\int_0^T\\phi(t)\\,dt$ normalizes the correction. In the fully discrete scheme the same construction is applied to piecewise-constant controls, with the $L^2$ projection onto step functions and with $J'$ replaced by its LSMC approximation $J'_N$; the contraction argument for the iteration then converts the projection error and the gradient approximation error $\\varepsilon_N$ into the stated first-order bounds.","core_discovery":"The central discovery is that for linear-drift state dynamics $b(y,u)=b'_y(t)y+b'_u(t)u+m(t)$, the projection onto the feasible control set $U(\\delta)$ has an explicit form: a tentative gradient step $u^{i+1/2,N}=u^{i,N}-\\rho J'_N(u^{i,N})$ is corrected by subtracting $\\rho\\,\\mu^{i,N}_{h,KL}\\sum_n \\psi_n b'_u(t_n)\\chi_{I_n^N}(t)$, where the multiplier $\\mu^{i,N}_{h,KL}$ is chosen by the formula $\\max\\{E[\\int_0^T \\hat y^{i+1,N}_t dt]-\\delta,0\\}/(\\rho \\int_0^T \\tilde\\phi(t)\\,dt)$. This choice guarantees $E[\\int_0^T y^{i+1,N}_t dt]\\le\\delta$ at every iteration. The paper proves error bounds $\\limsup_{i\\to\\infty}\\|u^*-u^{i+1,N}\\|\\le C(\\Delta t+\\varepsilon_N)$ and $\\limsup_{i\\to\\infty}|\\mu^* - \\mu^{i,N}_h|\\le C(\\varepsilon_N+\\Delta t)$, with $\\varepsilon_N=\\sup_i\\|J'(u^{i,N})-J'_N(u^{i,N})\\|$, and argues that with $L=O(N^2)$ paths and suitable basis functions $\\varepsilon_N=O(\\Delta t)$, yielding first-order convergence.","pith_inferences":["A natural extension the paper does not pursue is to replace the single scalar constraint by a vector of expected integral constraints; the same shift-along-adjoint idea would need one multiplier per constraint, with a vector-valued normalization matrix replacing $\\int_0^T\\phi\\,dt$.","The paper's error analysis separates time discretization error from projection error and Monte Carlo regression error; measuring those three terms separately on the paper's own examples would tell practitioners how to allocate $N$, $L$, and basis size.","The strong monotonicity and Lipschitz assumptions on $J'$ are likely to limit the method to convex-like problems; testing it on the nonconvex benchmark without exact solution in Example 5.3 with larger step sizes would reveal whether the contraction constant remains inside the unit interval.","If the deferred LSMC regression bound $O(\\Delta t)$ fails for some practical basis choice, one could still retain first-order control convergence by using a higher-order accurate regression estimator or by coupling LSMC with a variance-reduction strategy; this is an inference, not a claim of the paper."],"forward_implications":["If the error estimate is right, halving the time step halves the asymptotic error in both control and multiplier for linear-drift problems, so doubling $N$ yields slope-one convergence.","The state constraint is satisfied at every iterate by construction, not just in the limit, because the explicit multiplier formula is designed to keep $E[\\int_0^T y^{i+1,N}_t\\,dt]\\le\\delta$.","The method works without a space grid, using least-squares Monte Carlo, so it can be applied to higher-dimensional state spaces; a five-dimensional numerical example is included.","The grid-based version of the same algorithm also achieves $\\varepsilon_N=O(\\Delta t)$ under the cited regularity conditions, giving a first-order alternative when regression bases are hard to choose.","With $L=O(N^2)$ simulated paths and enough basis functions, the gradient approximation error is of order $\\Delta t$, so the fully discrete scheme inherits first-order convergence from the temporal discretization."],"supporting_citations":[{"why":"Supplies the gradient projection framework, the projection lemma, and the grid-method error estimate that the discrete analysis extends to state constraints.","marker":"[11]"},{"why":"Introduces the regression-based Monte Carlo basis functions (Voronoi partitions and hypercubes) used to approximate conditional expectations in the fully discrete scheme.","marker":"[25]"},{"why":"Provides the empirical regression error bound for BSDEs to which Remark 4.8 defers for the LSMC simulation error.","marker":"[27]"},{"why":"Companion analysis of empirical regression methods for BSDEs, also cited in Remark 4.8 for the deferred regression error bound.","marker":"[28]"},{"why":"Supplies Lemma 4.4, the temporal weak error expansions for the discretized BSDE used to estimate the time discretization part of $\\varepsilon_N$.","marker":"[31]"},{"why":"Supplies Lemma 4.5, the projection error estimate for least-squares regression of the BSDE solution used in the error decomposition.","marker":"[32]"},{"why":"Provides Robinson's regularity condition and the Lagrange multiplier theorem used to derive the optimality condition system.","marker":"[40]"}],"fun_headline_variants":["Explicit multiplier enforces state bound in stochastic control","First-order gradient projection with guaranteed state constraint","State-constrained stochastic control solved at first order","Projection method with closed-form multiplier for state constraints","Gradient projection ensures expected state bound each step"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The promised first-order convergence depends on the unproved assumption that the statistical regression error in the least-squares Monte Carlo gradient is no larger than the time step; the paper leaves that bound to earlier work rather than proving it for its algorithm.","fun_headline_variants_meta":{"raw":{"variants":["Explicit multiplier enforces state bound in stochastic control","First-order gradient projection with guaranteed state constraint","State-constrained stochastic control solved at first order","Projection method with closed-form multiplier for state constraints","Gradient projection ensures expected state bound each step"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000346,"raw_usage":{"total_tokens":1909,"prompt_tokens":967,"completion_tokens":942,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":884}},"tokens_in":583,"tokens_out":942,"duration_ms":7541,"temperature":1.0,"reasoning_tokens":884,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:34:09.954093+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Example 5.2 with $L=C N^2$ paths and measure $\\varepsilon_N = \\sup_i \\|J'(u^{i,N}) - J'_N(u^{i,N})\\|$ directly; if the measured quantity decays slower than $\\Delta t$, say like $\\Delta t^{1/2}$, then the first-order claim for the LSMC scheme fails because the deferred regression-error bound in Remark 4.8 is not satisfied. Equivalently, compare the convergence slopes with $L=N^2$ and $L=N^3$: if the slope drops below one when $L=N^2$, the missing regression term is the bottleneck.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the gradient projection framework, the projection lemma, and the grid-method error estimate that the discrete analysis extends to state constraints."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the regression-based Monte Carlo basis functions (Voronoi partitions and hypercubes) used to approximate conditional expectations in the fully discrete scheme."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the empirical regression error bound for BSDEs to which Remark 4.8 defers for the LSMC simulation error."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Lemma 4.4, the temporal weak error expansions for the discretized BSDE used to estimate the time discretization part of $\\varepsilon_N$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Lemma 4.5, the projection error estimate for least-squares regression of the BSDE solution used in the error decomposition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides Robinson's regularity condition and the Lagrange multiplier theorem used to derive the optimality condition system."}],"review_version":1}