{"id":"9611a7d8-2fd3-4062-8853-8cd239e3365b","arxiv_id":"2505.01041","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Single-sample single-timescale actor-critic provably finds the global optimum of linear quadratic regulation on continuous state-action space with O(ε^-2) sample complexity.","lead":"This paper proves that the classic single-sample, single-timescale actor-critic algorithm converges to the globally optimal linear feedback law for the linear quadratic regulator (LQR) control problem, using O(ε^-2) samples. It is the first global guarantee for this algorithm on continuous state-action spaces, though it rests on a stability assumption the paper does not fully enforce.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Assumption 1 is not enforced for the unprojected actor; the proof's one-step perturbation bound (34) does not close the induction, so Theorem 1's global guarantee is not established for Algorithm 1 as stated.","rationale":"I read the manuscript and supplement. The error recursions (Theorems 2-4), the AM-GM and Abel-summation steps, and the interconnected-system contraction condition h2h4^2+h2h4^2h7^2+2h5h7^2<1 are internally consistent, and the gradient-domination step legitimately converts CT into the actor performance error. The reader's algebra check is sound. The load-bearing concern is the unclosed induction for Assumption 1: the proof assumes every K_t satisfies the uniform spectral-radius and norm bounds, but Algorithm 1 never projects the actor, and the one-step perturbation bound (34) does not prevent crossing the boundary of K. The paper's own remark after Assumption 1 acknowledges this by deferring a projection and omitting the high-probability argument. This is not a disagreement with current consensus; it is an internal gap between the assumptions and the algorithm. The stationary-sampling oracle is a second idealization, but it is secondary to the stability question. The concern is repairable by adding a projection of K_t onto K with the same step-size analysis, or by supplying the high-probability confinement argument, so I do not recommend REJECT. The verdict remains CONDITIONAL; the abstract's unconditional O(ε^-2) sample-complexity statement should be qualified accordingly.","tokens_in":35170,"tokens_out":16188,"duration_ms":172326,"concrete_test":"Compute the invariant-set check on a scalar LQR with A=1, B=1, Q=R=D0=1, so ρ(A-BK)=|1-K|. Set λ=0.999 and K0=0.001, placing the initial iterate on the Assumption-1 boundary. Using the paper's constants, evaluate the right-hand side δ of Supp eq. (34); if δ > (1-λ)/||B|| = 0.001, then a perturbation of the permitted size in the direction K0-δ produces ρ(A-BK1)>1, so the proof's own bound cannot rule out instability. If δ≤0.001, simulate 10^4 independent runs of Algorithm 1 with exact stationary sampling and T large enough to satisfy eq. (33), recording the first exit time from {ρ(A-BK)≤λ}; any exit confirms the missing invariant-set induction. In either case, the test determines whether Theorem 1 requires an additional actor projection or a high-probability confinement argument.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is Theorem 1: Algorithm 1 with α_t=c/√T, β_t=γ_t=1/√T achieves min_{0≤t<T} E[J(K_t)-J(K*)] = O(1/√T). The proof is built on Assumption 1, which bounds every K in the policy class by K̄ and requires ρ(A-BK)≤λ<1. This assumption supplies the uniform bounds on D_K, P_K, the critic matrix A_K, the Lipschitz constants l1,l2 (Lemmas 5-11), and the perturbation bound in eq. (34) of the supplement. But Algorithm 1 projects only η_t and ω_t (Lines 7-8); the actor update on Line 9 is unprojected. The proof shows only that ||K_{t+1}-K_t|| is small (Supp eq. (34)), which is a bound on the size of the perturbation, not on its direction. Since the set {K: ρ(A-BK)≤λ} has a boundary and K_t is allowed to be arbitrarily close to it (e.g., K0 with ρ(A-BK0)=λ), a one-step perturbation of the permitted size can push ρ(A-BK_{t+1}) above λ or above 1. No induction argument establishes K_t∈K for all t; the authors explicitly defer a projection map and omit the high-probability argument that would handle bounded trajectories (remark after Assumption 1). Consequently every Lyapunov estimate and the gradient-domination conclusion (Lemma 13) is conditional on a property the algorithm does not enforce and the proof does not close. This is the load-bearing gap: if K_t exits the stable set, J(K_t) is infinite, the TD target and natural gradient are undefined, and the remaining error recursions have no meaning.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the classic single-sample, single-timescale actor-critic algorithm applied to the infinite-horizon time-average linear quadratic regulator (LQR) with continuous state and action spaces. The actor uses a natural-policy-gradient update based on a critic parameter vector updated by TD(0) with a single sample per iteration, together with a running cost estimator. The main claim (Theorem 1) is that, under Assumption 1 and with step sizes α_t = c/√T and β_t = γ_t = 1/√T, the average squared cost-estimation error, the average squared critic error, and the minimum over t of E[J(K_t)-J(K*)] are all O(1/√T), yielding O(ε^{-2}) sample complexity. The proof derives three coupled error recursions (Theorems 2-4 in the supplement), solves the resulting interconnected inequality system (Theorem 5), and uses the gradient-domination property of LQR to convert natural-gradient convergence into global optimality. The paper frames this as the first global optimality guarantee for single-sample single-timescale actor-critic on unbounded continuous state-action space.","tokens_in":35327,"tokens_out":3323,"duration_ms":36566,"significance":"If the result is established, it is a meaningful step beyond prior work: earlier single-sample single-timescale actor-critic analyses are local and restricted to finite state or action spaces, while the present paper targets global convergence for the LQR benchmark with the same O(ε^{-2}) sample complexity as those local results. The proof machinery is also noteworthy: rather than decoupling actor and critic, it analyzes their estimation errors as an interconnected system and identifies an explicit threshold on the actor/critic step-size ratio that guarantees contraction (condition h2 h4^2 + h2 h4^2 h7^2 + 2 h5 h7^2 < 1). The paper includes reproducible-looking numerical experiments and gives a transparent comparison with zeroth-order and double-loop baselines. However, the current manuscript leaves a load-bearing assumption unenforced: Assumption 1 is imposed on every iterate K_t, but Algorithm 1 does not project the actor and the proof does not close the induction. For this reason the central claim, as stated for Algorithm 1, is not yet established.","major_comments":[{"comment":"Assumption 1 requires every policy K in the class to satisfy ||K|| ≤ K̄ and ρ(A-BK) ≤ λ < 1, and the proof of Theorem 1 uses this uniform stability for all t to bound D_{K_t}, P_{K_t}, A_{K_t}, the Lipschitz constants l1, l2, and the perturbation bound (Supp. eq. (34)). However, Algorithm 1 projects only η_t and ω_t (Lines 7-8); the actor update on Line 9 is unprojected. The proof shows only that ||K_{t+1}-K_t|| is small (Supp. eq. (34)), which bounds the size of the perturbation but not its direction. Since K_t may be arbitrarily close to the boundary ρ(A-BK)=λ (or even to ρ(A-BK)=1), a one-step perturbation of the size allowed by (34) can move the iterate outside the stabilizing set, where J(K_t), D_{K_t}, P_{K_t}, and the TD recursion are undefined. No induction argument establishes K_t ∈ K for all t; the remark after Assumption 1 explicitly defers the projection map and omits the high-probability argument that would handle bounded trajectories. Thus every Lyapunov-type estimate in Lemmas 5-11 and the gradient-domination conclusion in Lemma 13 are conditional on a property that Algorithm 1 does not enforce and that the proof does not close. This is the central gap: Theorem 1 is not established for Algorithm 1 as stated.","section":"Assumption 1 and Algorithm 1, Theorem 1"},{"comment":"Line 3 requires that at each iteration the algorithm sample x_t from the exact stationary distribution ρ_{K_t} of the current policy. This is a strong oracle assumption: in an online, single-trajectory implementation the state distribution is not stationary, and the proof's martingale and variance bounds rely on the sampled pair (x_t,u_t) being drawn from the stationary distribution of K_t. The text cites geometric β-mixing to argue that a sufficiently long burn-in approximates stationarity, but no finite-time bias analysis is provided. Consequently, the advertised O(ε^{-2}) sample complexity is for an idealized sampling oracle, not for the single-trajectory algorithm described in the introduction. The authors should either state this idealization explicitly as a limitation or extend the analysis to the non-stationary (or transient) setting.","section":"Algorithm 1, Line 3 and Section 3"},{"comment":"The theorem states min_{0≤t<T} E[J(K_t)-J(K*)] = O(1/√T), and the proof obtains this by bounding the average of E[J(K_t)-J(K*)] via the average of E||E_{K_t}||^2 and then using min ≤ average. This step is valid. However, in Supp. A.5 the text says 'From the convergence of C_T, we know that 1/T Σ E||E_{K_t}|| = O(1/√T)', which should read E||E_{K_t}||^2. The correct statement is used implicitly in the displayed inequality, so this is a typographical error rather than a mathematical flaw, but it should be corrected for clarity.","section":"Theorem 1 statement vs. proof, Supp. A.5"}],"minor_comments":[{"comment":"Theorems 2, 3, and 4 each begin 'Suppose that Assumptions 1 and 1 hold'; the second '1' appears to be a typo for a differently numbered assumption or intended to be repeated Assumption 1.","section":"Supp. A.1, A.2, A.3"},{"comment":"In the statement of Theorem 4, the constant in the first term is written as (U + 2 c_4 c_α^2)/(2 σ_min(D0) c), while the proof uses c_α and c interchangeably. Since α_t = c/√T was introduced, the notation should be unified to avoid confusion about which constant is being optimized.","section":"Theorem 4 and Supp. eq. (43)"},{"comment":"In the proof sketch, the line 'we project η into a ball of radius U and project ω into a ball of radius ω̄' is helpful, but the text does not mention that the actor is not projected; given that this is the source of the main gap, the sketch should flag this explicitly.","section":"Section 4.1, proof sketch"},{"comment":"The abstract says 'we show that the popular single-timescale actor-critic can attain an epsilon-optimal solution with ... sample complexity for solving LQR'; given the sampling oracle and the Assumption 1 issue, this statement should be qualified as holding under the stated idealized assumptions.","section":"Abstract and Section 1"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea—treating the three estimation errors as an interconnected system and deriving a contraction condition on the step-size ratio—is strong and, conditionally on Assumption 1 being enforced, the algebra appears correct. The main obstacle is that Assumption 1 is not enforced by the algorithm and is explicitly deferred to future work in the remark after the assumption. This is fixable in principle by adding a projection of the actor onto a compact stable set or by supplying the high-probability argument that keeps iterates in the stable region, but until then the headline theorem does not cover Algorithm 1 as stated. The stationary-distribution oracle is a second idealization that should be acknowledged or removed. I would encourage the editor to seek a revision that either adds the actor projection and proves the analogous theorem, or clearly states that the guarantee is for the idealized stationary-sampling model with an invariant stability assumption."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper makes a genuine claim: a finite-time global optimality rate for the classic single-sample single-timescale actor-critic on an infinite state-action problem, LQR. The central algebra in the supplement is substantially correct, and I agree with the reader's stress test: the theorem is not actually established for Algorithm 1 as written. Assumption 1 is asserted for every iterate, but the actor update never projects K_t, and the proof's one-step perturbation bound (Supp (34)) does not prove that the trajectory stays inside the stable set. If K_t exits, J, the critic target, and the natural gradient are all undefined. That is a load-bearing gap, not a cosmetic one.\n\nWhere the paper earns credit: the interconnected-system analysis of cost estimator, critic, and natural-gradient errors is a real technical step forward; the contraction condition on the step-size ratio is explicit; and the gradient-domination step legitimately converts natural-gradient convergence into J(K_t) - J(K*) = O(1/sqrt(T)). I checked the AM-GM choices, the Abel summations, and the final system solve, and they hold together. The authors are also transparent: they acknowledge that a projection map would be needed and that high-probability arguments are omitted.\n\nThe smaller issues are appropriately smaller. The stationary-sampling oracle is strong; the O(epsilon^-2) statement hides problem-dependent constants, including the step-size threshold; experiments ship no code; and Table 1 misclassifies [Chen et al. 2021], a bilevel-optimization paper, as single-timescale AC. There are numerous typos in the appendix. None of these alone would change my verdict.\n\nWho should read it: people working on finite-time actor-critic theory and on LQR. The paper deserves a serious referee even though I would not currently cite Theorem 1 as a proven global guarantee. The right path is to send it to peer review with the expectation of heavy revision: the authors need to either add a projection of K_t onto a stable sublevel set, or prove a high-probability bound that the trajectory stays stable, or state the theorem conditionally on that event. The technique is promising; the claim as written overreaches.","headline":"First serious finite-time global-optimality claim for single-sample single-timescale actor-critic on LQR, with largely sound algebra, but the theorem overreaches because Assumption 1 is not enforced on the unprojected actor.","tokens_in":36127,"tokens_out":2520,"would_cite":false,"duration_ms":26412,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper establishes that the classic single-sample single-timescale actor-critic reaches the global optimum on LQR, the first such guarantee on continuous state-action space.","keywords":["actor-critic","single-timescale","linear quadratic regulator","global optimality","sample complexity","continuous state-action space","natural policy gradient","finite-time convergence"],"falsifier":"Run Algorithm 1 on the two-dimensional LQR example while recording $\\rho(A-BK_t)$ at every iteration: if any iterate violates $\\rho(A-BK_t)\\le\\lambda<1$ or leaves the assumed norm bound, the theorem's premise fails; alternatively, set the actor/critic stepsize ratio above the threshold in (51) and check whether the three average errors stop decaying at $O(1/\\sqrt{T})$.","tokens_in":34716,"feed_emoji":"🎯","tokens_out":16136,"duration_ms":139680,"temperature":0.7,"pith_summary":"The paper sets out to prove that the classic single-sample single-timescale actor-critic—one sample per iteration, actor and critic stepsizes kept in a constant ratio—can find a globally optimal policy on a continuous, unbounded state-action space. Using the linear quadratic regulator as the testbed, it shows that with step sizes $\\alpha_t = c/\\sqrt{T}$ and $\\beta_t = \\gamma_t = 1/\\sqrt{T}$, the average squared cost-estimator error, the average squared critic error, and the best actor suboptimality gap all shrink as $O(1/\\sqrt{T})$. Interpreting the average gap as an $\\epsilon$-accuracy criterion, this is an $O(\\epsilon^{-2})$ sample-complexity guarantee, the same order previously obtained only for local convergence or finite state-action spaces. The proof's core move is to stop analyzing actor and critic separately and instead solve a coupled system of inequalities for the three estimation errors.","feed_headline":"Single-sample actor-critic provably finds the global LQR optimum","feed_subtitle":"The most classic online actor-critic variant gets a global, not just local, guarantee on continuous state-action LQR.","key_machinery":"The central object is the three-variable interconnected error system $(A_T,B_T,C_T)$, where $A_T$ is the average squared cost-estimator error, $B_T$ is the average squared critic error, and $C_T$ is the average squared natural-gradient norm. The paper derives implicit bounds of the form $A_T\\le O(1/\\sqrt{T})+h_2B_T+h_2C_T$, $B_T\\le O(1/\\sqrt{T})+h_4\\sqrt{A_TB_T}+h_5C_T$, and $C_T\\le O(1/\\sqrt{T})+h_7\\sqrt{B_TC_T}$, then shows these inequalities force all three averages to $O(1/\\sqrt{T})$ when the actor/critic stepsize ratio is below an explicit threshold. Carrying the argument are two LQR-specific identities: the natural gradient $\\nabla^N_K J(K)=E_K=(R+B^\\top P_K B)K-B^\\top P_K A$, and the coercive, almost-smooth, gradient-dominated landscape of $J(K)$, capped by $J(K)-J(K^*)\\le \\frac{1}{\\sigma_{\\min}(R)}\\|D_{K^*}\\|\\operatorname{Tr}(E_K^\\top E_K)$.","core_discovery":"Stated on the paper's own terms, the discovery is that under Assumption 1 (every iterate $K_t$ stays norm-bounded with $\\rho(A-BK_t)\\le\\lambda<1$), Algorithm 1 achieves $\\min_{0\\le t<T}\\mathbb{E}[J(K_t)-J(K^*)] = O(1/\\sqrt{T})$, together with $\\frac{1}{T}\\sum_{t=0}^{T-1}\\mathbb{E}(\\eta_t-J(K_t))^2=O(1/\\sqrt{T})$ and $\\frac{1}{T}\\sum_{t=0}^{T-1}\\mathbb{E}\\|\\omega_t-\\omega^*_{K_t}\\|^2=O(1/\\sqrt{T})$. The paper reads this as the first global optimality proof for the single-sample single-timescale actor-critic on an infinite continuous state-action space, with sample complexity $O(\\epsilon^{-2})$ for an $\\epsilon$-optimal policy. The reason such a guarantee is not immediate is that the critic and cost estimates are noisy and coupled to the actor update; the paper's claim is that the coupling can be bounded as one interconnected iteration system rather than decoupled.","pith_inferences":["Not in the paper: if Assumption 1 were replaced by a projection onto the uniformly stable set $\\{K:\\rho(A-BK)\\le\\lambda\\}$, the proof might become self-contained; the paper defers exactly such a projection, so testing it would show whether the global claim can survive without the unprojected stable-trajectory premise.","Not in the paper: the same coupled-inequality scheme may transfer to other control problems whose cost is coercive, almost smooth, and gradient-dominated; checking whether the three inequalities close on an output-feedback or nonlinear benchmark would locate the boundary of the mechanism.","Not in the paper: the stationary-distribution sampling oracle (Line 3) could be relaxed by drawing each state after a short warm-up, since LQR trajectories mix geometrically; quantifying the extra burn-in cost would turn the algorithm into one implementable without an oracle."],"forward_implications":["For LQR with unbounded continuous state and action spaces, the single-sample single-timescale actor-critic is globally convergent, not merely locally convergent, at the same $O(\\epsilon^{-2})$ sample complexity previously attached to local or finite-space results.","The actor, critic, and cost estimator can all run online with one sample per step and proportional step sizes; no inner-loop policy evaluation or two-timescale separation is needed for the guarantee.","The step-size ratio between actor and critic must lie below a constant threshold; the paper gives this threshold in closed form, converting a heuristic tuning choice into a stated condition.","Relative to double-loop actor-critic for LQR, the paper claims an improvement in sample complexity from $O(\\epsilon^{-5})$ to $O(\\epsilon^{-2})$."],"supporting_citations":[{"why":"Supplies the LQR cost/gradient formulas and the quadratic Q-function parametrization that Algorithm 1 estimates, plus the double-loop baseline compared against.","marker":"[Yang et al., 2019]"},{"why":"Establishes the LQR landscape properties (coercivity, almost smoothness, gradient domination) used in the global convergence step.","marker":"[Fazel et al., 2018]"},{"why":"Derives the quadratic Q-function form for LQR used to justify the critic feature parametrization.","marker":"[Bradtke et al., 1994]"},{"why":"Defines the svec/smat notation and symmetric Kronecker product identities used to derive the matrix $A_K$ and its invertibility.","marker":"[Schacke, 2004]"},{"why":"Prior single-sample single-timescale actor-critic analysis with a local guarantee on infinite-state, finite-action space; the setting this paper extends.","marker":"[Chen et al., 2021]"},{"why":"Prior finite-state/action single-timescale analysis whose finite-dimensional feature concatenation cannot transfer to infinite state-action space.","marker":"[Olshevsky and Gharesifard, 2023]"},{"why":"Prior local finite-time analysis of single-timescale actor-critic whose Lipschitz constants depend on the finite size of the state-action space.","marker":"[Chen and Zhao, 2022]"},{"why":"Multi-sample single-timescale actor-critic for LQR whose LSTD critic decouples the analysis, the contrast that makes a single-sample coupled proof necessary.","marker":"[Zhou and Lu, 2023]"},{"why":"Provides the coercivity argument that places the optimum inside the bounded stable policy class used by Assumption 1.","marker":"[Duan et al., 2023]"}],"fun_headline_variants":["Global optimality proven for single-timescale actor-critic in LQR","First global guarantee for classic actor-critic on continuous LQR","Single-timescale actor-critic achieves global LQR optimum","Continuous LQR: first global proof for single-timescale actor-critic","Under stability, single-timescale actor-critic solves LQR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every controller produced by the unprojected actor update stays inside a fixed bounded set of stabilizing controllers; the proof assumes this uniform stability rather than enforcing it, and a single excursion outside the set would undo the Lyapunov estimates the convergence argument relies on.","fun_headline_variants_meta":{"raw":{"variants":["Global optimality proven for single-timescale actor-critic in LQR","First global guarantee for classic actor-critic on continuous LQR","Single-timescale actor-critic achieves global LQR optimum","Continuous LQR: first global proof for single-timescale actor-critic","Under stability, single-timescale actor-critic solves LQR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001091,"raw_usage":{"total_tokens":4561,"prompt_tokens":952,"completion_tokens":3609,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":3520}},"tokens_in":568,"tokens_out":3609,"duration_ms":23442,"temperature":1.0,"reasoning_tokens":3520,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:32:00.222260+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Algorithm 1 on the two-dimensional LQR example while recording $\\rho(A-BK_t)$ at every iteration: if any iterate violates $\\rho(A-BK_t)\\le\\lambda<1$ or leaves the assumed norm bound, the theorem's premise fails; alternatively, set the actor/critic stepsize ratio above the threshold in (51) and check whether the three average errors stop decaying at $O(1/\\sqrt{T})$.","supporting_citations":[{"cited_title":"Provably global convergence of actor-critic: A case for linear quadratic regulator with ergodic cost","cited_arxiv_id":null,"evidence_quote":"Supplies the LQR cost/gradient formulas and the quadratic Q-function parametrization that Algorithm 1 estimates, plus the double-loop baseline compared against."},{"cited_title":"Global convergence of policy gradient methods for the linear quadratic regulator","cited_arxiv_id":null,"evidence_quote":"Establishes the LQR landscape properties (coercivity, almost smoothness, gradient domination) used in the global convergence step."},{"cited_title":"Adaptive linear quadratic control using policy iteration","cited_arxiv_id":null,"evidence_quote":"Derives the quadratic Q-function form for LQR used to justify the critic feature parametrization."},{"cited_title":"On the kronecker product","cited_arxiv_id":null,"evidence_quote":"Defines the svec/smat notation and symmetric Kronecker product identities used to derive the matrix $A_K$ and its invertibility."},{"cited_title":"Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems","cited_arxiv_id":null,"evidence_quote":"Prior single-sample single-timescale actor-critic analysis with a local guarantee on infinite-state, finite-action space; the setting this paper extends."},{"cited_title":"A small gain analysis of single timescale actor critic","cited_arxiv_id":null,"evidence_quote":"Prior finite-state/action single-timescale analysis whose finite-dimensional feature concatenation cannot transfer to infinite state-action space."},{"cited_title":"Finite-time analysis of single-timescale actor-critic","cited_arxiv_id":"2210.09921","evidence_quote":"Prior local finite-time analysis of single-timescale actor-critic whose Lipschitz constants depend on the finite size of the state-action space."},{"cited_title":"Single timescale actor-critic method to solve the linear quadratic regulator with convergence guarantees","cited_arxiv_id":null,"evidence_quote":"Multi-sample single-timescale actor-critic for LQR whose LSTD critic decouples the analysis, the contrast that makes a single-sample coupled proof necessary."},{"cited_title":"Optimization landscape of policy gradient methods for discrete-time static output feedback","cited_arxiv_id":null,"evidence_quote":"Provides the coercivity argument that places the optimum inside the bounded stable policy class used by Assumption 1."}],"review_version":1}