Pith. sign in

REVIEW 1 major objections 4 minor 47 references

Statistical guarantees for continuous-time policy evaluation: blessing of ellipticity and new tradeoffs

T0 review · 1 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that, for elliptic diffusions, the LSTD estimator of the value function from one discretely observed trajectory attains O(1/√T) error in the Sobolev norm as long as trajectory length grows nearly linearly with mixing…

desk verdict Genuinely new finite-sample analysis of continuous-time LSTD, but the advertised linear-in-m trajectory length is conditional on an unproven regularity estimate. read the letter →

arxiv 2502.04297 v1 pith:R4Q5IU6O submitted 2025-02-06 cs.LG math.OCmath.PRmath.STstat.TH

Wenlong Mou This is my paper · ORCID
classification cs.LGmath.OCmath.PRmath.STstat.TH MSC 62M0560J6062G05
keywords continuous-timereinforcementlearningpolicyevaluationleast-squarestemporaldifferencediffusionprocessnon-asymptoticguaranteesSobolevnormellipticityMarkovchaincovariance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes non-asymptotic guarantees for estimating the value function of a continuous-time Markov diffusion from a single, discretely observed ergodic trajectory using least-squares temporal difference (LSTD). The main claim is that the estimator's error in the first-order Sobolev norm decreases at the rate O(1/√T) with trajectory length T, and that this rate is achieved once T is only mildly larger than the mixing time and the number m of basis functions. The mechanism is the ellipticity of the diffusion: it keeps the approximation factors bounded even as the effective horizon of the discretized Bellman equation diverges, and it lets the Markovian part of the statistical error be controlled by the approximation error itself. The result also uncovers a non-standard tradeoff: the Markovian statistical error shrinks with the approximation error while the martingale error grows sublinearly in m.

What carries the argument

The load-bearing object is the cross-covariance structure μ_k(f,g) of the Markov-chain functionals that appear in the martingale and Markovian components of the estimator's noise. The key analytic step re-distributes the generator and gradient acting on the error function g onto the basis f, bounding the Markovian covariance by ‖f‖²_H1 times terms involving Creg(T0)‖g‖²_{$W^{{1,2p}}$} log(1/η) plus exponentially damped and η-scaled second-order terms. This redistribution, together with ellipticity-based conditioning of the population matrix A and matrix concentration bounds, is what lets the error bound separate cleanly into approximation, Markovian, and martingale parts.

What would settle it

Find one elliptic diffusion satisfying the paper's assumptions for which the constant creg(T0) in Lemma 2 provably grows faster than √T0; then the linear-in-m threshold in Theorem 1 fails for that process. Alternatively, simulate LSTD on a torus diffusion with an order-k Hölder value function and check whether the H1 error obeys the T ≍ m regime of Corollary 1 or requires T ≳ $m^{{1+2ω}}$.

Watch

Extended reading notes

Core claim

The central discovery is a sharp, problem-dependent error decomposition for the empirical projected LSTD estimator applied to the diffusion. Under uniform ellipticity, smoothness, a Poincaré inequality, hypercontractivity of the basis, and boundedness of whitened features, the expected squared H1 error is bounded by the best approximation error ‖Δ*‖²_H1 plus a Markovian statistical term growing like (m/T) with a prefactor controlled by the approximation error, plus a martingale term (τ⁴/T)Tr($H1^{{-1}}$H0)(η + ‖∇f⋆‖²), plus the numerical error $η^{{2ν}}$, provided T exceeds a threshold. The decisive structural finding is that the Markovian noise can be written so that gradient operators fall on the basis rather than on the value-function error, which is why the leading statistical term is proportional to the approximation error itself and why the martingale term need not grow like m/T. The clean O(1/√T) rate with nearly linear T in m relies on an unproven regularity bound creg(T0) = O(√T0); when that bound is not assumed, the trajectory threshold degrades to $m^{{1+2ω}}$.

Load-bearing premise

The headline near-linear trajectory requirement rests on an unproven bound on how the high-order moments of the diffusion's log-density grow with time; the paper leaves creg(T0) = O(√T0) as an open question, and without it the trajectory threshold becomes $m^{{1+2ω}}$ instead of linear.

Editorial extensions

If this is right

  • If the result is correct, the O(1/√T) rate in H1 norm holds even though the effective horizon diverges as the discretization step η shrinks, because ellipticity replaces horizon-dependent constants.
  • The Markovian component of the statistical error is controlled by the approximation error, so improving the function class reduces both the bias and the leading part of the variance.
  • The martingale component grows only as Tr(H1^{-1}H0), which is O(1), O(log m), or O(m^{1-2/d}) for Fourier bases on a torus, so the price of more basis functions is weaker than the classical m/T price.
  • In the torus/Fourier example the resulting convergence rates beat the standard nonparametric minimax rates for estimating the gradient of a Hölder function from n independent observations.
  • For a well-specified model (Δ* = 0), the error reduces to the martingale term plus numerical error, so the Markovian part vanishes with the approximation error.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: If the open bound creg(T0) = O(√T0) holds for a class of diffusions, the linear-in-m trajectory threshold is generic; checking it for specific processes with explicit eigenfunctions would turn the paper's special case into a general theorem.
  • Editorial extension: The sublinear martingale term suggests that over-parameterized value-function classes are less harmful here than in discrete-time TD methods, so width selection in practice could be driven more by approximation error than by variance.
  • Editorial extension: The covariance-redistribution lemma is stated for LSTD but appears to transfer to Q-learning and actor-critic updates whenever the noise is driven by elliptic diffusion, since it only concerns functionals of the Markov chain.
  • Editorial extension: A direct empirical check would measure the H1 error of LSTD on a torus diffusion with a known smooth value function as m and T grow; the predicted scalings m^{-(2k-2)/d} plus g_d(m)/T are distinguishable from the classical m/T variance law.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 4 minor

Summary. The paper gives a non-asymptotic analysis of a least-squares temporal-difference (LSTD) estimator for infinite-horizon discounted value functions of ergodic diffusion processes, using a single discretely observed stationary trajectory. The estimator is a sample version of the projected fixed-point equation associated with a ν-th-order time discretization of the Bellman equation. The main result, Theorem 1, bounds the expected squared H1 error on a high-probability event by four terms: the best H1 approximation error, a Markovian statistical term that scales like m/T and has a prefactor that decays with the approximation error, a martingale statistical term scaling like Tr(H1^{-1}H0)/T, and a numerical error of order η^{2ν}. The bound is valid once the trajectory length T exceeds a threshold Tthres(m,T0) that depends on the mixing time, the feature dimension m, and a semigroup regularity constant Creg(T0). The paper also discusses well-specified and span-extended special cases, a Fourier-basis example leading to Corollary 1, and the technical mechanism by which the Markovian covariance is controlled by the approximation error.

Significance. If the advertised favorable regime were established, this would be a meaningful contribution: it would give the first non-asymptotic single-trajectory guarantee for continuous-time LSTD with an effective-horizon-independent rate, and it would identify a genuinely non-standard tradeoff in which the dominant Markovian error decays with the approximation error while the martingale error grows sublinearly in m. The proof is substantial and mostly modular, with explicit constants and careful use of Malliavin-calculus estimates, Poincaré inequalities, and matrix concentration. A notable strength is that Theorem 1 is stated honestly with the unresolved regularity constant Creg(T0) made explicit, rather than hidden. However, the paper's headline claim that only a nearly linear trajectory length in m is needed is conditional on the unproven estimate creg(T0) ≲ sqrt(T0), which the paper itself leaves open. Because that estimate is load-bearing for the abstract's central claim, the current version overstates what has been established; the result is best described as a conditional upper bound with a clearly identified open regularity problem.

major comments (1)
  1. [§3.2.1 and Theorem 1, Eq. (16a)–(16b)] The abstract and introduction state that the O(1/sqrt(T)) rate is achieved as long as T scales nearly linearly with the mixing time and the number of basis functions m. In the theorem, the trajectory-length threshold contains Tthres(m,T0), which includes a factor Creg(T0). The favorable regime in Eq. (16b), namely T = O((m/ρ*) log^4(m/(δη))) when Dm ≲ sqrt(m), is obtained under the supposition Creg(T0) ≲ T0, which follows from creg(T0) ≲ sqrt(T0). Section 3.2.1 explicitly states: 'It remains an open question to derive an upper bound on creg(T0) with explicit dependence on T0.' If creg(T0) grows faster than sqrt(T0), only the worst-case threshold in Eq. (17b), scaling like m^{1+2ω}, is available. Thus the advertised near-linear trajectory-length requirement is a conditional statement resting on an unresolved analytic estimate, not an established consequence of Theorem 1. The manuscript should either prove the needed regularity estimate or clearly and consistently frame the main contribution as conditional, with the worst-case m^{1+2ω} threshold as the unconditional guarantee. This is load-bearing because the abstract's main quantitative claim depends on it.
minor comments (4)
  1. [§5] The discussion calls the O(1/T) convergence rate 'optimal' and 'sharp', but no matching lower bound is proved anywhere in the paper; Section 5 itself lists information-theoretic optimality as an open direction. The wording should be softened to 'rate-optimal among the bounds established here' or similar.
  2. [§3.3, d = 2 case] In the displayed rate for d = 2, the exponent on (log^{3/2}T/T) appears to have the wrong sign: substituting m ≈ (T/log^{3/2}T)^{1/2} into m^{-(k-1)} gives (log^{3/2}T/T)^{(k-1)/2}, not a negative exponent. Please check and correct this expression.
  3. [§3.3] The sentence 'This is because the trajectory length requirement T /greaterorsimilar D_m^2 ensures...' contains a typesetting artifact ('/greaterorsimilar'); it should read 'T ≳ D_m^2 ensures...'.
  4. [§3.1, footnote 1] The footnote ends with 'the hyper-contractivity assumption can be relaxed to a small subset of K 1'; the trailing '1' appears to be a typo for a period or a missing clause.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the LSTD guarantee is derived from lemmas proved in the paper, and the load-bearing self-citations are prior independent results with stated assumptions.

full rationale

The derivation chain is self-contained: Theorem 1 is proved from Lemmas 1, 7, 8, and 9, which are derived in Section 4 using integration by parts, Poincaré and hypercontractivity inequalities, an external Malliavin-calculus estimate (Proposition 2, cited to Menozzi–Pesce–Zhang), and external matrix concentration inequalities (Tropp, NSW24). The approximation-factor bound (9) is imported from the author's prior work [MZ24], but that is a separate population-level theorem with stated assumptions that do not include the statistical target, so it is independent evidence rather than a circular premise. Likewise, the asymptotic covariance decomposition in Section 3.4 is attributed to [MPWB24] only as motivation, and the paper re-proves the needed non-asymptotic covariance bound as Lemma 1. No parameter in the theorem is fitted to data and then renamed a prediction, and no quantity is defined in terms of the outcome it is used to predict. The advertised 'nearly linear in m' trajectory requirement is conditional on the regularity estimate Creg(T0) ≲ T0, which the paper explicitly flags in Section 3.2.1 as an open question ('It remains an open question to derive an upper bound on creg(T0) with explicit dependence on T0'); that is an unresolved analytic assumption and a correctness risk, not a circular reduction. The use of 'optimal' in Section 5 without a matching lower bound is also a correctness concern, not circularity. Accordingly, no circular step is present.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. The central claim rests on a chain of standard assumptions (ellipticity, Poincaré, hypercontractivity, boundedness) plus one unproven regularity estimate that is not stated as an assumption but as an open question, yet the headline result depends on it. The free parameters T0 and p are proof artifacts, not data-fitted values.

free parameters (2)
  • T0 = arbitrary, e.g., T0=3 or T0 ≈ ρ^{-1} log(m/η)
    T0 is a free time horizon used in the threshold and the bound. It is not fitted to data but is chosen by the analyst to balance exponential mixing decay against the regularity constant Creg(T0). The theorem's constants depend on T0, and the clean rates require a specific scaling choice.
  • p = arbitrary p > 1
    p is a norm parameter in the Sobolev and hypercontractivity statements. The theorem holds for any p > 1, but the pre-factors and the condition q = max(2p, 4p/(4p-1), 8) depend on it. It is a proof parameter, not fitted to data.
assumptions (5)
  • domain assumption Poincaré inequality (PI(ρ*)) for the stationary distribution ξ
    Standard ergodicity assumption used to control mixing of the diffusion in Lemma 6 and the trajectory threshold. It is not proven and is assumed throughout Theorem 1.
  • domain assumption Hypercontractivity condition (Hyper(q,τ_q)) on the function subspace K
    Assumed to control higher moments of basis functions, used heavily in the concentration bounds. It is common in statistical learning but is an extra assumption on the feature class.
  • domain assumption Boundedness condition (Bou(Dm)) on feature vectors
    Assumes sup ||H1^{-1/2} ψ(x)||_2 times a gradient term is bounded by Dm^2. Needed for Bernstein-type concentrations; the clean linear threshold requires Dm ≲ √m, which is not guaranteed by the assumption itself.
  • domain assumption Smoothness and uniform ellipticity assumptions (Lip(ν)), (SL(Lξ)), (UE(λmin, λmax))
    Standard regularity conditions on drift, diffusion matrix, log-density, and ellipticity. They are used throughout the proofs, including the semigroup gradient estimates from Proposition 2.
  • ad hoc to paper Well-controlled semigroup regularity: creg(T0) ≲ √T0
    This is the key unproven assumption needed for the clean linear trajectory scaling and the O(m/T) rate in the abstract. The paper explicitly says it remains an open question. Without it, the threshold becomes m^{1+2ω} and the Markovian error term depends on second-order Sobolev norms of the approximation error.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Statistical guarantees for continuous-time policy evaluation: blessing of ellipticity and new tradeoffs." pith.science (2026). https://pith.science/paper/R4Q5IU6O

@misc{pith2026250204297,
  author       = {Pith},
  title        = {Pith review of: Statistical guarantees for continuous-time policy evaluation: blessing of ellipticity and new tradeoffs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R4Q5IU6O}},
  note         = {Machine review of arXiv:2502.04297}
}
abstract

We study the estimation of the value function for continuous-time Markov diffusion processes using a single, discretely observed ergodic trajectory. Our work provides non-asymptotic statistical guarantees for the least-squares temporal-difference (LSTD) method, with performance measured in the first-order Sobolev norm. Specifically, the estimator attains an $O(1 / \sqrt{T})$ convergence rate when using a trajectory of length $T$; notably, this rate is achieved as long as $T$ scales nearly linearly with both the mixing time of the diffusion and the number of basis functions employed. A key insight of our approach is that the ellipticity inherent in the diffusion process ensures robust performance even as the effective horizon diverges to infinity. Moreover, we demonstrate that the Markovian component of the statistical error can be controlled by the approximation error, while the martingale component grows at a slower rate relative to the number of basis functions. By carefully balancing these two sources of error, our analysis reveals novel trade-offs between approximation and statistical errors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 37 canonical work pages

  1. [1]

    Adamczak

    R. Adamczak. A tail inequality for suprema of unbounded empirical processes with applications to markov chains. Electronic Journal of Probability , 13:1000--1034, 2008

    work page 2008

  2. [2]

    S. J. Bradtke and A. G. Barto. Linear least-squares algorithms for temporal difference learning. Machine Learning , 22(1-3):33--57, 1996

    work page 1996

  3. [3]

    Bakry, F

    D. Bakry, F. Barthe, P. Cattiaux, and A. Guillin. A simple proof of the P oincar \'e inequality for a large class of probability measures. Electronic Communications in Probability , 13:60--66, 2008

    work page 2008

  4. [4]

    Bakry and M

    D. Bakry and M. \'E mery. Diffusions hypercontractives. In S \'e minaire de Probabilit \'e s XIX 1983/84: Proceedings , pages 177--206. Springer, 2006

    work page 1983

  5. [5]

    J. A. Boyan. Technical update: Least-squares temporal difference learning. Machine learning , 49(2-3):233--246, 2002

    work page 2002

  6. [6]

    K. Doya. Temporal difference learning in continuous time and space. Advances in neural information processing systems , 8, 1995

    work page 1995

  7. [7]

    Duoandikoetxea

    J. Duoandikoetxea. Fourier analysis , volume 29. American Mathematical Society, 2024

    work page 2024

  8. [8]

    Policy evaluation from a single path: Multi-step methods, mixing and mis-specification

    Yaqi Duan and Martin J Wainwright. Policy evaluation from a single path: Multi-step methods, mixing and mis-specification. arXiv preprint arXiv:2211.03899 , 2022

    arXiv 2022

Show all 47 references
  1. [9]

    M. I. Ganzburg. Multidimensional jackson theorems. Siberian Mathematical Journal , 22(2):223--231, 1981

    1981

  2. [10]

    Stochastic control for fine-tuning diffusion models: Optimality, regularity, and convergence

    Yinbin Han, Meisam Razaviyayn, and Renyuan Xu. Stochastic control for fine-tuning diffusion models: Optimality, regularity, and convergence. arXiv preprint arXiv:2412.18164 , 2024

    2024 arXiv

  3. [11]

    A. Isidori. Nonlinear control systems: an introduction . Springer, 1985

    1985

  4. [12]

    P. Jain, S. M. Kakade, R. Kidambi, P. Netrapalli, and A. Sidford. Accelerating stochastic gradient descent for least squares regression. In Conference On Learning Theory , pages 545--604. PMLR, 2018

    2018

  5. [13]

    Jia and X

    Y. Jia and X. Y. Zhou. Policy evaluation and temporal-difference learning in continuous time and space: A martingale approach. The Journal of Machine Learning Research , 23(1):6918--6972, 2022

    2022

  6. [14]

    Jia and X

    Y. Jia and X. Y. Zhou. Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms. The Journal of Machine Learning Research , 23(1):12603--12652, 2022

    2022

  7. [15]

    Jia and X

    Y. Jia and X. Y. Zhou. q -learning in continuous time. Journal of Machine Learning Research , 24(161):1--61, 2023

    2023

  8. [16]

    S. M. Kakade. A natural policy gradient. Advances in neural information processing systems , 14, 2001

    2001

  9. [17]

    Kobeissi and F

    Z. Kobeissi and F. Bach. Temporal difference learning with continuous time and state in the stochastic setting. 2023

    2023

  10. [18]

    Konda and J

    V. Konda and J. N. Tsitsiklis. Actor-critic algorithms. Advances in neural information processing systems , 12, 1999

    1999

  11. [19]

    Lecu \'e and S

    G. Lecu \'e and S. Mendelson. Learning sub G aussian classes: Upper and minimax bounds. arXiv preprint arXiv:1305.4825 , 2013

    2013 arXiv

  12. [20]

    X. Li, D. Verma, and L. Ruthotto. A neural network approach for stochastic optimal control. arXiv preprint arXiv:2209.13104 , 2022

    2022 arXiv

  13. [21]

    L. Li, M. Wang, and Y. Wang. Error estimates of the E uler's method for stochastic differential equations with multiplicative noise via relative entropy. arXiv preprint arXiv:2409.04991 , 2024

    2024 arXiv

  14. [22]

    Mendelson

    S. Mendelson. Learning without concentration. Journal of the ACM (JACM) , 62(3):1--25, 2015

    2015

  15. [23]

    W. Mou, N. Flammarion, M. J. Wainwright, and P. L. Bartlett. Improved bounds for discretization of L angevin diffusions: Near-optimal rates without convexity. Bernoulli , 28(3):1577--1601, 2022

    2022

  16. [24]

    W. Mou, A. Pananjady, and M. J. Wainwright. Optimal oracle inequalities for projected fixed-point equations, with applications to policy evaluation. Mathematics of Operations Research , 48(4):2308--2336, 2023

    2023

  17. [25]

    W. Mou, A. Pananjady, M. J. Wainwright, and P. L. Bartlett. Optimal and instance-dependent guarantees for markovian linear stochastic approximation. Mathematical Statistics and Learning , 2024

    2024

  18. [26]

    Menozzi, A

    S. Menozzi, A. Pesce, and X. Zhang. Density and gradient estimates for non-degenerate B rownian SDE s with unbounded measurable drift. Journal of Differential Equations , 272:330--369, 2021

    2021

  19. [27]

    Marinelli and M

    C. Marinelli and M. R \"o ckner. On the maximal inequalities of B urkholder, D avis and G undy. Expositiones Mathematicae , 34(1):1--26, 2016

    2016

  20. [28]

    Mou and Y

    W. Mou and Y. Zhu. On B ellman equations for continuous-time policy evaluation I : discretization and approximation. arXiv preprint arXiv:2407.05966 , 2024

    2024 arXiv

  21. [29]

    Neeman, B

    J. Neeman, B. Shi, and R. Ward. Concentration inequalities for sums of M arkov-dependent random matrices. Information and Inference: A Journal of the IMA , 13(4):iaae032, 2024

    2024

  22. [30]

    Dota 2 with large scale deep reinforcement learning

    OpenAI et al. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv: 1912.06680 , 2019

    1912 arXiv

  23. [31]

    B. Recht. A tour of reinforcement learning: The view from continuous control. Annual Review of Control, Robotics, and Autonomous Systems , 2(1):253--279, 2019

    2019

  24. [32]

    Ruthotto, S

    L. Ruthotto, S. J. Osher, W. Li, L. Nurbekyan, and S. W. Fung. A machine learning framework for solving high-dimensional mean field game and mean field control problems. Proceedings of the National Academy of Sciences , 117(17):9183--9193, 2020

    2020

  25. [33]

    R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. A Bradford Book , 2018

    2018

  26. [34]

    R. S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning , 3(1):9--44, 1988

    1988

  27. [35]

    Srikant and L

    R. Srikant and L. Ying. Finite-time error bounds for linear stochastic approximation andtd learning. In Conference on Learning Theory , pages 2803--2830. PMLR, 2019

    2019

  28. [36]

    Szepesv \'a ri

    Cs. Szepesv \'a ri. Algorithms for reinforcement learning . Springer Nature, 2022

    2022

  29. [37]

    W. Tang. Fine-tuning of diffusion models via stochastic control: entropy regularization and beyond. arXiv preprint arXiv:2403.06279 , 2024

    2024

  30. [38]

    J. A. Tropp. Freedman's inequality for matrix martingales. Electronic Communications in Probability , 16:262--270, 2011

    2011

  31. [39]

    A. B. Tsybakov. Introduction to Nonparametric Estimation . Springer Science & Business Media, 2008

    2008

  32. [40]

    J. N. Tsitsiklis and B. Van Roy. Analysis of temporal-diffference learning with function approximation. In Advances in Neural Information Processing Systems , pages 1075--1081, 1997

    1997

  33. [41]

    Uehara, Y

    M. Uehara, Y. Zhao, K. Black, E. Hajiramezanali, G. Scalia, N. L. Diamant, A. M. Tseng, T. Biancalani, and S. Levine. Fine-tuning of continuous-time diffusion models as entropy-regularized control. arXiv preprint arXiv:2402.15194 , 2024

    2024 arXiv

  34. [42]

    Uehara, Y

    M. Uehara, Y. Zhao, T. Biancalani, and S. Levine. Understanding reinforcement learning-based fine-tuning of diffusion models: A tutorial and review. arXiv preprint arXiv:2407.13734 , 2024

    2024 arXiv

  35. [43]

    F.-Y. Wang. A character of the gradient estimate for diffusion semigroups. Proceedings of the American Mathematical Society , 133(3):827--834, 2005

    2005

  36. [44]

    Wang and X

    H. Wang and X. Y. Zhou. Continuous-time mean--variance portfolio selection: A reinforcement learning framework. Mathematical Finance , 30(4):1273--1308, 2020

    2020

  37. [45]

    H. Wang, T. Zariphopoulou, and X. Y. Zhou. Reinforcement learning in continuous time and space: A stochastic control approach. The Journal of Machine Learning Research , 21(1):8145--8178, 2020

    2020

  38. [46]

    H. Zhao, H. Chen, J. Zhang, D. D. Yao, and W. Tang. Scores as actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning. arXiv preprint arXiv:2409.08400 , 2024

    2024 arXiv

  39. [47]

    M. Zhou, J. Han, and J. Lu. Actor-critic method for high dimensional static H amilton-- J acobi-- B ellman partial differential equations based on neural networks. SIAM Journal on Scientific Computing , 43(6):A4043--A4066, 2021

    2021

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.