REVIEW 2 major objections 5 minor 25 references
Dynamic mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets
T0 review · 2 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read Gaussian-randomized portfolios whose samples may short can learn the optimal no-shorting mean-variance strategy without knowing market parameters.
desk verdict Good new construction, load-bearing uniqueness gap, thinner numerics than claimed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a Gaussian exploratory policy with mean constrained to the nonnegative orthant and state-dependent covariance (x−w)²Ψ_t. This variance vanishes exactly at the boundary x = w, so the second-order term in the HJB equation degenerates there, yet in the interior the scaling Lemma 1 reduces the constrained minimization over the mean to a one-parameter family of quadratic programs. The resulting optimal mean takes the form −(x−w)µ^{Γ1}_t on one side and (x−w)µ^{Γ2}_t on the other, matching the known optimal no-shorting control and giving the equivalence between the auxiliary and target problems.
What would settle it
Take a one-dimensional market with constant volatility and constant market price of risk such that µ^{Γ1} > 0 and µ^{Γ2} = 0, and search for a second viscosity solution of the degenerate HJB with Hamiltonian inf_{a≥0}[p a σρ + ½P(a²σ² + (x−w)² tr(Ψσ²))]. Because the Hamiltonian is −∞ at P = 0 whenever pσρ < 0, the standard comparison principle is not automatic; producing one such counterexample would refute Theorem 1 directly.
Extended reading notes
Core claim
For the auxiliary exploratory problem with variance Σ(t,x) = (x−w)²Ψ_t, where Ψ_t is any measurable positive-definite matrix function, the optimal Gaussian policy is N(−(x−w)µ^{Γ1}_t, (x−w)²Ψ_t) when x−w < 0 and N((x−w)µ^{Γ2}_t, (x−w)²Ψ_t) when x−w ≥ 0, with µ^{Γ1}_t and µ^{Γ2}_t defined by quadratic minimization over the nonnegative orthant. The paper claims that the mean of this Gaussian policy is exactly the optimal control of the original no-shorting mean-variance problem, so although individual samples may short, the implemented mean is both feasible and optimal. The value function is piecewise quadratic, continuous across the boundary x = w, and is shown to satisfy the Hamilton-Jacobi-
Load-bearing premise
The central claim depends on the assertion that the piecewise-quadratic function is the unique viscosity solution of a degenerate Hamilton-Jacobi-Bellman equation; the paper invokes a general uniqueness result without verifying the equation's discontinuous Hamiltonian and vanishing second-order term at the boundary, so if uniqueness fails, the optimal Gaussian mean need not equal the optimal no-shorting control.
Editorial extensions
If this is right
- The no-shorting mean-variance optimal portfolio can be learned purely from trajectory data, without estimating drift, volatility, or interest rate.
- Because the exploration variance Ψ_t does not affect the optimal mean, exploration strength can be tuned freely without changing the policy that is ultimately implemented.
- Sampled portfolios may violate no-shorting during training, but the extracted mean policy always respects the constraint, so feasible policies emerge from infeasible exploration.
- At x = w the optimal Gaussian policy degenerates to zero risky investment, so wealth never crosses the boundary and learning can be confined to the region containing the initial wealth.
- The numerical results indicate that even for an extreme target return, the learned terminal mean and variance remain close to the theoretical values.
Reading between the lines
- The same mean-constrained randomization idea likely extends to other constrained linear-quadratic control problems, where the constraint is presumably enforced on the policy mean rather than on the sampled controls.
- A rigorous proof of viscosity uniqueness for the degenerate, discontinuous Hamiltonian would close the main theoretical gap; until then, the equality between the auxiliary optimum and the constrained optimum rests on an invoked standard result.
- The piecewise-linear parameterization of the policy matches the proven structure of the optimal control; a neural-network parameterization would need additional safeguards to preserve nonnegativity and convexity of the value function.
- One could empirically test the theory by varying Ψ₀ and checking whether the learned mean policy and terminal performance stay invariant; systematic drift would suggest the equivalence may fail outside the tested regime.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies continuous-time mean-variance portfolio selection with no-shorting constraints when the investment opportunity set is unknown. It proposes an auxiliary exploratory problem in which policies are Gaussian with nonnegative means and state-dependent variance Σ(t,x)=(x−w)^2 Ψ_t, so sampled controls may violate the no-shorting constraint but their means are admissible. The main theoretical claim (Theorem 1) is that the mean of the optimal Gaussian policy of the auxiliary problem coincides with the no-shorting optimal control of Li–Zhou–Lim (2002). The proof identifies a piecewise quadratic function (13) as the value function of the auxiliary HJB (8), verifies the equation in the interior regions and at the boundary jets, and then invokes a "standard" viscosity uniqueness result. Based on this equivalence, the paper develops a model-free policy-gradient algorithm with martingale loss and reports a numerical example.
Significance. If the main theorem is correct, the paper offers an appealing route to constrained continuous-time MV portfolio selection via RL without estimating (b,r,σ), and the device of constraining the policy mean rather than the policy support is natural and potentially useful beyond this problem. The scaling lemma (Lemma 1) is clean and correctly proven via KKT, and the interior HJB verification is algebraically sound. However, the central identification of (13) as the value function rests on an unproved uniqueness assertion for a degenerate, discontinuous HJB, and the algorithm's convergence guarantee in §4.2 is also asserted rather than established. These gaps are load-bearing for the paper's main claims, though they appear fixable within the manuscript's scope.
major comments (2)
- [Theorem 1 (proof, Step 2); Eq. (8) and Appendix A] The identification of (13) as the value function of the auxiliary problem (6) rests on the assertion "The uniqueness of the viscosity solution is standard; see e.g. Yong and Zhou (1999)", but the hypotheses of any such comparison theorem are not verified. The Hamiltonian in (8)/(27), G(t,x,a,p,P)=p a^T σ ρ + ½P(a^T σ σ^T a + (x−w)^2 tr(Ψ σ σ^T)), is discontinuous at P=0: when P=0 and some component of p^T σ ρ is negative, inf_{a≥0} G = −∞. The equation is also degenerate because the second-order term vanishes at x=w. The cited Yong–Zhou comparison principle is proved under continuity and structural conditions that are not transparently satisfied here. Without uniqueness, (13) is merely one viscosity solution, and the argmin a* computed from it need not be the true optimal mean of (6), so the equivalence with the target problem's optimal control u* is unsupported. Please supply a self-con
- [Section 4.2 (Policy Gradient), p. 13] The text states: "the proximal gradient method is guaranteed to converge because V^φ(0,x;w) is convex in φ; see (20)–(21)". Convexity alone does not imply convergence of (stochastic) projected gradient without additional conditions on Lipschitz smoothness of the gradient, step-size choices, and boundedness of the parameter space. Moreover, the policy gradient estimator in (26) is stochastic and the objective is not shown to be smooth in φ. Either provide a rigorous convergence guarantee with the relevant hypotheses verified, or rephrase this as a heuristic justification. This claim is not essential to the main equivalence theorem, but it is the only theoretical support for Algorithm 1's convergence.
minor comments (5)
- [Section 3, derivation of Eq. (8)] The step "Substituting this form into (7) we can further rewrite the equation as (8)" is not literally justified: equation (7) is written as the HJB for the unrestricted relaxed problem over all π∈P_+(R^m), and the infimum there would not retain the variance term tr[Σ σ σ^T]. Since the admissible class Π is restricted to Gaussian policies with fixed variance, (8) is the correct HJB for that restricted class, but it should be derived directly from the dynamics (5) under the parameterization π=N(a,Σ) rather than presented as an algebraic rewrite of (7).
- [Proof of Theorem 1 / Eq. (13)] The notation Ṗ̃ appears in the verification (LHS calculations) without being defined. It is clear from context that it means the time derivative of P̃, but it should be stated explicitly.
- [Section 4.1] The text says there are "2N+2 vectors to be optimized" for the discretized functions φ^(1) and φ^(2). Since there are two vectors at each of N+1 time points, this is correct, but after fixing φ^(1)_N=φ^(2)_N=0 the number of free vectors should be updated; as written it may confuse readers.
- [References] The reference "Haung, Y. and Zhou, X. Y. (2026)" contains a typo: "Haung" should be "Huang".
- [Algorithm 1] The notation "n warmup" and "nϕ, nθ, nw" is inconsistent; using subscripts (e.g., n_warmup, n_ϕ) would improve readability.
Circularity Check
The equality between the auxiliary problem's optimal mean and the target no-shorting optimal control is engineered into the construction, and the load-bearing uniqueness step is a self-cited, unverified assertion.
-
self definitional
[Section 3, equations (9), (13), (15)-(16)]
"Define μ^{Γ1}_t := argmin_{μ∈R^m_+} [1/2 μ^⊺σ_tσ_t^⊺ μ −(σ_tρ_t)^⊺ μ] ... The obtained optimal mean strategy a∗ turns out to be identical to the theoretically optimal strategy u∗ of the target problem, derived in Li et al. (2002): u∗(t, x;w) = (−(x−w)μ^{Γ1}_t, if x−w<0; (x−w)μ^{Γ2}_t, if x−w≥0)."
The candidate value function (13) is assembled from P^{Γ1}, P^{Γ2}, which are exponentials of the same μ^{Γ1}, μ^{Γ2} defined in (9) that already give the target optimal control (15). The chosen variance Σ(t,x)=(x−w)^2Ψ_t is exactly the one that makes the exploration term independent of a in HJB (8), so the infimum over a is the same quadratic minimization used to define μ^{Γi}. Hence a* = u* is a consequence of the definitions and the variance choice, not an independent output; the theorem verifies an ansatz built from the target solution.
-
uniqueness imported from authors
[Proof of Theorem 1, Section 3]
"This establishes that \V is a viscosity solution of (8). The uniqueness of the viscosity solution is standard; see e.g. Yong and Zhou (1999)."
The step that promotes the piecewise quadratic candidate (13) to the value function of the auxiliary problem, and therefore licenses the derived optimal mean a*, is the uniqueness assertion. It is invoked only by citing Yong and Zhou (1999), a textbook with a co-author of the present paper, and the paper neither states nor verifies the comparison-principle hypotheses for the degenerate, discontinuous HJB (8). Thus the forced identification of (13) as the value function rests on an unexamined self-citation rather than a demonstrated uniqueness result.
full rationale
The core construction is transparently engineered: the variance Σ(t,x)=(x−w)^2Ψ_t is chosen so that the control-dependent part of the HJB is unchanged from the target problem, and the candidate (13) is built from the same μ^{Γ1}, μ^{Γ2} that define the Li–Zhou–Lim optimal control. Consequently the theorem's statement that the mean of the optimal Gaussian policy equals the target optimal control is, to a substantial degree, a designed coincidence rather than a first-principles prediction. The numerical and algorithmic parts are genuinely model-free and are benchmarked against the analytical solution, so the paper is not merely renaming the known result; the 'any Ψ works' generalization gives the construction independent content. The load-bearing uniqueness step, however, is a self-cited standard result whose applicability to this degenerate, discontinuous Hamiltonian is not established in the paper; for the purposes of circularity scoring this counts as an imported uniqueness assertion. Overall, the central equality is partly by construction but rests on a genuine verification, giving a score of 4 rather than 6 or higher.
Assumptions & free parameters
free parameters (4)
- Ψ_0 (exploration variance scale) =
(1/5) I₃ in the numerical example
- learning rates α_θ, α_ϕ, α_w =
0.001, 0.1, 0.01
- batch size M and discretization step T/N =
M=512, T/N=0.01
- initial policy ϕ =
(0.1, 0.1, 0.1)
assumptions (6)
- domain assumption The model coefficients (b_t, r_t, σ_t) are deterministic, bounded, and σσᵀ − δI ≻ 0 (Section 2.2 standing assumptions).
- domain assumption Exploratory dynamics (5): randomized portfolios affect wealth through the mean and variance only, with exploration noise independent of the driving Brownian motion (relaxed-control framework of Wang et al. 2020).
- standard math The target no-shorting MV problem (3) is solved by the piecewise quadratic value P^{Γi}_t(x−w)² − (w−z)² and feedback control (15) from Li et al. (2002).
- ad hoc to paper Uniqueness of the viscosity solution of the degenerate HJB (8) in the relevant function class (asserted 'standard' via Yong and Zhou 1999).
- ad hoc to paper V^φ(0,x;w) is convex in φ, asserted in Section 4.2 to guarantee convergence of the proximal gradient method.
- standard math KKT characterization of the unique minimizer of a strictly convex quadratic over R^m_+ (Lemma 1, citing Boyd and Vandenberghe).
invented entities (2)
-
Auxiliary exploratory problem with mean-constrained Gaussian policies and state-dependent variance Σ(t,x) = (x−w)²Ψ_t
-
Gaussian exploration noise N(0,(x−w)²Ψ_t) added to admissible controls
Cite this review
Pith. "Pith review of Dynamic mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets." pith.science (2026). https://pith.science/paper/PVHHPXXX
@misc{pith2026260716625,
author = {Pith},
title = {Pith review of: Dynamic mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets},
year = {2026},
howpublished = {\url{https://pith.science/paper/PVHHPXXX}},
note = {Machine review of arXiv:2607.16625}
}
read the original abstract
We study continuous-time mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets from a reinforcement learning (RL) perspective. The problem is a constrained stochastic linear -- quadratic control problem for which the entropy-regularized exploratory formulation of Wang et al. (2020) leads to difficulty in theoretical analysis, because enforcing the constraint on the support of randomized policies nullifies the tractable Gaussian exploration. To tackle this challenge, we introduce an auxiliary exploratory problem without entropy in which exploratory policies are still Gaussian whose samples may violate the no-shorting requirement but their means satisfy it. We then prove that, for a suitable choice of exploration variance, the mean of the optimal Gaussian policy of the auxiliary problem coincides with the optimal policy of the original problem. Motivated by this theoretical result, we develop a model-free RL algorithm that learns the optimal policy of the auxiliary (and hence the original) problem directly from trajectory data without estimating the investment opportunity set. A numerical example demonstrates the performance of the proposed algorithm.
Figures
Reference graph
Works this paper leans on
-
[1]
Beck, A. (2017). First-Order Methods in Optimization . MOS-SIAM Series on Optimization. SIAM
2017
-
[2]
Bishop, C. M. and Bishop, H. (2024). Deep Learning: Foundations and Concepts . Springer
2024
-
[3]
and Vandenberghe, L
Boyd, S. and Vandenberghe, L. (2023). Convex Optimization . Cambridge University Press, Cambridge
2023
-
[4]
Chau, H., Nguyen, D., and Nguyen, T. (2026). Continuous-time optimal investment with portfolio constraints: A reinforcement learning approach. European Journal of Operational Research , 328(3):1068--1092
2026
-
[5]
Chen, S., Li, X., and Zhou, X.Y. (1998). Stochastic linear quadratic regulators with indefinite control weight costs. SIAM Journal on Control and Optimization , 36:1685--1702
1998
-
[6]
Dai, M., Dong, Y., Jia, Y., and Zhou, X. Y. (2025). Data-driven Merton 's strategies via policy randomization. arXiv:2312.11797, 2025
arXiv 2025
-
[7]
Guo, X., Li, X., and Xu, R. (2026). Fast policy learning for linear-quadratic control with entropy regularization. SIAM Journal on Control and Optimization , 64:124--151
2026
-
[8]
and Zhou, X
Hu, Y. and Zhou, X. Y. (2005). Constrained stochastic LQ control with random coefficients, and application to portfolio selection. SIAM Journal on Control and Optimization , 44:444--466
2005
Show all 25 references
-
[9]
Huang, Y., Jia, Y., and Zhou, X. Y. (2024). Mean–variance portfolio selection by continuous-time reinforcement learning: Algorithms, regret analysis, and empirical study. arXiv:2412.16175, 2024
2024
-
[10]
Huang, Y., Tang, W., and Zhou, X.Y. (2026). ART for diffusion sampling: Continuous-time control and actor--critic Learning. arXiv:2607.02137, 2026
2026 arXiv
-
[11]
and Zhou, X
Haung, Y. and Zhou, X. Y. (2026). Amortized guidance for image inpainting with pretrained diffusion models. arXiv:2605.13010, 2026
2026 arXiv
-
[12]
Huang, Y.-J., Wang, Z., and Zhou, Z. (2025). Convergence of policy iteration for entropy-regularized stochastic control problems. SIAM Journal on Control and Optimization , 63:752--777
2025
-
[13]
and Zhou, X
Jia, Y. and Zhou, X. Y. (2022a). Policy evaluation and temporal-difference learning in continuous time and space: A martingale approach. Journal of Machine Learning Research , 23:1--55
-
[14]
and Zhou, X
Jia, Y. and Zhou, X. Y. (2022b). Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms. Journal of Machine Learning Research , 23:1--50
-
[15]
and Zhou, X
Jia, Y. and Zhou, X. Y. (2023). q-learning in continuous time. Journal of Machine Learning Research , 24:1--61
2023
-
[16]
and Xu, Z
Li, X. and Xu, Z. Q. (2016). Continuous-time Markowitz 's model with constraints on wealth and portfolio. Operations Research Letters , 44(6):729--736
2016
-
[17]
Y., and Lim, A
Li, X., Zhou, X. Y., and Lim, A. E. B. (2002). Dynamic mean--variance portfolio selection with no-shorting constraints. SIAM Journal on Control and Optimization , 40:1540--1555
2002
-
[18]
Markowitz, H. (1952). Portfolio selection. The Journal of Finance , 7(1):77--91
1952
-
[19]
Sethi, D., S i s ka, D., and Zhang, Y. (2025). Entropy annealing for policy mirror descent in continuous time and space. SIAM Journal on Control and Optimization , 63(4):3006--3041
2025
-
[20]
Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction . Adaptive Computation and Machine Learning Series. The MIT Press, Cambridge, Massachusetts, 2nd edition
2018
-
[21]
P., and Zhou, X
Tang, W., Zhang, Y. P., and Zhou, X. Y. (2022). Exploratory HJB equations and their convergence. SIAM Journal on Control and Optimization , 60:3191--3216
2022
-
[22]
Wang, H., Zariphopoulou, T., and Zhou, X. Y. (2020). Reinforcement learning in continuous time and space: A stochastic control approach. Journal of Machine Learning Research , 21:1--34
2020
-
[23]
and Zhou, X
Wang, H. and Zhou, X. Y. (2020). Continuous-time mean--variance portfolio selection: A reinforcement learning framework. Mathematical Finance , 30:1273--1308
2020
-
[24]
and Zhou, X
Yong, J. and Zhou, X. Y. (1999). Stochastic Controls: Hamiltonian Systems and HJB Equations . Springer
1999
-
[25]
Zhou, X. Y. and Li, D. (2000). Continuous-time mean--variance portfolio selection: A stochastic LQ framework. Applied Mathematics & Optimization , 42:19--33
2000
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.