A non-zero-sum game with reinforcement learning under mean-variance framework

Hao Wang; Junyi Guo; Kam Chuen Yuen; Xia Han

REVIEW 5 major objections 7 minor 42 references

A non-zero-sum game with reinforcement learning under mean-variance framework

T0 review · 5 major / 7 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Two competing investors can learn a time-consistent Nash equilibrium in closed form under a mean-variance objective, even when market parameters are unknown.

desk verdict Real extension of Choquet-regularized mean-variance RL to non-zero-sum games; the equilibrium algebra holds up, but the paper overclaims the class of h, leaves Theorem 4 incomplete, and the numerics are inconsistent. read the letter →

arxiv 2502.04788 v1 pith:GY3U4XPX submitted 2025-02-07 math.OC

Junyi Guo , Xia Han , Hao Wang , Kam Chuen Yuen This is my paper · ORCID

classification math.OC MSC 91A2391G1093E35

keywords mean-variancetime-consistentNashequilibriumChoquetregularizerreinforcementlearningnon-zero-sumdifferentialgameexploratorypolicyGaussianmeanreturnmodeliteration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies two agents who invest competitively, each caring both about their own terminal wealth and about beating the other agent's wealth. Each agent maximizes a mean-variance objective that is regularized by a Choquet penalty on exploration, in a market that can be incomplete and whose parameters may be unknown. The paper's central claim is that this non-zero-sum differential game has a time-consistent Nash equilibrium, and that under a Gaussian mean-return model the equilibrium is given explicitly: each agent's randomized investment policy is a distribution whose quantile function is a known affine function of the market state plus a regularizer-dependent shape. If correct, the result gives a practical reinforcement-learning algorithm with uniform convergence to the equilibrium, even though the standard policy-improvement theorem does not hold for time-inconsistent problems.

What carries the argument

The load-bearing object is the Choquet regularizer $\Phi_h(\Pi) = \int h\circ\Pi([x,\infty))\,dx$, which measures the randomness of an exploratory distribution through a concave distortion function $h$. Its quantile representation $\Phi_h(\Pi)=\int_0^1 Q_\Pi(1-p)\,dh(p)$ reduces the infinite-dimensional maximization over distributions to a mean-variance constraint, and the imported Lemma 1 states that the maximizer has quantile $m + s\,h'(1-p)/\|h'\|_2$ with maximum value $s\|h'\|_2$. This identity lets the extended Hamilton-Jacobi-Bellman verification theorem for time-inconsistent control be solved explicitly, turning the two-agent game into a coupled pair of single-agent optimality conditions for the means while each variance is set independently.

What would settle it

Take the distortion $h(p)=\sqrt p - p$, which is continuous, of bounded variation, and satisfies $h(0)=h(1)=0$. Its derivative $h'(p)=\frac{1}{2\sqrt p}-1$ has divergent $\|h'\|_2$ over $[0,1]$, so the right-hand side of the equilibrium quantile formula (3.16) is undefined; computing this single example would show that Proposition 2 and Theorem 2 require a smoothness condition on $h$ that the paper never states.

Watch

Extended reading notes

Core claim

The authors establish that the game admits a specific time-consistent Nash equilibrium in which each agent's exploratory policy has a quantile function of the form $Q_{\Pi_i^*(t)}(p) = \frac{1}{1-k_1k_2}\big[\frac{\theta(t,y)}{b(t,y)}(\frac{1}{\gamma_i}+\frac{k_i}{\gamma_j}) - \frac{\rho v(t,y)}{b(t,y)}(\frac{\partial d_i(t,y)}{\partial y}+k_i\frac{\partial d_j(t,y)}{\partial y})\big] + \frac{\lambda_i(t)}{\gamma_i b^2(t,y)}h_i'(1-p)$. Here $k_i$ is the agent's sensitivity to the opponent's wealth, $\gamma_i$ is risk aversion, $\lambda_i$ is the exploration weight, and $h_i$ is the Choquet distortion function that measures randomness. The equilibrium value function has the quadratic form $x + \frac12 b_i^2(t)y^2 + b_i^1(t)y + b_i^0(t)$, with coefficient functions determined by one-dimensional ODEs. The paper also proves that a policy-iteration scheme, where the two agents update simultaneously, converges uniformly to this equilibrium despite the absence of monotone policy improvement.

Load-bearing premise

The whole closed-form equilibrium rests on a lemma saying that the most random distribution with given mean and variance has quantiles of the shape $m + s\, h'(1-p)/\|h'\|_2$; this lemma requires $h'$ to exist and have finite squared integral, while the paper only assumes $h$ is continuous with bounded variation, so the formula is not defined for legitimate distortion functions such as $h(p)=\sqrt p - p$ unless an extra regularity condition is added.

Editorial extensions

If this is right

If the central claim is correct, a competitive two-agent mean-variance market can be solved analytically up to one-dimensional ODEs, so no numerical PDE or game-tree search is needed for the Gaussian mean-return model.
The variance of each agent's exploratory distribution depends only on that agent's own risk aversion, exploration weight, and Choquet regularizer, while the mean couples the two agents; this separation explains why the multi-agent problem decomposes into two independent learning tasks.
The uniform convergence of the policy-iteration scheme means that reinforcement learning can be applied to a time-inconsistent equilibrium without relying on monotone improvement, extending RL to a class of problems where policy improvement fails.
Since the quantile formula works for any admissible Choquet regularizer, agents can choose different exploration styles (e.g., Gaussian-like for one agent and Gini/uniform-like for the other) and still share the same equilibrium structure.
In the Black-Scholes complete-market limit, the equilibrium distribution collapses to a formula showing that higher sensitivity to an opponent pushes an agent to take riskier positions while higher risk aversion reduces the mean investment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The mean-variance decomposition suggests an immediate testable extension: if the same model is run with three or more agents, the quantile formula should generalize with the coupling matrix $k_i k_j$ replaced by a Perron-Frobenius condition, and convergence of policy iteration should depend on the spectral radius of that matrix; the paper does not state this extension.
The authors' convergence theorem relies on the Gaussian assumption for the market-state dynamics; an editorially plausible conjecture is that the same uniform convergence holds for any affine mean-reverting state process, because the ODE arguments depend only on linearity and not on Gaussianity beyond the state equation.
A practical test the paper leaves implicit: the Algorithm's empirical convergence should degrade predictably when the Choquet distortion $h$ is chosen so that $h'$ is only of bounded variation but not square-integrable, because the quantile formula itself becomes undefined; a numerical experiment with $h(p)=\sqrt p - p$ would expose this boundary.
The separation of variance from opponent parameters implies that in a competitive setting an agent's exploration intensity is a private decision, not a strategic response; this could be used to design a decentralized multi-agent RL protocol that avoids the usual dimensionality explosion in centralized critics.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The short version: this is a genuine extension, not a repackaging. The authors construct the first non-zero-sum differential game with continuous-time RL and Choquet regularizers, and in the Gaussian mean return model they get a closed-form Nash equilibrium up to one-dimensional ODEs. If you work in continuous-time RL or time-inconsistent portfolio choice, this is worth knowing.

What is actually new: the two-agent mean-variance objective with relative wealth concerns, the Choquet-based exploration, and the observation that policy iteration converges uniformly even though the usual policy improvement theorem fails. The single-agent policy iteration argument in Theorem 3 is sound, and the contraction argument for the means is clean. Decomposing the game into two independent learning problems is a sensible way to avoid multi-agent dimensionality issues.

Now the soft spots, roughly in increasing order.

Mild: Theorem 1 is a verification theorem quoted without proof. Standard in this literature, but it should be sketched.

Medium: the abstract claims the RL algorithm achieves uniform convergence. What is actually proved is uniform convergence of the idealized policy iteration under known parameters. The implemented algorithm uses parameterized critics and a smoothed gradient, and no convergence proof is given for that. The claim needs scaling back.

More serious: the main equilibrium construction overclaims the class H. Lemma 1 needs h' in L^2 for the quantile formula Q = m + s h'/||h'||_2, but H is defined in Section 2.2 as concave functions of bounded variation with h(0)=h(1)=0. That admits h(p)=sqrt(p)-p, for which ||h'||_2 diverges. For such h, Eqs. (3.8), (3.11), and (3.16) are undefined. So Proposition 1, Theorem 2, and Proposition 2 are not established for the full class H; they hold only for a restricted class, say absolutely continuous h with h' in L^2 and not identically zero. This must be stated.

Also serious: the proof of Theorem 4 is incomplete. The simultaneous iteration recursion for the means uses the limiting coefficient functions a_i(t) instead of the current iterate a_i^{(n)}(t). The fixed-point idea is fine once the a_i^{(n)} have converged, but the proof as written does not establish joint convergence. It needs a real argument.

Finally, the numerical section cannot be checked as posted. Table 1 sets rho = -0.93 even though the model assumes rho in [0,1]. The normal variance and uniform support shown in Section 6 are inconsistent with Section 3 by factors involving lambda^2/(gamma^2 sigma^4), because the paper mixes standard deviation and variance. No code or error bars are provided. These are fixable but they undermine the numerical claims.

Bottom line: the equilibrium algebra and the policy iteration idea are legitimate and useful, but the paper as posted overclaims in three places. I would send it to peer review with a request for major revision: restrict H properly, repair the Theorem 4 proof, and redo the numerics. It deserves referee time. For a reading group, maybe; read it knowing the gaps.

Referee Report

5 major / 7 minor

Summary. The paper studies a two-agent non-zero-sum differential game in a continuous-time reinforcement-learning setting. Each agent has a Choquet-regularized, time-inconsistent mean-variance objective that includes a concern for relative wealth, and the market is incomplete with a stochastic factor. The paper derives a time-consistent Nash equilibrium by dynamic programming, obtains an explicit equilibrium in the Gaussian mean-return model, proves uniform convergence of an exact policy iteration, and proposes an actor-critic algorithm with numerical illustrations.

Significance. If the results are correct, the paper gives a useful extension of single-agent exploratory mean-variance RL to a competitive multi-agent setting, with explicit equilibrium policies up to coupled one-dimensional ODEs and a policy iteration that converges without a monotone improvement property. The paper is careful to state the time-inconsistency issue and the non-uniqueness caveat, and the equilibrium characterization is not circular: the target equilibrium solves the PDE system (3.14)-(3.15) independently of the iteration. The main contribution is the closed-form equilibrium plus the uniform-convergence analysis of the exact iteration; the numerical experiments are illustrative rather than a substitute for a convergence theorem for the sampled algorithm. The value of the paper is reduced by several proof gaps and an overly broad regularity class, but these are repairable.

major comments (5)

[Section 2.2, Lemma 1, Eq. (3.8)] The equilibrium construction relies on Lemma 1, whose quantile formula Q(p)=m + s h'(1-p)/||h'||_2 and value s||h'||_2 are defined only when h' exists and is square-integrable on [0,1]. The class H is introduced as concave functions of bounded variation with h(0)=h(1)=0, which admits h(p)=sqrt(p)-p; for this h, h'(p)=1/(2 sqrt(p))-1 is not in L^2, so ||h'||_2 is infinite and Eqs. (3.8), (3.11), (3.15), and (3.16) are undefined. The statements of Proposition 1, Theorem 2, Proposition 2, Corollary 1, and the policy-iteration theorems should be restricted to absolutely continuous, nonconstant h with h' in L^2 and h' not identically zero, or the proof of Lemma 1 under the stated hypotheses should be supplied. As written, the main equilibrium theorem is not established for the claimed class H; this is a load-bearing regularity gap, though it is fixable by adding the standard assumption.
[Section 3.1, Theorem 1] Theorem 1 is the verification theorem that justifies all subsequent equilibrium derivations, but it is stated without proof. The text says the result is analogous to Björk et al. (2017), yet the present objective contains a Choquet regularizer and a two-agent game structure, so the verification argument is not literally a special case. A proof or a precise statement of the hypotheses under which the cited result applies should be provided, including the integrability and smoothness conditions needed for the generator computations in Eqs. (3.2)-(3.4).
[Section 4, Theorem 4] The proof of Theorem 4 replaces the coefficients a^{in}_1(t), a^{in}_2(t) in the policy update (4.5) by the limiting coefficients a_i^1(t), a_i^2(t) inside the recursion for the means. However, (4.5) and (4.6) define a coupled, time-varying iteration: the a-coefficients themselves converge only as n tends to infinity, so the constant-coefficient contraction argument written in the proof does not apply directly. The theorem may be true, but the proof should address the joint convergence of (mu^n_1, mu^n_2, a^{1n}, a^{2n}) or provide a perturbation argument that controls the difference between the time-varying and the limiting recursions.
[Proposition 2, Eq. (3.20)] The displayed closed form for a_i^0(t) contains terms such as e^{2(iota+rho v)T} and e^{(iota+rho v)T} that do not involve (T-t), and it does not satisfy the terminal condition a_i^0(T)=0 implied by d_i(T,y)=0. As printed, Eq. (3.20) is not the solution of the ODE system (3.22). The equilibrium policy (3.17) does not use a_i^0, but the value function (3.18)-(3.19) is part of the claimed analytical solution, so this formula needs to be corrected.
[Abstract, Sections 4-5] The abstract states that 'the proposed algorithm achieves uniform convergence,' but Theorems 3 and 4 analyze the exact policy iteration with full knowledge of the model and exact updates (4.5)-(4.6), not Algorithm 1, which uses function approximation, stochastic gradients, finite samples, and a smoothed functional gradient. The manuscript itself, near the end of Section 6, lists factors that can prevent the sampled algorithm from converging to the true equilibrium. The claim in the abstract should be qualified or a convergence theorem for Algorithm 1 under the stated idealizations should be proved.

minor comments (7)

[Section 2.1, Table 1] The model assumes rho in [0,1] at the start of Section 2.1, but Table 1 uses rho = -0.93; either the assumption should be rho in [-1,1] or the numerical values should be changed to satisfy the stated condition.
[Section 4, Theorem 4 proof] The norm on R^2 is defined as ||x|| = max{x1,x2}, which is not a norm because it can be negative; it should be max{|x1|,|x2|} or the proof should use the max-norm on absolute values.
[Algorithm 1] The initialization block contains a repeated assignment 'xi(tn) <- xi' and the tilde notation for the perturbed wealth path is introduced inconsistently; also the sentence 'Use u to generate ui(tn) and ui(tn)' does not distinguish the unperturbed and perturbed actions. Please clean up the pseudocode.
[Section 3.3 title] The heading 'Gauss mean return model' should read 'Gaussian mean return model' for consistency with the text.
[Eqs. (3.15) and (3.21)] The sign of the term involving gamma_i k_i b^2 sigma_j^2 appears to differ between Eq. (3.15) and the equation for b_i^0'(t) in (3.21); please check whether the signs are consistent after substitution.
[Figure 2] The label 'Uniform Distributiion' in the second panel contains a typo; it should be 'Uniform Distribution'.
[Section 2.2, definition of H] The standing assumptions on h are stated twice in different ways: first 'Given a concave function h : [0,1] -> R of bounded variation with h(0)=h(1)=0' and then 'We denote the set of h : [0,1] -> R by H.' Please define H once, with the concavity and normalization requirements, and use that definition throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the equilibrium is characterized by an independently solved verification PDE system, and the policy iteration converges to that characterized target.

full rationale

The derivation is self-contained: the equilibrium response is defined through the extended HJB system (3.2)-(3.3), and the paper solves that system using the ansatz (3.5), obtaining the PDEs (3.14)-(3.15) and the explicit quantile formula (3.16). The main external input is Lemma 1, imported from Liu et al. (2020), which is a mathematical result about Choquet maximizers and is not a renamed version of the target equilibrium. The policy iteration in Theorems 3-4 is analyzed against that same independently characterized target: the ODEs (4.6) are shown to converge uniformly to the coefficients in (3.20), and the mean recursion is a contraction whose fixed point is the equilibrium mean computed separately in (3.13). Citations to Han et al. (2023) and Guo et al. (2023) are contextual, supporting properties of Choquet regularizers, and are not load-bearing for the equilibrium derivation. No parameter is fitted to a subset of data and then reported as a prediction. The potential regularity gap in Lemma 1 for distortion functions h whose derivative is not square-integrable is a correctness issue about the class H, not a circularity.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim carries no fitted numerical constants: gamma_i, k_i, lambda_i, h_i and the Gaussian market parameters are inputs, not parameters estimated from data in this paper. The main unpaid assumptions are the imported verification theorem, the exploratory wealth-process limit, and the quantile-maximizer lemma with an unstated square-integrability condition on h'. No new physical or economic entities are introduced; the idiosyncratic Brownian motions B_i in (2.9) are inherited from the exploratory-RL framework.

free parameters (6)

risk aversion gamma_i = gamma1=2, gamma2=1 in Table 1; gamma1=2, gamma2=3 in Table 2
Model primitives controlling the variance penalty in Eq. (2.10); chosen by the authors, not fitted.
sensitivity k_i = k1=0.1, k2=0.05 in Tables 1 and 2; varied in Figure 1
Model primitives measuring how much each agent cares about the wealth gap; chosen, not fitted.
exploration weight lambda_i(t) = lambda_i(t)=lambda0 e^{lambda0(T-t)}, lambda0=0.01; lambda1=0.015, lambda2=0.02 in Tables 1 and 2
Controls the variance of the exploratory distribution in Eq. (3.11); chosen by hand.
Choquet distortion h_i = h1(p)=integral_0^p z(1-s)ds, h2(p)=p-p^2 in Section 6
Selected regularizers; determine equilibrium support shape.
market parameters (r, sigma, iota, v, Y, rho) = r=0.017, sigma=0.15, iota=0.27, v=0.065, Y=0.273, rho=-0.93 in Table 1
Adopted from Wachter (2002) and Dai et al. (2023); rho=-0.93 conflicts with the model's rho in [0,1].
algorithm hyperparameters = N=250, alpha=0.001, kappa small, M=50,000, p(theta,t)=theta1 t^2 + theta0 t
Chosen for the numerical example; do not enter the theoretical equilibrium characterization.

assumptions (6)

standard math Brownian market with deterministic coefficients and standard filtered probability space.
Section 2.1, assumed for the SDEs and admissible controls.
domain assumption Exploratory wealth dynamics (2.9) are the correct aggregate limit of randomized strategies.
Section 2.1, Eqs. (2.4)-(2.9); law-of-large-numbers heuristic from Wang et al. (2020a), not rigorously proved for this game.
domain assumption Extended HJB verification theorem (Theorem 1) applies to the two-agent Choquet-regularized game.
Stated in Section 3.1 as analogous to Bjork et al. (2017), no proof given.
domain assumption The Choquet maximizer lemma (Liu et al. 2020) is valid for the h_i used, including square-integrability of h'_i.
Lemma 1 and Eq. (3.8); the paper does not state the required differentiability condition.
domain assumption In the Gaussian model all constants are positive and rho is in [0,1].
Section 3.3 and Example 1; Table 1 violates the condition with rho=-0.93.
ad hoc to paper The initial policy can be chosen in the form (4.4), and value functions have the quadratic form used in the ansatz.
Section 3.3 ansatz (3.18)-(3.19) and Section 4 initial policy (4.4); these are presentation choices verified by solving ODEs.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A non-zero-sum game with reinforcement learning under mean-variance framework." pith.science (2026). https://pith.science/paper/GY3U4XPX

@misc{pith2026250204788,
  author       = {Pith},
  title        = {Pith review of: A non-zero-sum game with reinforcement learning under mean-variance framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GY3U4XPX}},
  note         = {Machine review of arXiv:2502.04788}
}

read the original abstract

In this paper, we investigate a competitive market involving two agents who consider both their own wealth and the wealth gap with their opponent. Both agents can invest in a financial market consisting of a risk-free asset and a risky asset, under conditions where model parameters are partially or completely unknown. This setup gives rise to a non-zero-sum differential game within the framework of reinforcement learning (RL). Each agent aims to maximize his own Choquet-regularized, time-inconsistent mean-variance objective. Adopting the dynamic programming approach, we derive a time-consistent Nash equilibrium strategy in a general incomplete market setting. Under the additional assumption of a Gaussian mean return model, we obtain an explicit analytical solution, which facilitates the development of a practical RL algorithm. Notably, the proposed algorithm achieves uniform convergence, even though the conventional policy improvement theorem does not apply to the equilibrium policy. Numerical experiments demonstrate the robustness and effectiveness of the algorithm, underscoring its potential for practical implementation.

Figures

Figures reproduced from arXiv: 2502.04788 by the authors.

**Figure 2.** The mean value of Nash equilibrium 7 Conclusion This paper introduces a framework for continuous-time RL in a competitive market, where two agents consider both their own wealth and their opponent’s wealth under the mean-variance criterion. The Nash equilibrium distributions are derived through dynamic programming, revealing that an agent’s mean of equilibrium distribution for exploration is influenced not only by … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 40 canonical work pages

[1]

and Chabakauri, G

Basak, S. and Chabakauri, G. (2010). Dynamic mean-variance asset allocation. The Review of Financial Studies , 23(8), 2970–3016

work page 2010
[2]

and Frehse, J

Bensoussan, A. and Frehse, J. (2000). Stochastic games for n players. Journal of Optimization Theory and Application , 105(3), 543–565

work page 2000
[3]

and Yang, H

Bensoussan, A., Siu, C, Yam, S. and Yang, H. (2014). A class of non-zero-sum stochastic diﬀerential investment and reinsurance games. Automatica, 50(8), 2025–2037

work page 2014
[4]

and Prashanth, L

Bhatnagar, S., Prasad, H. and Prashanth, L. (2013). Stochastic Recursive Algorithms for Opti- mization: Simultaneous Perturbation Methods . Springer. Bj¨ork, T., Khapko, M. and Murgoci, A. (2017). On time-inconsis tent stochastic control in contin- uous time. Finance and Stochastics , 21, 331–360. Bj¨ork, T. and Murgoci, A. (2010). A general theory of Mar...

work page 2013
[5]

Browne, S. (2000). Stochastic diﬀerential portfolio games. Joumal of Applied Probability , 37(1), 126–147

work page 2000
[6]

and Shen, Y

Chen, L. and Shen, Y. (2019). Stochastic Stackelberg diﬀeren tial reinsurance games under time- inconsistent mean-variance framework. Insurance: Mathematics and Economics , 88, 120–137

work page 2019
[7]

and Jia, Y

Dai, M., Dong, Y. and Jia, Y. (2023). Learning equilibrium me an-variance strategy. Mathematical Finance, 33(4), 1166–1212

work page 2023
[8]

and Xu, Y

Dai, M., Jin, H., Kou, S. and Xu, Y. (2021). A dynamic mean-var iance analysis for log returns. Management Science, , 67(2), 1093–1108

work page 2021

Show all 42 references

[9]

Doya, K. (2000). Reinforcement learning in continuous time and space. Neural Computation, 12(1), 219–245

2000
[10]

Elliott, R. (1976). The existence of value in stochastic diﬀe rential games. SIAM Journal on Control and Optimization , 14(1), 85–94

1976
[11]

and Siu, T

Elliott, R. and Siu, T. (2011). A stochastic diﬀerential game for optimal investment of an insurer with regime switching. Quantitative Finance , 11(3), 365–380

2011
[12]

and Pirvu, T

Ekeland, I. and Pirvu, T. A. (2008). Investment and consumpt ion without commitment. Mathe- matics and Financial Economics , 2(1), 57–86

2008
[13]

and Touzi, N

Espinosa, G. and Touzi, N. (2015). Optimal investment under relative performance concerns. Mathematical Finance, 25(2), 221–257

2015
[14]

H., Kohli, P., and Whiteson, S

Foerster, J., Nardelli, N., Farquhar, G., Afouras, T., Torr , P. H., Kohli, P., and Whiteson, S. (2017). Stabilising experience replay for deep multi-agen t reinforcement learning. In Interna- tional Conference on Machine Learning , pages 1146–1155

2017
[15]

and Zitikis, R

Furman, E., Wang, R. and Zitikis, R. (2017). Gini-type measu res of risk and variability: Gini shortfall, capital allocation and heavy-tailed risks. Journal of Banking and Finance , 83, 70–84

2017
[16]

and Schmeidler, D

Gilboa, I. and Schmeidler, D. (1989). Maxmin expected utili ty with non-unique prior. Journal of Mathematical Economics, 18(2), 141–153

1989
[17]

and Wang, H

Guo, J., Han, X. and Wang, H. (2023). Exploratory mean-varia nce portfolio selection with Choquet regularizers. arXiv : 2307.03026

2023 arXiv
[18]

Isaacs, R. (1965). Diﬀerential Games , Wiley, New York

1965
[19]

and Zhou, X

Han, X., Wang, R. and Zhou, X. Y. (2023). Choquet regularizat ion for continuous-time reinforce- ment learning. SIAM Journal on Control and Optimization , 61(5), 2777–2801

2023
[20]

and Weng, C

Jiang, R., Saunders, D. and Weng, C. (2022). The reinforceme nt learning Kelly strategy. Quanti- tative Finance, 22(8), 1445–1464

2022
[21]

Kim, T. S. and Omberg, E. (1996). Dynamic nonmyopic portfoli o behavior. The Review of Finan- cial Studies , 9(1), 141–161

1996
[22]

Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv :1412.6980

2014 arXiv
[23]

and Tsitsiklis, J

Konda, V. and Tsitsiklis, J. (2000). Actor-critic algorith ms. In Advances in Neural Information Processing Systems, pages 1008–1004

2000
[24]

and Ng, W

Li, D. and Ng, W. L. (2000). Optimal dynamic portfolio select ion: Multiperiod mean-variance formulation, Mathematical Finance, 10, 287–406

2000
[25]

and Young, V

Li, D. and Young, V. R. (2021). Bowley solution of a mean-vari ance game in insurance. Insurance: Mathematics and Economics , 98, 35–43

2021
[26]

Littman, M. L. (1994). Markov games as a framework for multi- agent reinforcement learning. In Machine learning proceedings 1994 , pages 157–163

1994
[27]

Littman, M. L. (2001). Friend-or-foe Q-learning in general -sum games. In International Conference on Machine Learning , pages 322–328

2001
[28]

and Wang, R

Liu, F., Cai, J., Lemieux, C. and Wang, R. (2020). Convex risk functionals: Representation and applications. Insurance: Mathematics and Economics , 90, 66–79

2020
[29]

Liu, J. (2001). Dynamic Portfolio Choice and Risk Aversion, working paper, UCLA

2001
[30]

Markowitz, H. (1952). Portfolio selection. The Journal of Finance , 7(1), 77–91

1952
[31]

Merton, R. C. (1980). On estimating the expected return on th e market: An exploratory investi- 23 gation. Journal of Financial Economics , 8(4), 323–361

1980
[32]

Pontryagin, L.S. (1967). Linear diﬀerential games. I, II, in : Doklady Akademii Nauk. Russian Academy of Sciences , 175, 764–766

1967
[33]

Quiggin, J. (1982). A theory of anticipated utility. Journal of Economic Behavior and Organiza- tion, 3(4), 323–343

1982
[34]

Schmeidler, D. (1989). Subjective probability and expecte d utility without additivity. Economet- rica, 57(3), 571–587

1989
[35]

and Jia, G

Sun, Z. and Jia, G. (2023). Reinforcement learning for explo ratory linear-quadratic two-person zero-sum stochastic diﬀerential games. Applied Mathematics and Computation , 442, 127763

2023
[36]

Sutton, R. S. and Barto, A. G. (2018). Reinforcement learning:An introduction. Cambridge, MA: MIT Press

2018
[37]

Wachter, J. A. (2002). Portfolio and consumption decisions under mean-reverting returns: An exact solution for complete markets. Journal of Financial and Quantitative Analysis , 37(1), 63–91

2002
[38]

and Zhou, X

Wang, H. and Zhou, X. Y. (2020). Continuous-time mean-varia nce portfolio selection: A rein- forcement learning framework. Mathematical Finance, 30(4), 1273–1308

2020
[39]

Yaari, M. E. (1987). The dual theory of choice under risk. Econometrica, 55(1), 95–115

1987
[40]

Yang, Y., and Wang, J. (2020). An overview of multi-agent rei nforcement learning from game theoretical perspective. arXiv :2011.00583

2020 arXiv
[41]

Zhang, K., Yang, Z., and Ba¸ sar, T. (2021). Multi-agent rein forcement learning: A selective overview of theories and algorithms. Handbook of reinforcement learning and control , 321-384

2021
[42]

and Li, D

Zhou, X. and Li, D. (2000). Continuous-time mean-variance p ortfolio selection: A stochastic LQ framework. Applied Mathematics and Optimization , 42(1), 19–33. Zeng Y., Li D. and Gu A. (2016). Robust equilibrium reinsuran ce-investment strategy for a mean- variance insurer in ...

2000

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

and Chabakauri, G

Basak, S. and Chabakauri, G. (2010). Dynamic mean-variance asset allocation. The Review of Financial Studies , 23(8), 2970–3016

work page 2010

[2] [2]

and Frehse, J

Bensoussan, A. and Frehse, J. (2000). Stochastic games for n players. Journal of Optimization Theory and Application , 105(3), 543–565

work page 2000

[3] [3]

and Yang, H

Bensoussan, A., Siu, C, Yam, S. and Yang, H. (2014). A class of non-zero-sum stochastic diﬀerential investment and reinsurance games. Automatica, 50(8), 2025–2037

work page 2014

[4] [4]

and Prashanth, L

Bhatnagar, S., Prasad, H. and Prashanth, L. (2013). Stochastic Recursive Algorithms for Opti- mization: Simultaneous Perturbation Methods . Springer. Bj¨ork, T., Khapko, M. and Murgoci, A. (2017). On time-inconsis tent stochastic control in contin- uous time. Finance and Stochastics , 21, 331–360. Bj¨ork, T. and Murgoci, A. (2010). A general theory of Mar...

work page 2013

[5] [5]

Browne, S. (2000). Stochastic diﬀerential portfolio games. Joumal of Applied Probability , 37(1), 126–147

work page 2000

[6] [6]

and Shen, Y

Chen, L. and Shen, Y. (2019). Stochastic Stackelberg diﬀeren tial reinsurance games under time- inconsistent mean-variance framework. Insurance: Mathematics and Economics , 88, 120–137

work page 2019

[7] [7]

and Jia, Y

Dai, M., Dong, Y. and Jia, Y. (2023). Learning equilibrium me an-variance strategy. Mathematical Finance, 33(4), 1166–1212

work page 2023

[8] [8]

and Xu, Y

Dai, M., Jin, H., Kou, S. and Xu, Y. (2021). A dynamic mean-var iance analysis for log returns. Management Science, , 67(2), 1093–1108

work page 2021