Learning in Mean Field Games: A Survey
read the original abstract
Non-cooperative and cooperative games with a very large number of players have many applications but remain generally intractable when the number of players increases. Introduced by Lasry and Lions, and Huang, Caines and Malham\'e, Mean Field Games (MFGs) rely on a mean-field approximation to allow the number of players to grow to infinity. Traditional methods for solving these games generally rely on solving partial or stochastic differential equations with a full knowledge of the model. Recently, Reinforcement Learning (RL) has appeared promising to solve complex problems at scale. The combination of RL and MFGs is promising to solve games at a very large scale both in terms of population size and environment complexity. In this survey, we review the quickly growing recent literature on RL methods to learn equilibria and social optima in MFGs. We first identify the most common settings (static, stationary, and evolutive) of MFGs. We then present a general framework for classical iterative methods (based on best-response computation or policy evaluation) to solve MFGs in an exact way. Building on these algorithms and the connection with Markov Decision Processes, we explain how RL can be used to learn MFG solutions in a model-free way. Last, we present numerical illustrations on a benchmark problem, and conclude with some perspectives.
This paper has not been read by Pith yet.
Forward citations
Cited by 10 Pith papers
-
Universal Approximation of Nonlinear Operators and Their Derivatives
Proves the first universal approximation theorems for k-times differentiable nonlinear operators between Banach spaces and their derivatives uniformly on compact sets in weighted Sobolev norms via encoder-decoder oper...
-
Robust $Q$-learning for mean-field control under Wasserstein uncertainty in common noise
Robust Q-learning algorithm with convergence and finite-time bounds for mean-field control under Wasserstein uncertainty in common noise.
-
Travel-time tomography from mean field game dynamics
The authors formulate active travel-time tomography via mean field games in a PDE-constrained setting and introduce a two-stage diffusion-to-MFG inversion pipeline that recovers spatial cost fields stably under noise.
-
Travel-time tomography from mean field game dynamics
Introduces a mean field game model for active travel-time tomography with a two-stage diffusion-plus-MFG inversion pipeline and reports stable numerical recovery.
-
Universal Approximation of Nonlinear Operators and Their Derivatives
Proves first UATs for k-times differentiable nonlinear operators and their derivatives via OL architectures uniformly on compact sets in weighted Bastiani-Sobolev spaces on general Banach spaces.
-
Towards Model-Free Learning in Dynamic Population Games: An Application to Karma Economies
Model-free DQN learning achieves suboptimality bounds of O(1/sqrt(Ns)) + O(1/N) in Karma DPGs at equilibrium, and deep RL combined with fictitious play empirically reaches near-Stationary Nash Equilibrium from scratch.
-
Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithms
The authors propose actor-critic q-learning algorithms for mean-field control with common noise based on martingale orthogonality conditions and relaxed controls, establish convergence of inner iterations in the linea...
-
Who Restores the Peg? A Mean-Field Game Approach to Model Stablecoin Market Dynamics
Mean-field game equilibrium calibrated to historical de-pegs attributes most peg recovery to primary-market arbitrage and identifies a nonlinear stress threshold for slow recovery.
-
Neural Mean-Field Games: Extending Mean-Field Game Theory with Neural Stochastic Differential Equations
Neural mean-field games integrate mean-field game theory with neural SDEs to learn strategic interactions from data in a model-free way, demonstrated on games and viral dynamics.
-
Mean Field Reinforcement Learning
A monograph develops the probabilistic and control-theoretic framework connecting multi-agent reinforcement learning to mean field control, including analyses of Q-learning, policy gradients, and numerical methods for...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.