Pith. sign in

REVIEW 3 major objections 6 minor 70 references

ADAGE: A generic two-layer framework for adaptive agent based modelling

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ADAGE claims one Stackelberg formulation can handle four core agent-based modelling tasks.

desk verdict ADAGE is a genuinely useful framework synthesis, but its central claim to solve a Stackelberg equilibrium rests on an unproven—and likely overstated—equivalence. read the letter →

arxiv 2501.09429 v1 pith:IRD3OUGM submitted 2025-01-16 cs.MA cs.AIcs.LGecon.GNq-fin.CPq-fin.EC

Benjamin Patrick Evans , Sihan Zeng , Sumitra Ganesh , Leo Ardon This is my paper · ORCID
classification cs.MAcs.AIcs.LGecon.GNq-fin.CPq-fin.EC
keywords agent-basedmodellingStackelberggamesbi-leveloptimisationmulti-agentreinforcementlearningmodelcalibrationpolicydesignscenariogenerationmeta-learning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ADAGE formalizes adaptive agent-based modelling as a two-layer Stackelberg game: an outer leader chooses environment characteristics θ (tax rules, latent parameters, tax rates, or preference samples), and inner follower agents learn behavioural policies conditioned on those characteristics. The paper claims this one formulation subsumes four previously separate modelling tasks—policy design, calibration, scenario generation, and robust behavioural learning—because changing only the leader's reward and action space changes the task. If correct, modellers would not need bespoke algorithms for each task; the same coupled optimisation of leader and follower policies would adapt agents to environmental change while also adapting the environment to agent behaviour, directly addressing the Lucas critique. The paper demonstrates each task in a different economic or financial simulator, and swaps in different learning algorithms for the layers, to show that the framework is generic rather than tied to one simulator or one optimiser.

What carries the argument

The load-bearing object is the conditional behavioural policy $\pi_i(a \mid o_i, \hat{\theta}_i)$: each follower's action distribution is a function of its local observation plus its view of the leader-chosen characteristics θ. The leader's action space and reward define the task, and θ parameterises the environment the followers perceive. The training problem is then reduced to solving the coupled non-linear system of first-order optimality conditions, Eq. (4), for all $n+1$ agents; the paper invokes the gradient-domination property of policy-gradient returns [1] to claim every stationary point is globally optimal, and uses alternating gradient descent/ascent with leader learning rates much larger than follower rates to approximate a nested-loop best-response oracle. In practice the follower policies are PPO networks, but the framework is presented as algorithm-agnostic: an analytic maximum-entropy distribution replaces the learned outer policy in the robust-behaviour task, and a Bayesian optimiser replaces the RL outer layer in one calibration variant.

What would settle it

Run a targeted policy search around the leader policy ADAGE converges to in any of the four environments, with the learned follower policies held fixed: if a leader deviation improves its own return while followers are allowed to best-respond, the stationary point found by Eq. (4) is not a Stackelberg equilibrium. Directly, one can test the gradient-domination premise in the cobweb calibration by sweeping many random follower and leader policies and checking whether every low-gradient point has return close to the best found; if not, the single-agent gradient-domination result does not transfer to this multi-agent game.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a broad class of adaptive ABM problems can be written as a partially observable Markov game with $n+1$ agents: agent 0 is the leader and the remaining $n$ agents are followers, and the solution concept is a Stackelberg equilibrium—the leader maximises its expected return given that followers respond optimally to whatever the leader does. The leader's policy determines a characteristics variable θ, and each follower conditions its behavioural policy on an observation of θ, so follower behaviour adapts automatically when the environment changes. The paper argues that because the expected return in Eq. (2) satisfies the gradient-domination condition, every stationary point of the coupled first-order conditions ($\nabla_{\pi_i} R_i = 0$ for all $i$, and $\nabla_{\pi_L} R_L = 0$) is globally optimal; hence solving those non-linear equations by alternating gradient descent/ascent yields a Stackelberg equilibrium. Changing only the leader's reward and action space turns the same system into: policy design (maximise household welfare by setting tax parameters), calibration (match simulated prices to experimental data by tuning bounded-rationality costs), scenario generation (stabilise market volatility by setting a Tobin tax), and robust behavioural learning (maximise entropy over market-maker preferences to learn one policy that extrapolates across preferences). The experiments are demonstrations that these four tasks are special instances of the framework, not head-to-head comparisons of learning algorithms.

Load-bearing premise

The argument works only if a joint stationary point of the coupled gradient conditions is actually a Stackelberg equilibrium—the paper assumes the single-agent gradient-domination property transfers to the $n+1$-agent leader-follower Markov game, but gives no proof for that setting.

Editorial extensions

If this is right

  • If ADAGE is correct, a modeller can switch between policy design, calibration, scenario generation, and robust behavioural learning by editing only the leader's reward function and action space, keeping the same POMG representation and Stackelberg solution concept.
  • Follower policies that condition on θ should react to new environment characteristics without retraining, which is the direct mechanism by which ADAGE claims to soften the Lucas critique.
  • Because the framework is simulator-independent, results from one ABM (e.g., Tobin tax stabilising a market) transfer as a template to other simulation platforms, and algorithm-independent, so the inner or outer layer can be swapped for analytic or Bayesian solvers.
  • The calibration experiment implies that bounded-rationality parameters can be fit to human experimental data through the same bi-level loop that also handles welfare maximisation, so calibration and policy design are not separate methodological islands.
  • The robust-behaviour experiment implies that a single conditional policy can extrapolate across a continuum of agent preferences, so the framework offers a route to meta-learning within an ABM without a separate meta-loop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper's theoretical justification for Eq. (4) relies on a single-agent gradient-domination result, a natural next test is whether the learned stationary points in these environments are truly Stackelberg equilibria; if not, ADAGE still works as a bi-level heuristic but its 'solution of a coupled set of non-linear equations' claim is weaker than stated.
  • The same conditional-policy mechanism could be applied to tasks the paper does not run, such as inverse reinforcement learning of agent preferences from data, or robust policy design that optimises worst-case welfare across a distribution of θ rather than a single realisation.
  • The scenario-generation result suggests a cheap way to screen proposed market interventions: run ADAGE as an automated search over policy parameters and inspect which θ produce the desired emergent outcome, rather than hand-tuning those parameters.
  • Although the paper uses one leader, it notes multiple leaders are possible; a multi-leader extension would let several designers or regulators co-adapt, which would need a different equilibrium concept than the single-leader Stackelberg solution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ADAGE, a two-layer framework for adaptive agent-based modelling in which a leader agent in an outer layer adjusts environment characteristics θ (e.g., tax parameters, calibration targets, scenario controls, or sampling distributions) while follower agents in an inner simulation layer learn behavioural policies conditioned on their observations of θ. The problem is formulated as a partially observable Markov game with n+1 agents and is claimed to be a Stackelberg game; the paper argues that solving a coupled system of first-order equations (Eq. 4) yields the desired equilibrium. Four tasks are presented as instantiations of the framework: policy design (TaxAI), calibration (cobweb market with bounded-rational agents), scenario generation (Tobin tax for market stabilisation), and robust behavioural learning (market maker across preferences). The experiments use PPO for most agents, with a Bayesian outer layer and an analytically derived outer layer as algorithm-independence demonstrations. The paper includes source code in the supplementary material.

Significance. If the formal claims held, ADAGE would provide a useful unification of several common ABM tasks under a single bi-level optimisation view, and the paper demonstrates genuine breadth: multiple simulators, mixed discrete/continuous actions, different learning algorithms in the outer layer, and comparisons against external baselines such as free-market, rational-expectations, and fixed-ω training. The manuscript also honestly reports that convergence to equilibrium is not proven and that best responses are approximate. However, the central theoretical step—identifying solutions of Eq. (4) with Stackelberg equilibria—is not established, and the experimental evaluation is mostly illustrative. The paper is therefore best read as a framework-and-demonstration contribution; with a precise restatement of what is proved and what is heuristic, it could be a valuable consolidation of adaptive-ABM methodology.

major comments (3)
  1. [§3.2, Eq. (4)] The Stackelberg equilibrium defined in §3.1 is a leader commitment π_L maximising R_L against the follower best-response oracle ε(π_L), but §3.2 re-characterises the objective as a point at which no agent can improve its local objective holding all other agents' behaviours fixed, and Eq. (4) is the first-order condition for that Nash-style problem. The cited gradient-domination result [1] is proven for single-agent policy gradients; no argument is given that it extends to the n+1-agent partially observable Markov game, and even a full multi-agent analogue would at most identify global Nash equilibria of the simultaneous-move game, not the leader's commitment optimum against followers' best responses. The manuscript's caveat that it only learns approximate best responses is honest, but it means the statement that the framework is 'formalised' as a Stackelberg game and 'based on solving a coupled set of non-linear equations' is asserted rather than derived. Please provide a proof or precise reference for the multi-agent gradient-domination condition in this leader-follower setting, or reframe the contribution as bi-level optimisation with Stackelberg-inspired semantics rather than as an exact Stackelberg equilibrium solver.
  2. [§3.2, alternating gradient descent/ascent] Even if Eq. (4) were the correct stationarity condition, the proposed A-GD algorithm updates each policy against the latest iterates of all other agents and relies on the learning-rate separation α_L >> α_F to approximate nested-loop play. The cited convergence results [28, 57] are for two-time-scale stochastic approximation and bilevel optimisation under assumptions such as strong convexity/smoothness that are not verified for the POMG returns in Eq. (2). The paper therefore does not support the claim that running A-GD converges to a Stackelberg equilibrium; at best this is a heuristic. The text should state this explicitly wherever the optimisation method is described, rather than presenting Eq. (4) and A-GD as the solving mechanism for the formal Stackelberg problem.
  3. [§4.2–4.3, experimental evaluation] The empirical claims would be easier to assess with uncertainty quantification. In Table 1, MAE and RMSE are reported without confidence intervals or a statement of the number of seeds/rollouts; in Table 2, ADAGE is worse than baseline for 2 of 9 capacities under mean σ and for 3 of 9 under mean absolute percentage change, yet the text says the approach 'successfully restricts market volatility in the vast majority of configurations'. Reporting standard errors, per-capacity bootstrap intervals, and the number of independent runs behind each number would make the conclusions commensurate with the evidence and would also clarify whether the observed improvements are consistent rather than noise.
minor comments (6)
  1. [§3.2, Eq. (4)] The notation ∇_{π*_i} R_i = 0 is confusing because π*_i denotes the unknown solution; the intended meaning is ∇_{π_i} R_i evaluated at π_i = π*_i, which should be written explicitly.
  2. [§4.3.2] The text refers to 'the conventional case of c = 0.6 [4]', but the experiments vary capacity C over 2,...,18 and no capacity of 0.6 appears; the cited reference [4] also does not appear to define this convention.
  3. [§4.4.1] The sentence 'By optimising for eq. (18)' is incorrect: Eq. (18) defines market share, whereas the objective to be optimised is the MM reward in Eq. (16).
  4. [Table 3] Table 3 reports 'Training iterations 100, 1000'; please specify which environment uses which value, otherwise the configuration is ambiguous.
  5. [Supplementary Fig. 9] Supplementary Fig. 9 omits the C=12 panel even though Table 2 reports results for C=12; please include the missing panel or explain the omission.
  6. [§4.1.2] The policy-design experiment compares only against a free-market baseline; adding a standard tax schedule, such as a flat or proportional tax, would make the claim of improved social welfare more informative.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the main caveat is an unproved Stackelberg/Nash equivalence in Section 3.2, which is a correctness concern rather than a circular reduction.

full rationale

ADAGE's central contribution is a formulation: modelling two-layer adaptive agent-based modelling as a Stackelberg game and solving the resulting coupled first-order conditions. The claimed encapsulation of tasks (policy design, calibration, scenario generation, robust behavioural learning) is achieved by explicit construction: Section 3.1.1 states 'By varying the leader’s reward function and action space, we can capture many common ABM tasks', so the derivation of tasks as special instances is a definitional specialisation, not a fitted parameter disguised as a prediction. Experimental evaluations are against external baselines and data: the free-market baseline in Section 4.1.2, the experimental cobweb data of Hommes et al. in Section 4.2.2 (Table 1), and the fixed-omega retraining baseline in Section 4.4.2. No fitted parameter is relabelled as a forecast or as an independent prediction. The self-citations ([3], [16], [17], [37]) supply environments, information-cost penalties, and experimental setups; they are not load-bearing evidence for the framework's equilibrium claim. The genuine caveat is in Section 3.2, where the paper equates a Stackelberg equilibrium with a joint no-agent-can-improve condition—which is Nash stationarity—and invokes the single-agent gradient-domination result [1] to justify equation (4). This is a correctness or validity gap, because the multi-agent gradient-domination condition is not proved and the cited theorem does not bridge the leader-commitment problem. However, it is not circularity under the hard rules: equation (4) is not equivalent to the paper's input by construction, and the authors explicitly disclaim convergence guarantees in Section 3.1 ('The focus of this work is not on theoretical convergence guarantees to the equilibrium... instead we approximate the equilibria by learning (approximate) best responses'). Unsupported inference is not circular reduction, so the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or mathematical entities are introduced; the characteristics variable theta is a modeling construct, not an invented entity. The main assumptions are the validity of the Stackelberg formulation and the transfer of single-agent gradient domination to the multi-agent setting.

free parameters (3)
  • mu (mean of information processing cost distribution) = learned (exact value not reported)
    In the Cobweb calibration experiment (Section 4.2), the leader fits distributional parameters mu, sigma of the lambda_i penalties to match experimental price data.
  • sigma (std of information processing cost distribution) = learned (exact value not reported)
    Same as mu; the calibrator updates sigma to minimize price prediction error.
  • Tobin tax upper bound tau_max = 0.1
    The scenario generator action space constrains tau_t to [0, 0.1] without theoretical justification; this hand-chosen bound affects the attainable volatility reduction.
assumptions (4)
  • domain assumption The Stackelberg game with one leader and n followers is an appropriate formalization of adaptive agent-based modelling.
    Used throughout Section 3 to justify the bi-level structure; no evidence that alternative game-theoretic formulations would not be equally or more appropriate.
  • ad hoc to paper The gradient domination condition from [1] holds for the multi-agent returns in Eq. (2), so every stationary point of the joint system is globally optimal.
    Invoked in Section 3.2 to claim Eq. (4) suffices for a Stackelberg equilibrium; the cited result is for single-agent policy gradients and is not established for n+1 interacting agents.
  • domain assumption The alternating gradient descent/ascent algorithm converges to a Stackelberg equilibrium with appropriately chosen learning rates.
    The paper states convergence is guaranteed under strong structure [28,57] but the required conditions are not verified for the example environments; otherwise A-GD is heuristic.
  • domain assumption Follower policies conditioned on the characteristics theta_hat_i can represent the optimal best responses needed for the Stackelberg equilibrium.
    The framework assumes the function class of neural network policies used by PPO can approximate the true best responses; no universal approximation argument is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ADAGE: A generic two-layer framework for adaptive agent based modelling." pith.science (2026). https://pith.science/paper/IRD3OUGM

@misc{pith2026250109429,
  author       = {Pith},
  title        = {Pith review of: ADAGE: A generic two-layer framework for adaptive agent based modelling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IRD3OUGM}},
  note         = {Machine review of arXiv:2501.09429}
}
read the original abstract

Agent-based models (ABMs) are valuable for modelling complex, potentially out-of-equilibria scenarios. However, ABMs have long suffered from the Lucas critique, stating that agent behaviour should adapt to environmental changes. Furthermore, the environment itself often adapts to these behavioural changes, creating a complex bi-level adaptation problem. Recent progress integrating multi-agent reinforcement learning into ABMs introduces adaptive agent behaviour, beginning to address the first part of this critique, however, the approaches are still relatively ad hoc, lacking a general formulation, and furthermore, do not tackle the second aspect of simultaneously adapting environmental level characteristics in addition to the agent behaviours. In this work, we develop a generic two-layer framework for ADaptive AGEnt based modelling (ADAGE) for addressing these problems. This framework formalises the bi-level problem as a Stackelberg game with conditional behavioural policies, providing a consolidated framework for adaptive agent-based modelling based on solving a coupled set of non-linear equations. We demonstrate how this generic approach encapsulates several common (previously viewed as distinct) ABM tasks, such as policy design, calibration, scenario generation, and robust behavioural learning under one unified framework. We provide example simulations on multiple complex economic and financial environments, showing the strength of the novel framework under these canonical settings, addressing long-standing critiques of traditional ABMs.

Figures

Figures reproduced from arXiv: 2501.09429 by the authors.

Figure 1
Figure 1. ADAGE: Two-layer framework. 3.1 Formulation We model the problem as a Partially Observable Markov Game (POMG) with 𝑛 + 1 agents [23] – agent 0 is reserved as the leader while the remaining 𝑛 agents are the followers2 . 𝐿 = 0 indexes the leader agent, and F = {1, . . . , 𝑛} denotes the set of follower agents, and we refer to them as the outer layer and inner simulation layer, respectively. The game can be characteris… view at source ↗
Figure 2
Figure 2. TaxAI: Economic Simulator Policy design is an important use case for modelling, for example, helping to design COVID containment strategies in epidemiology [40], optimal auction design [42], and assisting government and economic policy-making [59, 61]. As we demonstrate below, policy design naturally fits within the ADAGE framework. 4.1.1 Environment. To demonstrate the use of ADAGE for policy design, we base an env… view at source ↗
Figure 3
Figure 3. Social welfare throughout training. The solid line [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Policy Design: Resulting mean household work, savings, and wage rates throughout training. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Household inequality from multiple rollouts. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Calibration results (experimental data from [ [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Market entrances over multiple rollouts. In each [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Robust behaviour across 𝝎. The mean for the pro￾posed (baseline) is indicated by the solid line (stars), and the standard deviation by the filled region (vertical star bars) across multiple rollouts. 4.5 Key takeaways We demonstrated how ADAGE encapsulates common (prev…
Figure 9
Figure 9. Figure 9: Attendances across additional entrance capacities [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Additional calibration of processing resources plots [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

70 extracted references · 60 canonical work pages

  1. [1]

    Alekh Agarwal, Sham M Kakade, Jason D Lee, and Gaurav Mahajan. 2021. On the theory of policy gradient methods: Optimality, approximation, and distribution shift. Journal of Machine Learning Research 22, 98 (2021), 1–76

    2021

  2. [2]

    Li An, Volker Grimm, Abigail Sullivan, BL Turner Ii, Nicolas Malleson, Alison Heppenstall, Christian Vincenot, Derek Robinson, Xinyue Ye, Jianguo Liu, et al

  3. [3]

    Leo Ardon, Jared Vann, Deepeka Garg, Thomas Spooner, and Sumitra Ganesh

    work page

  4. [4]

    W Brian Arthur. 2021. Foundations of complexity economics. Nature Reviews Physics 3, 2 (2021), 136–145

    work page 2021

  5. [5]

    Robert L Axtell and J Doyne Farmer. 2022. Agent-based modeling in economics and finance: Past, present, and future. Journal of Economic Literature (2022), 1–101

    work page 2022

  6. [6]

    Christoph Aymanns, J Doyne Farmer, Alissa M Kleinnijenhuis, and Thom Wetzer

    work page

  7. [7]

    Ginestra Bianconi, Tobias Galla, Matteo Marsili, and Paolo Pin. 2009. Effects of Tobin taxes in minority game markets. Journal of Economic Behavior & Organization 70, 1-2 (2009), 231–240

    work page 2009

  8. [8]

    Gianluca Brero, Darshan Chakrabarti, Alon Eden, Matthias Gerstgrasser, Vincent Li, and David C Parkes. 2022. Learning stackelberg equilibria and applications to economic design games. (2022)

    work page 2022

Show all 70 references
  1. [9]

    Gianluca Brero, Alon Eden, Darshan Chakrabarti, Matthias Gerstgrasser, Vincent Li, and David C Parkes. 2022. Stackelberg POMDP: A Reinforcement Learning Approach for Economic Design. arXiv preprint arXiv:2210.03852 (2022)

    2022 arXiv

  2. [10]

    Ermanno Catullo, Mauro Gallegati, and Alberto Russo. 2022. Forecasting in a complex environment: Machine learning sales expectations in a Stock Flow Consistent Agent-Based simulation model. Journal of Economic Dynamics and Control 139 (2022), 104405

    2022

  3. [11]

    Liam Collins, Aryan Mokhtari, Sewoong Oh, and Sanjay Shakkottai. 2022. MAML and ANIL provably learn representations. In ICML. 4238–4310

    2022

  4. [12]

    M Ale Ebrahim Dehkordi, JM Lechner, Amineh Ghorbani, Igor Nikolic, EJL Chappin, and PM Herder. 2023. Using machine learning for agent specifications in agent-based models and simulations: A critical review and guidelines. Journal of Artificial Societies and Social Simulation 2...

    2023

  5. [13]

    Kshama Dwarakanath, Svitlana Vyetrenko, Peyman Tavallali, and Tucker Balch

  6. [14]

    Doyne Farmer, Anisoara Ca- linescu, and Michael Wooldridge

    Joel Dyer, Arnau Quera-Bofarull, Nicholas Bishop, J. Doyne Farmer, Anisoara Ca- linescu, and Michael Wooldridge. 2024. Population Synthesis as Scenario Genera- tion for Simulation-based Planning under Uncertainty. In Proceedings of the 23rd International Conference on Autonomo...

    2024

  7. [15]

    Joel Dyer, Arnau Quera-Bofarull, Ayush Chopra, J Doyne Farmer, Anisoara Calinescu, and Michael Wooldridge. 2023. Gradient-assisted calibration for financial agent-based models. In Proceedings of the Fourth ACM International Conference on AI in Finance . 288–296

    2023

  8. [16]

    Benjamin Patrick Evans and Sumitra Ganesh. 2024. Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforce- ment Learning. AAMAS (2024)

    2024

  9. [17]

    Benjamin Patrick Evans and Mikhail Prokopenko. 2024. Bounded rationality for relaxing best response and mutual consistency: the quantal hierarchy model of decision making. Theory and Decision 96, 1 (2024), 71–111

    2024

  10. [18]

    J Doyne Farmer, Paolo Patelli, and Ilija I Zovko. 2005. The predictive power of zero intelligence in financial markets. Proceedings of the National Academy of Sciences 102, 6 (2005), 2254–2259

    2005

  11. [19]

    Tanner Fiez, Benjamin Chasnov, and Lillian Ratliff. 2020. Implicit Learning Dy- namics in Stackelberg Games: Equilibria Characterization, Convergence Analysis, and Empirical Study. In Proceedings of the 37th International Conference on Ma- chine Learning (Proceedings of Machin...

    2020

  12. [20]

    Carmel Fiscko, Brian Swenson, Soummya Kar, and Bruno Sinopoli. 2019. Control of parametric games. In 2019 18th European Control Conference (ECC) . IEEE, 1036–1042

    2019

  13. [21]

    John Geanakoplos, Robert Axtell, Doyne J Farmer, Peter Howitt, Benjamin Conlee, Jonathan Goldstein, Matthew Hendrey, Nathan M Palmer, and Chun-Yi Yang

  14. [22]

    Matthias Gerstgrasser and David C Parkes. 2022. Meta-RL for Multi-Agent RL: Learning to Adapt to Evolving Agents. In Sixth Workshop on Meta-Learning at the Conference on Neural Information Processing Systems

    2022

  15. [23]

    Matthias Gerstgrasser and David C Parkes. 2023. Oracles & followers: Stackelberg equilibria in deep multi-agent reinforcement learning. InInternational Conference on Machine Learning. PMLR, 11213–11236

    2023

  16. [24]

    Jonathan Heathcote, Kjetil Storesletten, and Giovanni L Violante. 2017. Optimal tax progressivity: An analytical framework. The Quarterly Journal of Economics 132, 4 (2017), 1693–1754

    2017

  17. [25]

    Arthur Hjorth, Bryan Head, Corey Brady, and Uri Wilensky. 2020. Levelspace: A netlogo extension for multi-level agent-based modeling. Journal of Artificial Societies and Social Simulation 23, 1 (2020)

    2020

  18. [26]

    Cars Hommes, Mario He, Sebastian Poledna, Melissa Siqueira, and Yang Zhang

  19. [27]

    Cars Hommes, Joep Sonnemans, Jan Tuinstra, and Henk Van De Velden. 2007. Learning in cobweb experiments. Macroeconomic Dynamics 11, S1 (2007), 8–33

    2007

  20. [28]

    Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. 2023. A two- timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic. SIAM Journal on Optimization 33, 1 (2023), 147–180

    2023

  21. [29]

    Franziska Klügl and Hildegunn Kyvik Nordås. 2023. Modelling Agent Decision Making in Agent-based Simulation-Analysis Using an Economic Technology Uptake Model. In 22nd International Conference on Autonomous Agents and Mul- tiagent Systems (AAMAS 2023), London, United Kingdom, ...

    2023

  22. [30]

    Dan Ladley. 2012. Zero intelligence in economics and finance. The Knowledge Engineering Review 27, 2 (2012), 273–286

    2012

  23. [31]

    Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Gold- berg, Joseph Gonzalez, Michael Jordan, and Ion Stoica. 2018. RLlib: Abstractions for distributed reinforcement learning. In International conference on machine learning. PMLR, 3053–3062

    2018

  24. [32]

    Robert E Lucas Jr. 1976. Econometric policy evaluation: A critique. In Carnegie- Rochester conference series on public policy , Vol. 1. North-Holland, 19–46

    1976

  25. [33]

    Qirui Mi, Siyu Xia, Yan Song, Haifeng Zhang, Shenghao Zhu, and Jun Wang

  26. [34]

    Mauro Napoletano. 2018. A short walk on the wild side: Agent-based models and their implications for macroeconomic analysis. Revue de l’OFCE 3 (2018), 257–281

    2018

  27. [35]

    Fernando Nogueira. 2014–. Bayesian Optimization: Open source constrained global optimization tool for Python. https://github.com/bayesian-optimization/ BayesianOptimization

    2014

  28. [36]

    Pedro A Ortega and Daniel A Braun. 2013. Thermodynamics as a theory of decision-making with information-processing costs. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 469, 2153 (2013), 20120683

    2013

  29. [37]

    Benjamin Patrick Evans and Mikhail Prokopenko. 2023. Bounded strategic reasoning explains crisis emergence in multi-agent market games. Royal Society Open Science 10, 2 (2023), 221164

    2023

  30. [38]

    Donovan Platt. 2020. A comparison of economic agent-based model calibration methods. Journal of Economic Dynamics and Control 113 (2020), 103859

    2020

  31. [39]

    Pisit Praiwattana and Abdennour El Rhalibi. 2016. Survey: Development and analysis of a games-based crisis scenario generation system. In E-Learning and Games: 10th International Conference, Edutainment 2016, Hangzhou, China, April 14-16, 2016, Revised Selected Papers 10 . Spr...

    2016

  32. [40]

    In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (Auckland, New Zealand) (AAMAS ’24)

    TaxAI: A Dynamic Economic Simulator and Benchmark for Multi-agent Reinforcement Learning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (Auckland, New Zealand) (AAMAS ’24). International Foundation for Autonomous Agents and Mul...

  33. [41]

    Isabelle L Salle. 2015. Modeling expectations in agent-based models—An applica- tion to central bank’s communication and monetary policy. Economic Modelling 46 (2015), 130–141

    2015

  34. [42]

    Tuomas Sandholm. 2003. Automated mechanism design: A new application area for search algorithms. In International Conference on Principles and Practice of Constraint Programming. Springer, 19–36

    2003

  35. [43]

    Dipyaman Sanyal. 2019. Effect of Tobin Tax on Trading Decisions in an Ex- perimental Minority Game . Springer International Publishing, Cham, 71–79. https://doi.org/10.1007/978-3-030-11364-3_6

    2019 doi

  36. [44]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  37. [45]

    Egemen Sert, Yaneer Bar-Yam, and Alfredo J Morales. 2020. Segregation dynamics with reinforcement learning and agent based modeling. Scientific reports 10, 1 (2020), 11771

    2020

  38. [46]

    Vinzenz Thoma, Barna Pasztor, Andreas Krause, Giorgia Ramponi, and Yifan Hu. 2024. Stochastic Bilevel Optimization with Lower-Level Contextual Markov Decision Processes. arXiv preprint arXiv:2406.01575 (2024)

    2024 arXiv

  39. [47]

    Arnau Quera-Bofarull, Ayush Chopra, Joseph Aylett-Bullock, Carolina Cuesta- Lazaro, Anisoara Calinescu, Ramesh Raskar, and Michael Wooldridge. 2023. Don’t Simulate Twice: One-Shot Sensitivity Analyses via Automatic Differentiation. In Proceedings of the 2023 International Conf...

    2023

  40. [48]

    James Tobin. 1978. A proposal for international monetary reform. Eastern economic journal 4, 3/4 (1978), 153–159

    1978

  41. [49]

    Alexander Trott, Sunil Srinivasa, Douwe van der Wal, Sebastien Haneuse, and Stephan Zheng. 2021. Building a foundation for data-driven, interpretable, and robust policy design using the ai economist. arXiv preprint arXiv:2108.02904 (2021)

    2021 arXiv

  42. [50]

    Arthur Turrell. 2016. Agent-based models: understanding the economy from the bottom up. Bank of England Quarterly Bulletin (2016), Q4

    2016

  43. [51]

    Nelson Vadori, Leo Ardon, Sumitra Ganesh, Thomas Spooner, Selim Amrouni, Jared Vann, Mengda Xu, Zeyu Zheng, Tucker Balch, and Manuela Veloso. 2023. Towards multi-agent reinforcement learning-driven over-the-counter market simulations. Mathematical Finance (2023)

    2023

  44. [52]

    Nelson Vadori, Sumitra Ganesh, Prashant Reddy, and Manuela Veloso. 2020. Calibration of shared equilibria in general sum partially observable Markov games. NeurIPS 33 (2020), 14118–14128

    2020

  45. [53]

    Víctor A Vargas-Pérez, Pablo Mesejo, Manuel Chica, and Oscar Cordón. 2023. Deep reinforcement learning in agent-based simulations for optimal media plan- ning. Information Fusion 91 (2023), 644–664

    2023

  46. [54]

    Xiaohan Wang, Lin Zhang, Yuanjun Laili, Kunyu Xie, Han Lu, and Chun Zhao

  47. [55]

    Callum Rhys Tilbury. 2022. Reinforcement Learning for Economic Policy: A New Frontier? arXiv preprint arXiv:2206.08781 (2022)

    2022 arXiv

  48. [56]

    Sihan Zeng and Thinh Doan. 2024. Fast two-time-scale stochastic gradient method with applications in reinforcement learning. InThe Thirty Seventh Annual Conference on Learning Theory . PMLR, 5166–5212

    2024

  49. [57]

    Sihan Zeng, Thinh T Doan, and Justin Romberg. 2024. A two-time-scale stochastic optimization framework with applications in control and reinforcement learning. SIAM Journal on Optimization 34, 1 (2024), 946–976

    2024

  50. [58]

    Brian Zhang, Gabriele Farina, Ioannis Anagnostides, Federico Cacciamani, Stephen McAleer, Andreas Haupt, Andrea Celli, Nicola Gatti, Vincent Conitzer, and Tuomas Sandholm. 2024. Computing optimal equilibria and mechanisms via learning in zero-sum extensive-form games. Advances...

    2024

  51. [59]

    Parkes, Milind Tambe, and Yiling Chen

    Edwin Zhang, Sadie Zhao, Tonghan Wang, Safwan Hossain, Henry Gasztowtt, Stephan Zheng, David C. Parkes, Milind Tambe, and Yiling Chen. 2024. Position: Social Environment Design Should be Further Developed for AI-based Policy- Making. (2024)

    2024

  52. [60]

    Liyuan Zheng, Tanner Fiez, Zane Alumbaugh, Benjamin Chasnov, and Lillian J Ratliff. 2022. Stackelberg actor-critic: Game-theoretic reinforcement learning algorithms. In Proceedings of the AAAI conference on artificial intelligence , Vol. 36. 9217–9224

    2022

  53. [61]

    Stephan Zheng, Alexander Trott, Sunil Srinivasa, David C Parkes, and Richard Socher. 2022. The AI Economist: Taxation policy design via two-level deep multiagent reinforcement learning. Science Advances 8, 18 (2022), eabk2607. SUPPLEMENTARY MATERIAL A REINFORCEMENT LEARNING CO...

    2022

  54. [63]

    In 2021 Annual Modeling and Simula- tion Conference (ANNSIM)

    MADES: A Unified Framework for Integrating Agent-Based Simulation with Multi-Agent Reinforcement Learning. In 2021 Annual Modeling and Simula- tion Conference (ANNSIM). 1–12. https://doi.org/10.23919/ANNSIM52504.2021. 9552052

    2021

  55. [64]

    Mason Wright and Michael P Wellman. 2018. Evaluating the stability of non- adaptive trading in continuous double auctions. In 17th International Conference on Autonomous Agents and Multiagent Systems

    2018

  56. [2012]

    American Economic Review 102, 3 (2012), 53–58

    Getting at systemic risk via an agent-based model of the housing market. American Economic Review 102, 3 (2012), 53–58

    2012

  57. [2017]

    arXiv preprint arXiv:1707.06347 (2017)

    Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  58. [2018]

    In Handbook of computational economics

    Models of financial stability and their application in stress tests. In Handbook of computational economics . Vol. 4. Elsevier, 329–391

  59. [2021]

    Ecological Modelling 457 (2021), 109685

    Challenges, tasks, and opportunities in modeling agent-based complex systems. Ecological Modelling 457 (2021), 109685

    2021

  60. [2022]

    Technical Report

    Canvas: A canadian behavioral agent-based model . Technical Report. Bank of Canada

  61. [2023]

    In AAMAS

    Phantom-A RL-driven Multi-Agent Framework to Model Complex Systems. In AAMAS. 2742–2744

  62. [2024]

    arXiv preprint arXiv:2402.09563 (2024)

    ABIDES-Economist: Agent-Based Simulation of Economic Systems with Learning Agents. arXiv preprint arXiv:2402.09563 (2024)

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.