Recognition: unknown
Multi-Agent Digital Twins for Strategic Decision-Making using Active Inference
Pith reviewed 2026-05-10 14:22 UTC · model grok-4.3
The pith
Active inference lets decentralized agents coordinate in multi-agent digital twins for strategic decisions.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Agents can interact and coordinate inside a shared digital twin while each maintains its own generative model; contextual inference improves responsiveness to changing conditions and streaming machine learning inside the models permits adjustable goal-directed behavior, all while keeping computational cost low, as shown by stable outcomes in the Cournot competition illustration.
What carries the argument
Contextual inference together with streaming machine learning embedded in each agent's decentralized generative model under active inference.
If this is right
- Agents gain better adaptability to dynamic environments through contextual inference.
- Streaming machine learning inside the models allows goal-oriented behavior to be adjusted on the fly.
- The overall system remains efficient and scalable for larger numbers of agents.
- Coordinated strategic choices emerge in socio-economic settings represented by market competition examples.
Where Pith is reading between the lines
- The same structure could be tried in domains such as traffic management or robotic swarms where agents share space but must act locally.
- A direct test would compare coordination quality against centralized planning when communication between agents is limited or delayed.
- The approach hints that active inference models might handle strategic interactions in place of classical game-theoretic assumptions about perfect rationality.
Load-bearing premise
Decentralized generative models kept by separate agents can still produce effective coordination in a shared environment without extra explicit communication rules.
What would settle it
Running the Cournot competition simulation with the framework and finding that agents fail to reach stable prices or outputs, or that performance collapses when market conditions change abruptly.
Figures
read the original abstract
Active Inference is an emerging framework providing a quantitative account of behavioral processes in neuroscience and a principled approach to decision-making under uncertainty. Its application to agency problems is natural, offering an autopoietic interpretation of action while addressing classical challenges such as the exploration-exploitation trade-off. Recently, Active Inference has been applied to digital twin scenarios for adaptive and predictive modeling of complex systems. In this work, we extend Active Inference to multi-agent digital twins in which agents interact within a shared environment while maintaining decentralized generative models. Our multi-agent framework features two innovations: (i) contextual inference to improve adaptability in dynamic environments, and (ii) the integration of streaming machine learning within agents' generative structures, enabling tunable goal-oriented behavior while preserving efficiency and scalability. The framework is illustrated through a Cournot competition example, providing a digital twin representation of a socio-economic system and highlighting its potential for coordinated decision-making in multi-agent contexts.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript extends Active Inference to multi-agent digital twins for strategic decision-making. Agents maintain decentralized generative models in a shared environment and incorporate two claimed innovations: contextual inference for improved adaptability and streaming machine learning embedded in each agent's generative structure to support tunable goal-oriented behavior. The framework is illustrated via a Cournot competition example representing a socio-economic system.
Significance. If the coordination mechanism and innovations are rigorously formalized and validated, the work could contribute a scalable, uncertainty-aware approach to multi-agent digital twins grounded in an established framework. The emphasis on decentralization and streaming updates addresses practical constraints in socio-economic modeling, but the absence of quantitative validation or explicit cross-agent inference rules limits assessment of whether the approach advances beyond existing active-inference multi-agent extensions.
major comments (2)
- [Framework and Cournot example] The central claim that decentralized generative models enable effective coordination without explicit communication protocols or cross-agent inference is load-bearing yet unsupported. No equations, belief-update rules, or factorization of the joint generative model p(o,s,a,other agents) are supplied showing how one agent's variational free-energy minimization accounts for competitors' latent states or policies (see the multi-agent framework description and Cournot illustration).
- [Innovations (i) and (ii)] The two innovations are stated at a high level but lack formalization. No derivation or pseudocode is given for contextual inference (how context modulates the generative model or free-energy gradients) or for embedding streaming ML updates inside the agent's generative structure while preserving the active-inference objective (see sections introducing the innovations).
minor comments (2)
- [Abstract and Introduction] The abstract and introduction reference 'autopoietic interpretation' and 'exploration-exploitation trade-off' but do not explicitly link these concepts to the multi-agent or digital-twin extensions.
- [Notation and Framework] Notation for agent-specific generative models, contextual variables, and streaming updates should be introduced consistently with standard active-inference symbols (e.g., Q, F, G) to aid readability.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed report. The comments highlight important areas where additional rigor will strengthen the manuscript. We respond to each major comment below and indicate the revisions we will make.
read point-by-point responses
-
Referee: [Framework and Cournot example] The central claim that decentralized generative models enable effective coordination without explicit communication protocols or cross-agent inference is load-bearing yet unsupported. No equations, belief-update rules, or factorization of the joint generative model p(o,s,a,other agents) are supplied showing how one agent's variational free-energy minimization accounts for competitors' latent states or policies (see the multi-agent framework description and Cournot illustration).
Authors: We agree that the multi-agent framework section and Cournot illustration present the coordination mechanism at a conceptual level without supplying the explicit joint generative model factorization or the precise variational update rules that would show how each agent’s free-energy minimization incorporates an approximation of other agents’ latent states. We will add a new subsection that (i) states the factorization of the joint distribution over shared observations, individual states, actions, and other agents’ policies, (ii) derives the mean-field approximation used by each agent, and (iii) gives the resulting belief-update equations. These additions will be placed immediately after the current high-level framework description and will be cross-referenced in the Cournot example. revision: yes
-
Referee: [Innovations (i) and (ii)] The two innovations are stated at a high level but lack formalization. No derivation or pseudocode is given for contextual inference (how context modulates the generative model or free-energy gradients) or for embedding streaming ML updates inside the agent's generative structure while preserving the active-inference objective (see sections introducing the innovations).
Authors: We accept that both innovations are introduced conceptually rather than with the requested derivations. We will revise the relevant sections to include: (a) a mathematical specification of contextual inference, showing how context variables augment the generative model priors and likelihoods and how they enter the gradients of the variational free energy; (b) an algorithmic description (in pseudocode) of the streaming-ML update step embedded inside the active-inference loop, together with a short proof sketch that the overall objective remains the minimization of expected free energy. These formal elements will be added without altering the existing narrative flow. revision: yes
Circularity Check
No circularity: extension of established active-inference framework with independent illustrative example
full rationale
The paper extends the pre-existing active inference formalism to multi-agent digital twins by adding contextual inference and streaming ML components inside per-agent generative models. The Cournot competition is presented purely as an illustration of coordinated decision-making under decentralized models, without any claim that the coordination outcome is derived from or fitted to quantities defined inside the paper itself. No equations reduce a prediction to a fitted parameter, no uniqueness theorem is imported from self-citation, and no ansatz is smuggled via prior work by the same authors. The derivation chain therefore remains self-contained against the external active-inference literature.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Active inference provides a quantitative account of behavioral processes in neuroscience and a principled approach to decision-making under uncertainty.
Reference graph
Works this paper leans on
-
[1]
T. Parr, G. Pezzulo, K. J. Friston, Active Inference: The Free Energy Principle in Mind, Brain, and Behavior, MIT Press, Cambridge, MA, 2022.doi:10.7551/mitpress/12441.001.0001
-
[2]
R. A. Adams, K. E. Stephan, H. R. Brown, C. D. Frith, K. J. Friston, The computational anatomy of psychosis, Frontiers in Psychiatry 4 (2013) 47.doi:10.3389/fpsyt.2013.00047
-
[3]
M. Cullen, B. Davey, K. J. Friston, R. J. Moran, Active Inference in OpenAI Gym: A Paradigm for Computational Investigations Into Psychiatric Illness, Biological Psychiatry: Cognitive Neuroscience and Neuroimaging 3 (9) (2018) 809–818.doi:10.1016/j.bpsc.2018.06.010
-
[4]
C. Heins, B. Millidge, L. Da Costa, R. P. Mann, K. J. Friston, I. D. Couzin, Collective behavior from surprise minimization, Proceedings of the National Academy of Sciences 121 (17) (2024).doi: 10.1073/pnas.2320239121
-
[5]
Quantum processor-inspired machine learning in the biomedical sciences
K. Friston, C. Frith, A duet for one, Consciousness and Cognition 36 (2015) 390–405.doi:10.1016/j. concog.2014.12.003
work page doi:10.1016/j 2015
- [6]
-
[7]
G. Levchuk, K. Pattipati, D. Serfaty, A. Fouse, R. McCormack, Active Inference in Multiagent Sys- tems: Context-Driven Collaboration and Decentralized Purpose-Driven Team Adaptation, in: W. Law- less, R. Mittu, D. Sofge, I. S. Moskowitz, S. Russell (Eds.), Artificial Intelligence for the Internet of Everything, Academic Press, 2019, pp. 67–85.doi:10.1016/...
-
[8]
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
L. Beckenbauer, J.-L. Loewe, G. Zheng, A. Brintrup, Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Tasks, arXiv preprint arXiv:2509.05651v1 (2025).doi:10.48550/arXiv. 2509.05651. 21
work page internal anchor Pith review doi:10.48550/arxiv 2025
-
[9]
R. Laubenbacher, B. Mehrad, I. Shmulevich, N. Trayanova, Digital twins in medicine, Nature Compu- tational Science 4 (3) (2024) 184–191.doi:10.1038/s43588-024-00607-6
-
[10]
D. M. Botín-Sanabria, A.-S. Mihaita, R. E. Peimbert-García, M. A. Ramírez-Moreno, R. A. Ramírez- Mendoza, J. d. J. Lozoya-Santos, Digital Twin Technology Challenges and Applications: A Comprehen- sive Review, Remote Sensing 14 (6) (2022) 1335.doi:10.3390/rs14061335
-
[11]
G. White, A. Zink, L. Codecá, S. Clarke, A digital twin smart city for citizen feedback, Cities 110 (2021) 103064.doi:10.1016/j.cities.2020.103064
-
[12]
M. Torzoni, D. Maisto, A. Manzoni, F. Donnarumma, G. Pezzulo, A. Corigliano, Active digital twins via active inference, Engineering Applications of Artificial Intelligence 174 (2026) 114519.doi:10.1016/ j.engappai.2026.114519
-
[13]
L. Pio-Lopez, F. Kuchling, A. Tung, G. Pezzulo, M. Levin, Active inference, morphogenesis, and compu- tational psychiatry, Frontiers in Computational Neuroscience 16 (2022) 988977.doi:10.3389/fncom. 2022.988977
-
[14]
K. Friston, S. Samothrakis, R. Montague, Active inference and agency: optimal control without cost functions, Biological Cybernetics 106 (8) (2012) 523–541.doi:10.1007/s00422-012-0512-8
-
[15]
J. Gama, I. Žliobait˙ e, A. Bifet, M. Pechenizkiy, A. Bouchachia, A survey on concept drift adaptation, ACM computing surveys (CSUR) 46 (4) (2014) 1–37.doi:10.1145/2523813
-
[16]
J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, G. Zhang, Learning under Concept Drift: A Review, IEEE transactions on knowledge and data engineering 31 (12) (2019) 2346–2363.doi:10.1109/TKDE.2018. 2876857
-
[17]
G. Pezzulo, F. Rigoli, K. J. Friston, Hierarchical Active Inference: A Theory of Motivated Control, Trends in Cognitive Sciences 22 (4) (2018) 294–306.doi:10.1016/j.tics.2018.01.009
-
[18]
N. Sajid, P. Tigas, K. Friston, Active inference, preference learning and adaptive behaviour, in: IOP Conference Series: Materials Science and Engineering: International Workshop on Embodied Intelli- gence, Vol. 1261, 2022, p. 012020.doi:10.1088/1757-899X/1261/1/012020
-
[19]
K. Friston, T. FitzGerald, F. Rigoli, P. Schwartenbeck, G. Pezzulo, Active Inference: A Process Theory, Neural Computation 29 (1) (2017) 1–49.doi:10.1162/NECO\_a\_00912
-
[20]
T. Parr, K. J. Friston, Generalised free energy and active inference, Biological Cybernetics 113 (5) (2019) 495–513.doi:10.1007/s00422-019-00805-w
-
[21]
A. Tschantz, B. Millidge, A. K. Seth, C. L. Buckley, Reinforcement learning through active inference, arXiv preprint arXiv:2002.12636v1 (2020).doi:10.48550/arXiv.2002.12636
-
[22]
J. Ruiz-Serra, P. Sweeney, M. S. Harré, Factorised Active Inference for Strategic Multi-Agent Interac- tions, arXiv preprint arXiv:2411.07362v2 (2025).doi:10.48550/arXiv.2411.07362
-
[23]
D. Maisto, F. Donnarumma, G. Pezzulo, Interactive Inference: A Multi-Agent Model of Cooperative Joint Actions, IEEE Transactions on Systems, Man, and Cybernetics: Systems 54 (2) (2023) 704–715. doi:10.1109/tsmc.2023.3312585
-
[24]
H. Leibenstein, Bandwagon, snob, and veblen effects in the theory of consumers’ demand, The Quarterly Journal of Economics 64 (2) (1950) 183–207.doi:10.2307/1882692
-
[25]
K. Friston, M. Levin, B. Sengupta, G. Pezzulo, Knowing one’s place: a free-energy approach to pattern regulation, Journal of the Royal Society Interface 12 (105) (2015) 20141383.doi:10.1098/rsif.2014. 1383
-
[26]
C. Heins, B. Millidge, D. Demekas, B. Klein, K. Friston, I. D. Couzin, A. Tschantz, pymdp: A Python library for active inference in discrete state spaces, Journal of Open Source Software 7 (73) (2022) 4098. doi:10.21105/joss.04098. 22
-
[27]
K. P. Murphy, Probabilistic Machine Learning: Advanced Topics, MIT Press, 2023
2023
-
[28]
M. J. Wainwright, M. I. Jordan, Graphical Models, Exponential Families, and Variational Inference, Now Publishers Inc., Hanover, MA, 2008.doi:10.1561/2200000001
-
[29]
Fudenberg, J
D. Fudenberg, J. Tirole, Game Theory, MIT press, 1991
1991
-
[30]
Friedman, J
D. Friedman, J. Yellin, Evolving landscapes for population games, unpublished (1997)
1997
- [31]
-
[32]
S. Bindra, D. Sharma, N. Parameswar, S. Dhir, J. Paul, Bandwagon effect revisited: A systematic review to develop future research agenda, Journal of Business Research 143 (2022) 305–317.doi: 10.1016/j.jbusres.2022.01.085
-
[33]
Journal of The Royal Society Interface , year =
K. Friston, Life as we know it, Journal of The Royal Society Interface 10 (86) (2013) 20130475.doi: 10.1098/rsif.2013.0475
-
[34]
P. Schwartenbeck, J. Passecker, T. U. Hauser, T. H. FitzGerald, M. Kronbichler, K. J. Friston, Computational mechanisms of curiosity and goal-directed exploration, eLife 8 (2019) e41703.doi: 10.7554/eLife.41703
-
[35]
R. Proietti, T. Parr, A. Tessari, K. Friston, G. Pezzulo, Active inference and cognitive control: Balancing deliberation and habits through precision optimization, Physics of Life Reviews 54 (2025) 27–51.doi: 10.1016/j.plrev.2025.05.008
-
[36]
T. K. Ho, The random subspace method for constructing decision forests, IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (8) (1998) 832–844.doi:10.1109/34.709601
-
[37]
A. Bifet, G. Holmes, B. Pfahringer, Leveraging Bagging for Evolving Data Streams, in: Machine Learn- ing and Knowledge Discovery in Databases, Springer Berlin Heidelberg, Berlin, Heidelberg, 2010, pp. 135–150.doi:10.1007/978-3-642-15880-3\_15
-
[38]
G. Louppe, Understanding random forests: From theory to practice, Ph.D. thesis, Universite de Liege (2014).doi:10.13140/2.1.1570.5928. Appendix A. Likelihood derivation for multi-state modalities This appendix details the derivation of a likelihood matrix that accounts for the two components p(warehouse signal|warehouse)andp(warehouse signal|context). Let...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.