pith. machine review for the scientific record. sign in

arxiv: 2604.12657 · v1 · submitted 2026-04-14 · 💻 cs.CE

Recognition: unknown

Multi-Agent Digital Twins for Strategic Decision-Making using Active Inference

Authors on Pith no claims yet

Pith reviewed 2026-05-10 14:22 UTC · model grok-4.3

classification 💻 cs.CE
keywords active inferencemulti-agent systemsdigital twinscontextual inferencestreaming machine learningstrategic decision-makingCournot competitiondecentralized models
0
0 comments X

The pith

Active inference lets decentralized agents coordinate in multi-agent digital twins for strategic decisions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper extends the active inference framework from single agents to multiple agents that share an environment but keep separate generative models. Two additions are contextual inference, which helps agents adapt when conditions shift, and the embedding of streaming machine learning inside each agent's model, which lets behavior be tuned toward goals without losing speed or scalability. The approach is tested in a Cournot market example that stands in for socio-economic competition. A reader would care because the work offers a concrete way to build digital twins that support coordinated choices under uncertainty while avoiding a central controller.

Core claim

Agents can interact and coordinate inside a shared digital twin while each maintains its own generative model; contextual inference improves responsiveness to changing conditions and streaming machine learning inside the models permits adjustable goal-directed behavior, all while keeping computational cost low, as shown by stable outcomes in the Cournot competition illustration.

What carries the argument

Contextual inference together with streaming machine learning embedded in each agent's decentralized generative model under active inference.

If this is right

  • Agents gain better adaptability to dynamic environments through contextual inference.
  • Streaming machine learning inside the models allows goal-oriented behavior to be adjusted on the fly.
  • The overall system remains efficient and scalable for larger numbers of agents.
  • Coordinated strategic choices emerge in socio-economic settings represented by market competition examples.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same structure could be tried in domains such as traffic management or robotic swarms where agents share space but must act locally.
  • A direct test would compare coordination quality against centralized planning when communication between agents is limited or delayed.
  • The approach hints that active inference models might handle strategic interactions in place of classical game-theoretic assumptions about perfect rationality.

Load-bearing premise

Decentralized generative models kept by separate agents can still produce effective coordination in a shared environment without extra explicit communication rules.

What would settle it

Running the Cournot competition simulation with the framework and finding that agents fail to reach stable prices or outputs, or that performance collapses when market conditions change abruptly.

Figures

Figures reproduced from arXiv: 2604.12657 by Alberto Corigliano, Andrea Manzoni, Domenico Maisto, Francesco Donnarumma, Francesco Maria Mancinelli, Giovanni Pezzulo, Matteo Torzoni.

Figure 1
Figure 1. Figure 1: Generative process (GP) and generative model (GM) in the proposed multi-agent Figure 1: Generative process (GP) and generative model (GM) in the proposed multi-agent framework. The GP represents the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Dynamic Bayesian network encoding the relations among the real-world physical space ynamic Bayesian network encoding the relations among the realworld physical space and its digital rep es represent random variablessquare nodes denote taken actionsdiamond-shaped nodes symbolize t [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Dynamic Bayesian network illustrating the use of AIF generative models to navigate the par Figure 3: Dynamic bayesian network for the active inference generative model underlying the digital twin problem. Circular [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 4
Figure 4. Figure 4: Sales likelihood matrices across production contexts: (a) acceptable production context; (b) reduced production [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Production likelihood matrices across production contexts: (a) acceptable production context; (b) reduced production [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Context-sensitive likelihood p(warehouse signal | warehouse) under noisy conditions: (a) acceptable production con￾text; (b) reduce production context [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Example of warehouse transition assuming a one-step best response of 5 units. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Behavior of firms 1 and 2 over time. Bars show the number of units produced at each time step (darker bars) and [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Real and predicted price over time. The black line represents the actual market price, while the cyan and red lines [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison of sales likelihood matrices when [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Cournot duopoly simulation with firm 2 exhibiting higher precision (reduced variance) in the sales observational [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Cournot duopoly simulation with firm 2 exhibiting higher precision (reduced variance) in the sales observational [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Cournot three-firm extension simulation with well-designed generative models. [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Cournot three-firm extension simulation with firm 2 exhibiting higher precision (reduced variance) in the sales [PITH_FULL_IMAGE:figures/full_fig_p020_14.png] view at source ↗
read the original abstract

Active Inference is an emerging framework providing a quantitative account of behavioral processes in neuroscience and a principled approach to decision-making under uncertainty. Its application to agency problems is natural, offering an autopoietic interpretation of action while addressing classical challenges such as the exploration-exploitation trade-off. Recently, Active Inference has been applied to digital twin scenarios for adaptive and predictive modeling of complex systems. In this work, we extend Active Inference to multi-agent digital twins in which agents interact within a shared environment while maintaining decentralized generative models. Our multi-agent framework features two innovations: (i) contextual inference to improve adaptability in dynamic environments, and (ii) the integration of streaming machine learning within agents' generative structures, enabling tunable goal-oriented behavior while preserving efficiency and scalability. The framework is illustrated through a Cournot competition example, providing a digital twin representation of a socio-economic system and highlighting its potential for coordinated decision-making in multi-agent contexts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript extends Active Inference to multi-agent digital twins for strategic decision-making. Agents maintain decentralized generative models in a shared environment and incorporate two claimed innovations: contextual inference for improved adaptability and streaming machine learning embedded in each agent's generative structure to support tunable goal-oriented behavior. The framework is illustrated via a Cournot competition example representing a socio-economic system.

Significance. If the coordination mechanism and innovations are rigorously formalized and validated, the work could contribute a scalable, uncertainty-aware approach to multi-agent digital twins grounded in an established framework. The emphasis on decentralization and streaming updates addresses practical constraints in socio-economic modeling, but the absence of quantitative validation or explicit cross-agent inference rules limits assessment of whether the approach advances beyond existing active-inference multi-agent extensions.

major comments (2)
  1. [Framework and Cournot example] The central claim that decentralized generative models enable effective coordination without explicit communication protocols or cross-agent inference is load-bearing yet unsupported. No equations, belief-update rules, or factorization of the joint generative model p(o,s,a,other agents) are supplied showing how one agent's variational free-energy minimization accounts for competitors' latent states or policies (see the multi-agent framework description and Cournot illustration).
  2. [Innovations (i) and (ii)] The two innovations are stated at a high level but lack formalization. No derivation or pseudocode is given for contextual inference (how context modulates the generative model or free-energy gradients) or for embedding streaming ML updates inside the agent's generative structure while preserving the active-inference objective (see sections introducing the innovations).
minor comments (2)
  1. [Abstract and Introduction] The abstract and introduction reference 'autopoietic interpretation' and 'exploration-exploitation trade-off' but do not explicitly link these concepts to the multi-agent or digital-twin extensions.
  2. [Notation and Framework] Notation for agent-specific generative models, contextual variables, and streaming updates should be introduced consistently with standard active-inference symbols (e.g., Q, F, G) to aid readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed report. The comments highlight important areas where additional rigor will strengthen the manuscript. We respond to each major comment below and indicate the revisions we will make.

read point-by-point responses
  1. Referee: [Framework and Cournot example] The central claim that decentralized generative models enable effective coordination without explicit communication protocols or cross-agent inference is load-bearing yet unsupported. No equations, belief-update rules, or factorization of the joint generative model p(o,s,a,other agents) are supplied showing how one agent's variational free-energy minimization accounts for competitors' latent states or policies (see the multi-agent framework description and Cournot illustration).

    Authors: We agree that the multi-agent framework section and Cournot illustration present the coordination mechanism at a conceptual level without supplying the explicit joint generative model factorization or the precise variational update rules that would show how each agent’s free-energy minimization incorporates an approximation of other agents’ latent states. We will add a new subsection that (i) states the factorization of the joint distribution over shared observations, individual states, actions, and other agents’ policies, (ii) derives the mean-field approximation used by each agent, and (iii) gives the resulting belief-update equations. These additions will be placed immediately after the current high-level framework description and will be cross-referenced in the Cournot example. revision: yes

  2. Referee: [Innovations (i) and (ii)] The two innovations are stated at a high level but lack formalization. No derivation or pseudocode is given for contextual inference (how context modulates the generative model or free-energy gradients) or for embedding streaming ML updates inside the agent's generative structure while preserving the active-inference objective (see sections introducing the innovations).

    Authors: We accept that both innovations are introduced conceptually rather than with the requested derivations. We will revise the relevant sections to include: (a) a mathematical specification of contextual inference, showing how context variables augment the generative model priors and likelihoods and how they enter the gradients of the variational free energy; (b) an algorithmic description (in pseudocode) of the streaming-ML update step embedded inside the active-inference loop, together with a short proof sketch that the overall objective remains the minimization of expected free energy. These formal elements will be added without altering the existing narrative flow. revision: yes

Circularity Check

0 steps flagged

No circularity: extension of established active-inference framework with independent illustrative example

full rationale

The paper extends the pre-existing active inference formalism to multi-agent digital twins by adding contextual inference and streaming ML components inside per-agent generative models. The Cournot competition is presented purely as an illustration of coordinated decision-making under decentralized models, without any claim that the coordination outcome is derived from or fitted to quantities defined inside the paper itself. No equations reduce a prediction to a fitted parameter, no uniqueness theorem is imported from self-citation, and no ansatz is smuggled via prior work by the same authors. The derivation chain therefore remains self-contained against the external active-inference literature.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work rests on the validity of the active-inference framework for single agents and assumes this framework extends naturally to decentralized multi-agent interaction without new axioms being introduced in the abstract.

axioms (1)
  • domain assumption Active inference provides a quantitative account of behavioral processes in neuroscience and a principled approach to decision-making under uncertainty.
    Stated directly in the opening sentence of the abstract as the foundational framework being extended.

pith-pipeline@v0.9.0 · 5472 in / 1180 out tokens · 48355 ms · 2026-05-10T14:22:16.420644+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 35 canonical work pages · 1 internal anchor

  1. [1]

    T. Parr, G. Pezzulo, K. J. Friston, Active Inference: The Free Energy Principle in Mind, Brain, and Behavior, MIT Press, Cambridge, MA, 2022.doi:10.7551/mitpress/12441.001.0001

  2. [2]

    R. A. Adams, K. E. Stephan, H. R. Brown, C. D. Frith, K. J. Friston, The computational anatomy of psychosis, Frontiers in Psychiatry 4 (2013) 47.doi:10.3389/fpsyt.2013.00047

  3. [3]

    Cullen, B

    M. Cullen, B. Davey, K. J. Friston, R. J. Moran, Active Inference in OpenAI Gym: A Paradigm for Computational Investigations Into Psychiatric Illness, Biological Psychiatry: Cognitive Neuroscience and Neuroimaging 3 (9) (2018) 809–818.doi:10.1016/j.bpsc.2018.06.010

  4. [4]

    Heins, B

    C. Heins, B. Millidge, L. Da Costa, R. P. Mann, K. J. Friston, I. D. Couzin, Collective behavior from surprise minimization, Proceedings of the National Academy of Sciences 121 (17) (2024).doi: 10.1073/pnas.2320239121

  5. [5]

    Quantum processor-inspired machine learning in the biomedical sciences

    K. Friston, C. Frith, A duet for one, Consciousness and Cognition 36 (2015) 390–405.doi:10.1016/j. concog.2014.12.003

  6. [6]

    Maisto, D

    D. Maisto, D. Nuzzi, G. Pezzulo, What the flock knows that the birds do not: exploring the emergence of joint agency in multi-agent active inference, arXiv preprint arXiv:2511.10835v2 (2026).doi:10. 48550/arXiv.2511.10835

  7. [7]

    Levchuk, K

    G. Levchuk, K. Pattipati, D. Serfaty, A. Fouse, R. McCormack, Active Inference in Multiagent Sys- tems: Context-Driven Collaboration and Decentralized Purpose-Driven Team Adaptation, in: W. Law- less, R. Mittu, D. Sofge, I. S. Moskowitz, S. Russell (Eds.), Artificial Intelligence for the Internet of Everything, Academic Press, 2019, pp. 67–85.doi:10.1016/...

  8. [8]

    Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

    L. Beckenbauer, J.-L. Loewe, G. Zheng, A. Brintrup, Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Tasks, arXiv preprint arXiv:2509.05651v1 (2025).doi:10.48550/arXiv. 2509.05651. 21

  9. [9]

    Laubenbacher, B

    R. Laubenbacher, B. Mehrad, I. Shmulevich, N. Trayanova, Digital twins in medicine, Nature Compu- tational Science 4 (3) (2024) 184–191.doi:10.1038/s43588-024-00607-6

  10. [10]

    D. M. Botín-Sanabria, A.-S. Mihaita, R. E. Peimbert-García, M. A. Ramírez-Moreno, R. A. Ramírez- Mendoza, J. d. J. Lozoya-Santos, Digital Twin Technology Challenges and Applications: A Comprehen- sive Review, Remote Sensing 14 (6) (2022) 1335.doi:10.3390/rs14061335

  11. [11]

    White, A

    G. White, A. Zink, L. Codecá, S. Clarke, A digital twin smart city for citizen feedback, Cities 110 (2021) 103064.doi:10.1016/j.cities.2020.103064

  12. [12]

    Torzoni, D

    M. Torzoni, D. Maisto, A. Manzoni, F. Donnarumma, G. Pezzulo, A. Corigliano, Active digital twins via active inference, Engineering Applications of Artificial Intelligence 174 (2026) 114519.doi:10.1016/ j.engappai.2026.114519

  13. [13]

    On the maximum storage capacity of the Hopfield model.Frontiers in Computational Neuroscience, 10:144, 2017

    L. Pio-Lopez, F. Kuchling, A. Tung, G. Pezzulo, M. Levin, Active inference, morphogenesis, and compu- tational psychiatry, Frontiers in Computational Neuroscience 16 (2022) 988977.doi:10.3389/fncom. 2022.988977

  14. [14]

    Friston, S

    K. Friston, S. Samothrakis, R. Montague, Active inference and agency: optimal control without cost functions, Biological Cybernetics 106 (8) (2012) 523–541.doi:10.1007/s00422-012-0512-8

  15. [15]

    J. Gama, I. Žliobait˙ e, A. Bifet, M. Pechenizkiy, A. Bouchachia, A survey on concept drift adaptation, ACM computing surveys (CSUR) 46 (4) (2014) 1–37.doi:10.1145/2523813

  16. [16]

    J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, G. Zhang, Learning under Concept Drift: A Review, IEEE transactions on knowledge and data engineering 31 (12) (2019) 2346–2363.doi:10.1109/TKDE.2018. 2876857

  17. [17]

    Pezzulo, F

    G. Pezzulo, F. Rigoli, K. J. Friston, Hierarchical Active Inference: A Theory of Motivated Control, Trends in Cognitive Sciences 22 (4) (2018) 294–306.doi:10.1016/j.tics.2018.01.009

  18. [18]

    Sajid, P

    N. Sajid, P. Tigas, K. Friston, Active inference, preference learning and adaptive behaviour, in: IOP Conference Series: Materials Science and Engineering: International Workshop on Embodied Intelli- gence, Vol. 1261, 2022, p. 012020.doi:10.1088/1757-899X/1261/1/012020

  19. [19]

    Neural Computation , volume=

    K. Friston, T. FitzGerald, F. Rigoli, P. Schwartenbeck, G. Pezzulo, Active Inference: A Process Theory, Neural Computation 29 (1) (2017) 1–49.doi:10.1162/NECO\_a\_00912

  20. [20]

    T. Parr, K. J. Friston, Generalised free energy and active inference, Biological Cybernetics 113 (5) (2019) 495–513.doi:10.1007/s00422-019-00805-w

  21. [21]

    CoRRabs/2002.12636(2020)

    A. Tschantz, B. Millidge, A. K. Seth, C. L. Buckley, Reinforcement learning through active inference, arXiv preprint arXiv:2002.12636v1 (2020).doi:10.48550/arXiv.2002.12636

  22. [22]

    Ruiz-Serra, P

    J. Ruiz-Serra, P. Sweeney, M. S. Harré, Factorised Active Inference for Strategic Multi-Agent Interac- tions, arXiv preprint arXiv:2411.07362v2 (2025).doi:10.48550/arXiv.2411.07362

  23. [23]

    Maisto, F

    D. Maisto, F. Donnarumma, G. Pezzulo, Interactive Inference: A Multi-Agent Model of Cooperative Joint Actions, IEEE Transactions on Systems, Man, and Cybernetics: Systems 54 (2) (2023) 704–715. doi:10.1109/tsmc.2023.3312585

  24. [24]

    Leibenstein, Bandwagon, snob, and veblen effects in the theory of consumers’ demand, The Quarterly Journal of Economics 64 (2) (1950) 183–207.doi:10.2307/1882692

    H. Leibenstein, Bandwagon, snob, and veblen effects in the theory of consumers’ demand, The Quarterly Journal of Economics 64 (2) (1950) 183–207.doi:10.2307/1882692

  25. [25]

    Friston, M

    K. Friston, M. Levin, B. Sengupta, G. Pezzulo, Knowing one’s place: a free-energy approach to pattern regulation, Journal of the Royal Society Interface 12 (105) (2015) 20141383.doi:10.1098/rsif.2014. 1383

  26. [26]

    Heins, B

    C. Heins, B. Millidge, D. Demekas, B. Klein, K. Friston, I. D. Couzin, A. Tschantz, pymdp: A Python library for active inference in discrete state spaces, Journal of Open Source Software 7 (73) (2022) 4098. doi:10.21105/joss.04098. 22

  27. [27]

    K. P. Murphy, Probabilistic Machine Learning: Advanced Topics, MIT Press, 2023

  28. [28]

    M. J. Wainwright, M. I. Jordan, Graphical Models, Exponential Families, and Variational Inference, Now Publishers Inc., Hanover, MA, 2008.doi:10.1561/2200000001

  29. [29]

    Fudenberg, J

    D. Fudenberg, J. Tirole, Game Theory, MIT press, 1991

  30. [30]

    Friedman, J

    D. Friedman, J. Yellin, Evolving landscapes for population games, unpublished (1997)

  31. [31]

    H. M. Gomes, J. Read, A. Bifet, Streaming Random Patches for Evolving Data Stream Classification, in: 2019 IEEE international conference on data mining (ICDM), 2019, pp. 240–249.doi:10.1109/ ICDM.2019.00034

  32. [32]

    Bindra, D

    S. Bindra, D. Sharma, N. Parameswar, S. Dhir, J. Paul, Bandwagon effect revisited: A systematic review to develop future research agenda, Journal of Business Research 143 (2022) 305–317.doi: 10.1016/j.jbusres.2022.01.085

  33. [33]

    Journal of The Royal Society Interface , year =

    K. Friston, Life as we know it, Journal of The Royal Society Interface 10 (86) (2013) 20130475.doi: 10.1098/rsif.2013.0475

  34. [34]

    Schwartenbeck, J

    P. Schwartenbeck, J. Passecker, T. U. Hauser, T. H. FitzGerald, M. Kronbichler, K. J. Friston, Computational mechanisms of curiosity and goal-directed exploration, eLife 8 (2019) e41703.doi: 10.7554/eLife.41703

  35. [35]

    Proietti, T

    R. Proietti, T. Parr, A. Tessari, K. Friston, G. Pezzulo, Active inference and cognitive control: Balancing deliberation and habits through precision optimization, Physics of Life Reviews 54 (2025) 27–51.doi: 10.1016/j.plrev.2025.05.008

  36. [36]

    T. K. Ho, The random subspace method for constructing decision forests, IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (8) (1998) 832–844.doi:10.1109/34.709601

  37. [37]

    Bifet, G

    A. Bifet, G. Holmes, B. Pfahringer, Leveraging Bagging for Evolving Data Streams, in: Machine Learn- ing and Knowledge Discovery in Databases, Springer Berlin Heidelberg, Berlin, Heidelberg, 2010, pp. 135–150.doi:10.1007/978-3-642-15880-3\_15

  38. [38]

    acceptable prod

    G. Louppe, Understanding random forests: From theory to practice, Ph.D. thesis, Universite de Liege (2014).doi:10.13140/2.1.1570.5928. Appendix A. Likelihood derivation for multi-state modalities This appendix details the derivation of a likelihood matrix that accounts for the two components p(warehouse signal|warehouse)andp(warehouse signal|context). Let...