Pith. sign in

REVIEW 1 major objections 3 minor 45 references

A "good regulator theorem" for embodied agents

T0 review · 1 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A system that reliably regulates a coupled environment can always be interpreted as holding beliefs about it, with the model supplied by the observer and allowed to be trivial.

desk verdict A correct but permissive formal equivalence: the theorem works, but the weak belief-update condition makes the 'every good regulator has a model' result largely a re-description. read the letter →

arxiv 2508.06326 v2 pith:K62LPYJW submitted 2025-08-04 cs.AI cs.SYeess.SY

Nathaniel Virgo , Martin Biehl , Manuel Baltieri , Matteo Capucci This is my paper · ORCID
classification cs.AIcs.SYeess.SY
keywords goodregulatortheoreminterpretationmapspossibilisticbeliefsbeliefupdatingembodiedagentssensorimotorloopforward-closedsetsas-ifagency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proves that any system that reliably keeps a coupled environment inside a prescribed set of good joint states can be interpreted by an observer as holding beliefs about the environment and updating them after each sensor reading. This replaces the classic claim that every good regulator must be a model with a wider 'as-if' version: the regulator admits a model, imposed by the observer, rather than containing one. The only required ingredient is a non-empty, forward-closed regulating set, and the proof constructs the agent's beliefs as slices of that set at the agent's current state. Because the attributed model can be trivial, simple reactive systems that seem to lack models are not counterexamples. The result holds whether the task is to regulate the environment or to regulate the agent's own internal state.

What carries the argument

The load-bearing object is the possibilistic belief map $\psi:X\to \mathcal P(Z)$, a function assigning to each agent state a set of possible environment states. It is consistent when, for every $x$ and sensor value $s$, the set $\operatorname{update}(\psi(x),r(x),s)$ is contained in $\psi(u(x,s))$, where $\operatorname{update}(B,a,s)=\{z'\in Z \mid \exists z\in B:(z',s)=f(z,a)\}$. This is a set-valued, non-probabilistic analogue of Bayesian filtering, and the containment rather than equality is what permits forgetting. Lemma 3.2 says a subset $R\subseteq X\times Y$ is forward-closed if and only if the slice map $\psi(x)=\{y\in Y \mid (x,y)\in R\}$ is a consistent belief map; this turns the existence of a regulating set into the existence of a belief interpretation. A second map $\varphi(x)=\{y\in Y \mid (x,y)\in G\}$ carries the goal, and the theorem requires $\psi(x)\subseteq\varphi(x)$ for all $x$.

What would settle it

Enumerate all small finite agent-environment machines, say with up to four internal and four environment states, together with a chosen good set. The theorem predicts that every non-empty forward-closed subset $R\subseteq G$ induces a slice map $\psi(x)=\{y\in Y \mid (x,y)\in R\}$ satisfying the consistency inclusion of Definition 3.1; one machine for which this inclusion fails would refute the central claim.

Watch

Extended reading notes

Core claim

The central discovery is an equivalence, Theorem 3.4, between 'objective' and 'subjective' regulation. Given a deterministic agent $(X,r,u)$, an environment $(Y,e)$, a good set $G\subseteq X\times Y$, and a non-empty forward-closed set $R\subseteq G$, the agent is a good regulator with good set $G$ and regulating set $R$ if and only if it is a subjective good regulator with model $(Z,f)=(Y,e)$, belief map $\psi$ defined by $\psi(x)=\{y\in Y \mid (x,y)\in R\}$, and normative map $\varphi$ defined by $\varphi(x)=\{y\in Y \mid (x,y)\in G\}$. In other words, the regulating set is exactly a consistent possibilistic belief state and the good set is exactly a normative state, as attributed by the observer. The substantive direction runs from forward-closedness to belief consistency: if $R$ is forward-closed, the slices of $R$ update according to the environment's dynamics, with the possibility of deliberately forgetting information built into the consistency condition.

Load-bearing premise

The result depends on letting an agent's attributed beliefs become less precise over time: after each observation the new belief set only has to contain the exact update, so forgetting is always allowed; if update had to be exact, the theorem would stop being true.

Editorial extensions

If this is right

  • Every system that possesses a non-empty forward-closed subset of the good set can be attributed a consistent possibilistic belief model of the environment it is coupled to.
  • The model in the theorem can be chosen to be the true environment, so no additional assumption such as full observability or an explicit internal encoding is required.
  • Because the good set is a subset of the joint state space, the theorem covers both regulating an external environment and regulating one's own internal state, attributing a model of the environment in both cases.
  • Trivial models are allowed, so systems with no internal dynamics, such as a one-state doorstop, receive a constant belief set rather than refuting the claim that every good regulator has a model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same physical system can count as model-based or model-free depending on the observer's choice of good set and regulating set; modelhood becomes a relational property rather than an intrinsic one.
  • A natural next step, not pursued here, is to measure how non-trivial an attributed interpretation is, for example by how much the belief map changes with the agent's state or how much information it carries about the environment.
  • The set-valued, forgetting-tolerant update rule is suggestive for robust control and for theories of internal models under uncertainty; testing whether the equivalence survives in stochastic or continuous-time settings would require a measure-theoretic analogue of the inclusion condition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 3 minor

Summary. The paper proposes a formal framework in which an embodied agent is a Moore machine and its environment a Mealy machine, and defines 'good regulation' as the existence of a non-empty forward-closed subset R of a given good set G in the coupled state space. The main theorem (3.4) states that an agent is such a good regulator if and only if it can be interpreted as a 'subjective good regulator' with a possibilistic belief map ψ(x)={y:(x,y)∈R}, a normative map φ(x)={y:(x,y)∈G}, and the true environment as its model. Lemma 3.2 establishes the key equivalence between forward-closedness of R and the consistency condition update(ψ(x),r(x),s) ⊆ ψ(u(x,s)) for belief updating. The paper frames the result as a rehabilitation of the Conant–Ashby idea that every good regulator has a model, while emphasizing that models are observer-attributed and may be trivial, as in the doorstop example.

Significance. If the theorem's interpretation is accepted, it provides a very general, minimal formal sense in which any regulator has a model of its environment, without the extra assumptions of Conant and Ashby or the internal model principle. The proof is transparent and relies only on elementary set theory, and the definitions are explicit about the weakness of the regulation notion and the 'forgetting' allowance. The paper also makes a useful contribution by clarifying that such models are observer-relative and can be trivial. However, the significance is tempered by the fact that the notion of 'belief updating' is very weak: the consistency condition permits the posterior to remain a superset of the exact possibilistic update, even if that superset contains states incompatible with the observed sensor value. Whether the result counts as showing that agents 'update beliefs in response to sensory input' is therefore a matter of interpretation.

major comments (1)
  1. [Definition 3.1, after Eq. (5)] The 'forgetting' step in Definition 3.1 allows any posterior C with update(B,a,s) ⊆ C. As stated, this permits C to contain states that are ruled out by the observed sensor value under the model f (for example, with X={x}, S={s1,s2}, Z={z1,z2}, f(z1,a)=(z1,s1), f(z2,a)=(z2,s2), the constant belief ψ(x)=Z satisfies the condition after observing s2 even though z1 cannot produce s2). Thus the condition is not merely an allowance for forgetting; it permits observation-incompatible beliefs. Since Lemma 3.2 and Theorem 3.4 depend on this weak inclusion, the abstract's claim that the agent 'updates' beliefs 'in response to sensory input' overstates the result. Please either strengthen the definition (if possible without breaking the theorem) or explicitly qualify the language in the abstract and conclusions to say that the attributed belief sets are not required to be sound with respect to the observations, and that 'updating' here means only that the posterior contains the exact possibilistic update, not that it is equal to it or that it excludes ruled-out states.
minor comments (3)
  1. [Definition 2.2 and Definition 2.3] The type of the Mealy machine evolution function is inconsistent; Definition 2.2 writes e : Y × S → Y × A, but Definition 2.3 and the surrounding text use e : Y × A → Y × S. Please correct the type in Definition 2.2.
  2. [Paragraph after Eq. (5)] The text introduces an imaginary person who 'can decide to forget information'; given the issue raised in the major comment, I suggest adding a sentence there clarifying that a posterior C may contain states incompatible with the observed s, which is a deliberate weakening beyond ordinary forgetting.
  3. [Lemma 3.2 proof] In the chain of equivalences, the third line uses a set-builder notation where s appears both as a bound variable and as a component of e(y,r(x)); this is correct but could be clarified for readers.

Circularity Check

2 steps flagged · score 8.0 of 10

Main theorem is a definitional equivalence: ψ and φ are declared to be fibers of R and G, so 'every good regulator has a model' restates the existence of a forward-closed set in belief vocabulary.

  1. self definitional [Section 3.1, Eqs. (7)–(8) and Theorem 3.4]
    "As mentioned previously, the interpretation maps ψ : X → P(Z) and ϕ : X → P(Z) are really just the regulating set R ⊆ X × Y and the good set G ⊆ X × Y in disguise. Given G and R, we can set (Z, f) = (Y, e) and define ψ(x) = { y ∈ Y | (x, y) ∈ R } as in eq. (7), along with ϕ(x) := { y ∈ Y | (x, y) ∈ G }. (8)"

    This is the reduction: the 'model' attributed to the agent is the fiber decomposition of the already-given regulating set R, and the 'norm' is the fiber decomposition of G. Theorem 3.4 then checks that the three clauses of Definition 3.3 are the three clauses of Definition 2.5 expressed through these fibers. The biconditional is therefore true by construction; the claimed conclusion that every good regulator 'can be interpreted as having a model' restates the existence of R in the new vocabulary of ψ, rather than deriving a model from regulation.

  2. other [Definition 3.1 and the paragraph after Eq. (5)]
    "Although the ideal posterior is given by update(B, a, s), we will allow the person to adopt any posterior C as long as update(B, a, s) ⊆ C. The set C can contain less information than update(B, a, s), in the sense that it puts less constraint on what the environment’s state might be."

    The 'consistency' of a belief map is weakened to a superset inclusion. Because only inclusion is required, Lemma 3.2 can equate consistency with forward-closedness of R, which is exactly the condition the regulator definition already imposes. The main theorem's notion of belief updating is thus calibrated to make every forward-closed set a belief map; if equality were required instead, the theorem would fail. This is a definitional choice that builds the conclusion into Definition 3.1.

full rationale

The paper is honest that models are observer-imposed and may be trivial, and the mathematical proof is valid. But the central theorem is not an independent first-principles result: it chooses ψ and ϕ as the vertical fibers of the regulating and good sets, and Definition 3.1's subset (forgetting) condition is precisely what makes consistency equivalent to forward-closedness. Hence the headline claim reduces by construction to a relabelling of the inputs. There is no hidden fitted parameter or load-bearing self-citation; the self-citations to the authors' earlier interpretation-map work supply terminology but not the theorem's validity. The circularity is definitional rather than evidential: the equivalence is explicit and correctly proved, but the 'prediction' that every good regulator has a model is entailed by the definitions.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted; the central claim rests on observer-imposed choices (G and R), the deterministic discrete-time formalism, and the weak inclusion condition for belief updating. No new physical entities are introduced.

assumptions (4)
  • domain assumption Systems are deterministic and in discrete time; the formalism uses Moore and Mealy machines rather than stochastic or continuous dynamics.
    Section 2: 'We restrict our attention to deterministic systems in discrete time'; the paper notes the extension to stochastic is not obvious.
  • ad hoc to paper Consistent belief maps require only that the posterior contains the exact possibilistic update (subset, not equality), i.e., forgetting is allowed.
    Paragraph after eq. (5) and Definition 3.1; this choice is essential for Lemma 3.2 and Theorem 3.4.
  • domain assumption The attributed model (Z,f) is taken to be the true environment (Y,e) in the theorem.
    Theorem 3.4 and its proof; the paper notes other models are possible but not covered.
  • domain assumption The good set G and regulating set R are chosen by an observer, not derived from the system.
    Section 2, 'Some philosophical context'; this is the interpretive stance underlying the definitions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A "good regulator theorem" for embodied agents." pith.science (2026). https://pith.science/paper/K62LPYJW

@misc{pith2026250806326,
  author       = {Pith},
  title        = {Pith review of: A "good regulator theorem" for embodied agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K62LPYJW}},
  note         = {Machine review of arXiv:2508.06326}
}
read the original abstract

In a classic paper, Conant and Ashby claimed that "every good regulator of a system must be a model of that system." Artificial Life has produced many examples of systems that perform tasks with apparently no model in sight; these suggest Conant and Ashby's theorem doesn't easily generalise beyond its restricted setup. Nevertheless, here we show that a similar intuition can be fleshed out in a different way: whenever an agent is able to perform a regulation task, it is possible for an observer to interpret it as having "beliefs" about its environment, which it "updates" in response to sensory input. This notion of belief updating provides a notion of model that is more sophisticated than Conant and Ashby's, as well as a theorem that is more broadly applicable. However, it necessitates a change in perspective, in that the observer plays an essential role in the theory: models are not a mere property of the system but are imposed on it from outside. Our theorem holds regardless of whether the system is regulating its environment in a classic control theory setup, or whether it's regulating its own internal state; the model is of its environment either way. The model might be trivial, however, and this is how the apparent counterexamples are resolved.

Figures

Figures reproduced from arXiv: 2508.06326 by the authors.

Figure 1
Figure 1. Schematics drawn in continuous time (cf. Beer, 1995), although the definitions in the main text are discrete. (a) A pair of coupled systems together form a dynamical system, some of whose variables belong to each system; in this schematic we assume one variable each, resulting in a two dimensional phase space. (b) The same phase space, equipped with a “good set” G. Not every trajectory that starts within the good se… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 38 canonical work pages

  1. [1]

    Ashby, W. R. (1960). Design for a brain . Wiley New York

    work page 1960

  2. [2]

    Baez, J. (2016). The internal model principle. Blog post on `Azimuth' https://johncarlosbaez.wordpress.com/2016/01/27/the-good-regulator-theorem/. Accessed: 2025-03-26

    work page 2016

  3. [3]

    Baez, J. C. and Stay, M. (2010). Physics, topology, logic and computation: a Rosetta Stone . In New structures for physics , pages 95--172. Springer

    work page 2010

  4. [4]

    Baltieri, M., Biehl, M., Capucci, M., and Virgo, N. (2025). A Bayesian interpretation of the internal model principle. arXiv preprint 2503.00511 https://arxiv.org/abs/2503.00511

    arXiv 2025

  5. [5]

    Beer, R. D. (1995). A dynamical systems perspective on agent-environment interaction . Artificial Intelligence , 72(1-2):173--215

    work page 1995

  6. [6]

    Beer, R. D. (1997). The dynamics of adaptive behavior: A research program. Robotics and Autonomous Systems , 20(2-4):257--289

    work page 1997

  7. [7]

    Beer, R. D. (2003). The dynamics of active categorical perception in an evolved model agent. Adaptive Behavior , 11(4):209--243

    work page 2003

  8. [8]

    Beer, R. D. (2004). Autopoiesis and Cognition in the Game of Life . Artificial Life , 10(3):309--326

    work page 2004

Show all 45 references
  1. [9]

    D., McShaffrey, C., and Gaul, T

    Beer, R. D., McShaffrey, C., and Gaul, T. M. (2024). Deriving the intrinsic viability constraint of an emergent individual from first principles. In The 2024 Conference on Artificial Life , ALIFE 2024. MIT Press

    2024

  2. [10]

    Bertschinger, N., Olbrich, E., Ay, N., and Jost, J. (2006). Information and closure in systems theory. In Proceedings of the 7th German Workshop on Artificial Life , pages 9--19

    2006

  3. [11]

    and Virgo, N

    Biehl, M. and Virgo, N. (2023). Interpreting systems as solving POMDPs : A step towards a formal understanding of agency. In Buckley, C. L. e. a., editor, Active Inference. IWAI 2022. Communications in Computer and Information Science , pages 16--31. Springer

    2023

  4. [12]

    Braitenberg, V. (1986). Vehicles: Experiments in synthetic psychology . MIT press

    1986

  5. [13]

    and Kissinger, A

    Coecke, B. and Kissinger, A. (2017). Picturing Quantum Processes: A First Course in Quantum Theory and Diagrammatic Reasoning . Cambridge University Press

    2017

  6. [14]

    Conant, R. C. and Ashby, W. R. (1970). Every good regulator of a system must be a model of that system. International journal of systems science , 1(2):89--97

    1970

  7. [15]

    Da Costa, L., Friston, K., Heins, C., and Pavliotis, G. A. (2021). Bayesian mechanics for stationary processes. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , 477(2256)

    2021

  8. [16]

    and Di Paolo, E

    De Jaegher, H. and Di Paolo, E. (2007). Participatory sense-making: An enactive approach to social cognition. Phenomenology and the Cognitive Sciences , 6(4):485–507

    2007

  9. [17]

    Dennett, D. (2006). Intentional systems theory. In McLaughlin, B., Beckermann, A., and Walter, S., editors, The Oxford Handbook of Philosophy of Mind . Oxford University Press

    2006

  10. [18]

    Dennett, D. C. (1991). Real patterns. Journal of Philosophy , 88(1):27--51

    1991

  11. [19]

    Di Paolo, E. A. (2005). Autopoiesis, adaptivity, teleology, agency. Phenomenology and the Cognitive Sciences , 4(4):429–452

    2005

  12. [20]

    Egbert, M. D. and P \'e rez-Mercader, J. (2018). Methods for measuring viability and evaluating viability indicators. Artificial life , 24(02):106--118

    2018

  13. [21]

    Fong, B. (2013). Causal theories: A categorical perspective on Bayesian networks. arXiv preprint 1301.6201 https://arxiv.org/abs/1301.6201

    2013 arXiv

  14. [22]

    Francis, B. A. and Wonham, W. M. (1976). The internal model principle of control theory. Automatica , 12(5):457--465

    1976

  15. [23]

    Friston, K. J. (2019). A free energy principle for a particular physics. arXiv preprint arXiv:1906.10184

    2019 arXiv

  16. [24]

    and Klingler, A

    Fritz, T. and Klingler, A. (2023). The d-separation criterion in categorical probability. Journal of Machine Learning Research , 24(46):1--49

    2023

  17. [25]

    Jacobs, B. (2020). A channel-based perspective on conjugate priors. Mathematical Structures in Computer Science , 30(1):44--61

    2020

  18. [26]

    Jacobs, B., Kissinger, A., and Zanasi, F. (2019). Causal inference by string diagram surgery. In Boja \' n czyk, M. and Simpson, A., editors, Foundations of Software Science and Computation Structures , pages 313--329, Cham. Springer International Publishing

    2019

  19. [27]

    Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of basic Engineering , 82(1):35--45

    1960

  20. [28]

    Klyubin, A., Polani, D., and Nehaniv, C. (2004). Organization of the information flow in the perception-action loop of evolved agents. In 2004 NASA/DoD Conference on Evolvable Hardware, 2004. Proceedings , pages 177--180

    2004

  21. [29]

    and Wolpert, D

    Kolchinsky, A. and Wolpert, D. H. (2018). Semantic information, autonomous agency and non-equilibrium statistical physics. Interface Focus , 8(6):20180041

    2018

  22. [30]

    Maturana, H. R. and Varela, F. J. (1980). Autopoiesis and cognition: The realization of the living . Springer Science & Business Media

    1980

  23. [31]

    McGregor, S. (2016). A more basic version of agency? As if! In Tuci, E., Giagkos, A., Wilson, M., and Hallam, J., editors, From Animals to Animats 14 , pages 183--194, Cham. Springer International Publishing

    2016

  24. [32]

    McGregor, S. (2017). The Bayesian stance: Equations for ‘as-if’ sensorimotor agency. Adaptive Behavior , 25(2):72--82

    2017

  25. [33]

    McGregor, S., timorl, and Virgo, N. (2025a). Formalising the intentional stance 1: attributing goals and beliefs to stochastic processes. arXiv preprint 2405.16490 https://arxiv.org/abs/2405.16490

    2025 arXiv

  26. [34]

    McGregor, S., timorl, and Virgo, N. (2025b). Formalising the intentional stance 2: a coinductive approach. arXiv preprint 2501.09173 https://arxiv.org/abs/2501.09173

    2025 arXiv

  27. [35]

    and Beer, R

    McShaffrey, C. and Beer, R. D. (2023). Decomposing viability space. In Artificial Life Conference Proceedings 35 , page 51. MIT Press

    2023

  28. [36]

    Myers, D. J. (2023). Categorical systems theory. Online book draft http://davidjaz.com/Papers/DynamicalBook.pdf. Accessed: 2025-05-07

    2023

  29. [37]

    A., Wang, J

    Ortega, P. A., Wang, J. X., Rowland, M., Genewein, T., Kurth-Nelson, Z., Pascanu, R., Heess, N., Veness, J., Pritzel, A., Sprechmann, P., Jayakumar, S. M., McGrath, T., Miller, K., Azar, M., Osband, I., Rabinowitz, N., György, A., Chiappa, S., Osindero, S., Teh, Y. W., van Has...

    2019 arXiv

  30. [38]

    Richens, J., Abel, D., Bellot, A., and Everitt, T. (2025). General agents need world models. arXiv preprint 2506.01622 https://arxiv.org/abs/2506.01622

    2025

  31. [39]

    E., Geiger, B

    Rosas, F. E., Geiger, B. C., Luppi, A. I., Seth, A. K., Polani, D., Gastpar, M., and Mediano, P. A. M. (2024). Software in the natural world: A computational approach to hierarchical emergence. arXiv preprint 2402.09090 https://arxiv.org/abs/2402.09090

    2024 arXiv

  32. [40]

    Selinger, P. (2010). A survey of graphical languages for monoidal categories. In New structures for physics , pages 289--355. Springer

    2010

  33. [41]

    Seth, A. K. and Tsakiris, M. (2018). Being a beast machine: The somatic basis of selfhood. Trends in cognitive sciences

    2018

  34. [42]

    Virgo, N., Biehl, M., and McGregor, S. (2021). Interpreting dynamical systems as Bayesian reasoners. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 726--762. Springer

    2021

  35. [43]

    Wentworth, J. S. (2021). Fixing the good regulator theorem. Post on `AI Alignment Forum' https://www.alignmentforum.org/posts/Dx9LoqsEh3gHNJMDk/fixing-the-good-regulator-theorem, also posted on the `LessWrong' blog at https://www.lesswrong.com/posts/Dx9LoqsEh3gHNJMDk/fixing-th...

    2021

  36. [44]

    Zahedi, K., Ay, N., and Der, R. (2010). Higher coordination with less control—a result of information maximization in the sensorimotor loop. Adaptive Behavior , 18(3-4):338--355

    2010

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.