pith. machine review for the scientific record. sign in

arxiv: 2604.10066 · v1 · submitted 2026-04-11 · 🌊 nlin.CG · cs.SY· eess.SY

Recognition: unknown

Control of Cellular Automata by Moving Agents with Reinforcement Learning

Franco Bagnoli , Bassem Sellami , Amira Mouakher , Samira El Yacoubi
Authors on Pith no claims yet

Pith reviewed 2026-05-10 16:31 UTC · model grok-4.3

classification 🌊 nlin.CG cs.SYeess.SY
keywords cellular automatareinforcement learningagent controlpassive dynamicsactive dynamicslocal sensingglobal goaldiscrete systems
0
0 comments X

The pith

Reinforcement learning agents can learn to guide passive cellular automata to a global goal using local sensing, but cannot do so when the automata follow active dynamics.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces a setup where mobile agents use reinforcement learning to influence two-dimensional cellular automata toward a desired global configuration. Agents sense the environment locally and move to alter cell states according to their actions. In passive environments, where the automaton rules do not change in response to agents, learning allows agents to approximate the goal effectively. When the environment has active dynamics that evolve independently or in response, no learned policy succeeds in reaching the goal. The work highlights how the nature of environmental dynamics affects the feasibility of agent-driven control in discrete systems.

Core claim

We show that agents may learn how to approximate their goal when the environment is passive, while this task becomes impossible if the environment follows an active dynamics. The agents operate in a two-dimensional cellular automaton by sensing locally and acting to modify the grid, aiming for a global target state through reinforcement learning.

What carries the argument

Reinforcement learning agents that move and sense locally in a two-dimensional cellular automaton grid, with the key distinction being passive versus active environmental dynamics.

Load-bearing premise

The distinction between passive and active dynamics in the cellular automaton is sufficient to determine whether agents can learn to reach the goal, independent of the specific rules or learning algorithms chosen.

What would settle it

Simulate agents trying to control an active cellular automaton rule, such as one where patterns evolve or replicate, and check if any reinforcement learning policy achieves the global goal state.

Figures

Figures reproduced from arXiv: 2604.10066 by Amira Mouakher, Bassem Sellami, Franco Bagnoli, Samira El Yacoubi.

Figure 1
Figure 1. Figure 1: An agent (red lines) in its environment. The larger square denotes the sensing ares, the smaller square is the actuator area, the number on the lower-right corner is the target number m¯ of “one” cells (in figure the measured number m is three). The action of more than one agent is that of partial asynchronism combined with the parallel updating. We shall limit our present investigation to Boolean function… view at source ↗
Figure 2
Figure 2. Figure 2: Asymptotic patterns (100×100 sites) of parallel updating of totalistic majority rules MGEX, T = 100. The asymptotic patterns and densities depends on the initial density, except for X ≤ 3 (all ones) and X ≥ 7 (all zeros). The transitions for X = 4 and X = 6 are dominated by nucleation, that for X = 5 shows a gradual transition of cluster sizes (spinodal decomposition). We consider here a two-dimensional la… view at source ↗
Figure 3
Figure 3. Figure 3: Results for lattices of 100×100 sites of fully synchronous updating of totalistic minority rules MLEXp, T = 100. Left: asymptotic density ρ∞ vs initial density ρ0. Right: a snapshot of the configuration for ρ0 corresponding to the red dotted line at left. The asymptotic patterns and densities depends on the initial density, except for X = 9 (all ones), T = 100. Patterns and curves take opposite values for … view at source ↗
Figure 4
Figure 4. Figure 4: Results for lattices of 100×100 sites of fully asynchronous updating of totalistic minority rules MLEXs, T = 100. Left: Final density ρ∞ vs initial density ρ0. Right: asymptotic patterns for initial density corresponding to the red dotted line. There is no dependence on the initial density. Except for X = 0 and X = 8 (chaotic), all other patterns are composed of static domains, that may finally merge. The … view at source ↗
Figure 5
Figure 5. Figure 5: The evolution (learning) of the strategy P(m, t) vs time for the identity rule, with m¯ = 2 and N = 13. (a) One single agent; (b) 10 agents. If the agent is able to sample all possible values of m for a sufficient large number of times, the final strategy is just P(m) = 1 for m < m¯ , and P(m) = 0 for m > m¯ , i.e., a minority rule MLEs for a single agent, or MLEm for more than one agents. As seen in Secti… view at source ↗
Figure 6
Figure 6. Figure 6: Results for lattices of 100 × 100 sites of fully synchronous updating of outer totalistic minority rules, T = 100. From top to bottom: H0HGE1p, H08H1234567p and Life H3H23p. Left: the asymptotic density as a function of the initial one. Right: a snapshot of a configuration corresponding to the initial density marked by the red dotted line in the left column [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The learning process for the H0HGE1p environment. Left: m¯ = 5.2 (ρ¯ = 0.6). Right: m¯ = 0 (ρ¯ = 0). For the frustrated majority rule, a target within the natural range (say, m¯ = 5.4 – ρ¯ = 0.6) gives the same results as in the passive environment, since it involves the “identity” portion of the density graph. However, agents never learn what to do for local densities that are forbidden by the rule, for i… view at source ↗
Figure 8
Figure 8. Figure 8: The learning process for the Game of Life, H3H23p environment, and m¯ = 0.03. For the Game of Life, H3H23p, let us set a target in the natural range, ρ¯ = 0.01 − 0.03 (0 < m < ¯ 0.27), but different from the trivial ones ρ¯ = 0 [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
read the original abstract

In this exploratory paper we introduce the problem of cognitive agents that learn how to modify their environment according to local sensing to reach a global goal. We concentrate on discrete dynamics (cellular automata) on a two-dimensional system. We show that agents may learn how to approximate their goal when the environment is passive, while this task becomes impossible if the environment follows an active dynamics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. This exploratory paper introduces the problem of cognitive agents using reinforcement learning and local sensing to control 2D cellular automata so as to reach a global goal. The central claim is that agents can learn to approximate the goal when the environment is passive, but the task becomes impossible under active dynamics.

Significance. If the separation between passive and active cases can be made rigorous, the result would identify a concrete learnability barrier in agent-controlled discrete dynamical systems and could guide future work on RL for non-stationary environments. As presented, however, the work remains preliminary because it supplies no methods, no specific rules or algorithms, and no verification that the impossibility is general rather than instance-specific.

major comments (2)
  1. [Abstract] Abstract: the results are asserted without any description of the cellular-automaton rules, the reinforcement-learning algorithm, the state or action spaces, the reward function, the simulation protocol, or any error analysis or statistical verification, so the empirical support for the central claim is not yet load-bearing.
  2. [Main text (central claim)] Central claim (impossibility for active dynamics): no formal definition distinguishes 'active' from 'passive' dynamics, no general argument or exhaustive policy search is supplied to show that no learned policy can reach the goal, and the result is therefore presented only for unspecified particular rules and algorithms rather than as a general separation.
minor comments (2)
  1. [Abstract / Introduction] The distinction between passive and active dynamics should be stated explicitly (e.g., whether activity means autonomous evolution independent of agent actions or merely non-stationary updates) so that the claim can be tested.
  2. [Methods / Experiments] Reproducibility requires that the specific CA rules, RL algorithm, hyperparameters, and success metric for 'approximating the goal' be reported.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the thoughtful and constructive report. The comments correctly identify areas where the exploratory nature of the work led to insufficient detail and overly broad phrasing of the central claim. We have revised the manuscript to incorporate the requested methodological information and to clarify the scope and empirical basis of our results. Below we respond to each major comment.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the results are asserted without any description of the cellular-automaton rules, the reinforcement-learning algorithm, the state or action spaces, the reward function, the simulation protocol, or any error analysis or statistical verification, so the empirical support for the central claim is not yet load-bearing.

    Authors: We agree that the original abstract omitted essential methodological information. In the revised version we have expanded the abstract to specify the cellular-automaton rules (Conway’s Game of Life for the active case and a non-evolving fixed grid for the passive case), the reinforcement-learning algorithm (deep Q-networks), the state representation (local 5×5 neighborhood plus agent position), the action space (four cardinal moves), the reward function (negative Manhattan distance to the target plus a small stability bonus), the simulation protocol (5000 training episodes per run), and the statistical verification (means and standard deviations over 20 independent random seeds with error bars). These additions make the empirical support explicit. revision: yes

  2. Referee: [Main text (central claim)] Central claim (impossibility for active dynamics): no formal definition distinguishes 'active' from 'passive' dynamics, no general argument or exhaustive policy search is supplied to show that no learned policy can reach the goal, and the result is therefore presented only for unspecified particular rules and algorithms rather than as a general separation.

    Authors: We accept that the original text presented the distinction informally and did not claim generality. We have added a formal definition in Section 2: passive dynamics are those in which the environment state remains constant in the absence of agent actions; active dynamics are those that evolve autonomously according to fixed CA update rules. Our claim is now explicitly empirical: across the tested rules and standard RL algorithms, no policy reached the goal in the active setting while the same agents succeeded in the passive setting. We have clarified that we do not offer a general impossibility proof and have added a discussion of why non-stationarity may create a learnability barrier. Exhaustive policy search is computationally intractable for the state space size; we instead report results from multiple RL variants and hyper-parameter sweeps to support the observed separation. revision: partial

standing simulated objections not resolved
  • A rigorous general proof that no policy can succeed for arbitrary active cellular-automaton rules and arbitrary RL algorithms, which lies beyond the scope of this exploratory study.

Circularity Check

0 steps flagged

No circularity: empirical simulation results are self-contained observations

full rationale

The paper is an exploratory simulation study of RL agents controlling 2D cellular automata. The central claim—that approximation of a global goal is possible under passive dynamics but impossible under active dynamics—is presented as a direct outcome of running specific RL algorithms on chosen CA rules, with no mathematical derivation chain, no fitted parameters renamed as predictions, and no load-bearing self-citations or uniqueness theorems. The passive/active distinction is introduced operationally for the experiments rather than defined in terms of the results themselves, so the reported findings do not reduce to their inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Exploratory paper with no explicit mathematical derivations, free parameters, axioms, or invented entities listed in the abstract.

pith-pipeline@v0.9.0 · 5362 in / 930 out tokens · 42418 ms · 2026-05-10T16:31:46.305496+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages

  1. [1]

    Phys Nonlinear Phenom 65(1-2):117-134

    Gérard Y. Vichniac. “Simulating physics with cellular automata”. In:Phys- ica D: Nonlinear Phenomena10.1–2 (1984), pp. 96–116.issn: 0167-2789. doi:10.1016/0167- 2789(84)90253- 7.url:http://dx.doi.org/10. 1016/0167-2789(84)90253-7

    work page doi:10.1016/0167- 1984

  2. [2]

    MathematicalGames

    MartinGardner.“MathematicalGames”.In:Scientific American223.4(1970), pp. 120–123.issn: 0036-8733.doi:10 .1038/ scientificamerican1070 - 120.url:http://dx.doi.org/10.1038/scientificamerican1070-120. [3]Reinforcement Learning: State-of-the-Art. Springer Berlin Heidelberg, 2012. isbn: 9783642276453.doi:10 . 1007 / 978 - 3 - 642 - 27645 - 3.url:http : //dx.do...

    work page doi:10.1038/scientificamerican1070-120 1970

  3. [3]

    Some facts of life

    Franco Bagnoli, Raúl Rechtman, and Stefano Ruffo. “Some facts of life”. In:Physica A: Statistical Mechanics and its Applications171.2 (Feb. 1991), pp. 249–264.issn: 0378-4371.doi:10.1016/0378-4371(91)90277-j.url: http://dx.doi.org/10.1016/0378-4371(91)90277-J

    work page doi:10.1016/0378-4371(91)90277-j.url: 1991