Pith. sign in

REVIEW 4 major objections 4 minor 45 references

Intelligence from Learnable Novelty

T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Learnable novelty—the portion of surprise a bounded observer can compress—is a single quantity whose maximization yields complex dynamics, unsupervised clustering, and exploratory behavior.

desk verdict A useful, well-engineered score, an unvalidated bridge to epiplexity, and demonstrations that overreach in the abstract; still worthy of serious refereeing. read the letter →

arxiv 2607.18433 v1 pith:WNVT3I7E submitted 2026-07-20 cs.LG cs.AInlin.AO

Yanbo Zhang , Michael Levin This is my paper
classification cs.LGcs.AInlin.AO
keywords learnablenoveltyepiplexityminimumdescriptionlengthreservoircomputingcellularautomatarule110unsupervisedrepresentationlearningintrinsicreward
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the projections of intelligence—compression, universal computation, and adaptive behavior—are all consequences of one objective: maximizing learnable novelty, defined as the description length of the best model a bounded observer can extract from data. It introduces a closed-form, differentiable estimator built on a frozen random reservoir and a ridge readout, which measures how many bits of independently learnable structure remain in a signal. Used as a measure, the score recovers the classic complexity hierarchy of elementary cellular automata, placing computationally universal rule 110 at the top with no supervision. Used as an objective, gradient ascent drives a neural cellular automaton into solitons, organizes an MNIST encoder into digit clusters without labels, and, as an intrinsic reward, improves reinforcement-learning exploration in nine of ten environments. The payoff would be a common quantitative footing for phenomena normally studied with unrelated objectives.

What carries the argument

The load-bearing object is the reservoir epiplexity estimator, S_phi = 1/2 sum_i log_2(1 + eta s_i(W_lambda)^2), the spectral description length of the optimal ridge readout W_lambda = (H~^T H~ + lambda I)^-1 H~^T Y~ of a frozen random reservoir. It is the central identity because it converts an intractable search over bounded-compute models into a closed-form differentiable score: the log-determinant prices each independent readout direction logarithmically in its singular value, so redundant directions add almost nothing and scale costs only logarithmically. This makes the same number usable both as a measurement and as an objective whose gradient can be backpropagated into the system gene

What would settle it

Construct a data stream with abundant learnable structure that is not linearly readable from random reservoir features (for example, high-order logic sequences or deep compositional patterns): if S_phi stays near zero while a small trained network extracts the structure, the estimator is measuring linear predictability from random features rather than epiplexity. Alternatively, train the MNIST encoder with the reservoir's nonlinearity removed; if digit clusters still form, the nonlinear observer is not the mechanism.

Watch

Extended reading notes

Core claim

The central claim is that surprise splits into a learnable part and an unlearnable residual, and that intelligence-like behavior comes from pursuing only the learnable part—the epiplexity of the data. The paper operationalizes this with the score S_phi(Y|X) = 1/2 log_2 det(I_m + eta W_lambda W_lambda^T), where W_lambda is the closed-form ridge optimum of a linear readout on a fixed random reservoir feature map; this score is cheap, deterministic, and differentiable in whatever generated the data. As a measure it ranks rule 110 highest among elementary cellular automata, matching the known complexity hierarchy; as an objective, ascent on it produces solitons in a neural cellular automaton, un

Load-bearing premise

The whole argument rests on the assumption that the closed-form ridge readout on a fixed random reservoir actually measures the true epiplexity—that the description length of the learned readout is the learnable structure a bounded observer would extract—yet no theorem connects the ridge solution to that optimum, and the score's meaning shifts with hand-set hyperparameters.

Editorial extensions

If this is right

  • If correct, complexity classification of cellular automata is recoverable without labels or training, simply by asking how much structure a fixed reservoir readout can extract.
  • Gradient ascent on this single score can autonomously create complex, information-carrying dynamics (solitons) from a simple rule, suggesting an unsupervised route to computational capacity.
  • Representation learning can be driven by observer decodability alone; the digit classes of MNIST emerge without labels, implying that category structure is what a bounded linear observer finds most learnable.
  • Used as an intrinsic reward, learnable novelty reliably supplies exploration where task rewards are sparse or deceptive, improving return on nine of ten benchmark environments and never collapsing.
  • The three classical faces of intelligence—compression, universal computation, and exploration—are claimed to be the same optimization on one quantity, which if true would unify their objectives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: because the score is differentiable in the data generator, the same objective could be applied to open-ended search or creative generation tasks, where current methods often rely on hand-chosen novelty descriptors.
  • The paper's observer is frozen; a natural corollary, which the authors explicitly flag, is that letting observer and observed co-evolve might avoid saturation of the score—a setting where in-context learners could serve as both observer and generator.
  • If the identification of the ridge score with epiplexity is accepted, the estimator could serve as a general unsupervised 'interestingness' measure for selecting data or guiding exploration beyond the three testbeds.
  • A cautionary inference: because the ECA ranking depends on the reservoir having sufficient receptive field, the measure implicitly encodes a spatial scale; other tasks would need the observer matched to the structure of interest.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes 'learnable novelty' (epiplexity) as a unifying quantity behind complexity generation, abstraction, and exploration. It defines epiplexity as the program length of the best bounded model in a fixed model class (Section 2, Eq. 4) and introduces a closed-form, differentiable estimator S_phi based on a frozen random reservoir with a ridge linear readout (Section 3, Eq. 9). The authors then report three demonstrations: (i) S_phi ranks elementary cellular automata with rule 110 highest, matching the known Turing-complete/class IV status; (ii) gradient ascent on S_phi makes a neural cellular automaton develop solitons and makes an MNIST encoder organize representations around digit classes without labels; and (iii) using S_phi as an intrinsic reward improves PPO returns in nine of ten environments relative to task reward alone. The paper is explicit that the estimator is a shallow random-feature linear-predictability score and that the observer is fixed and hyperparameter-dependent.

Significance. If the central identification held, this would be a substantial unification of dynamical-systems complexity, unsupervised representation learning, and RL exploration under one objective, and the differentiable closed-form estimator would be practically valuable. The paper has real strengths: an exhaustive 88-rule ECA sweep with ten reservoir draws per rule, one-at-a-time robustness sweeps in Appendix B, an exact-MDL comparison in Appendix D, a state-magnitude control in the RL experiments, and a public code repository. The MNIST and RL results are honest and include robustness checks. However, the load-bearing step—identifying the ridge-reservoir score with epiplexity—is made by analogy and approximation, not derivation, and the paper's own Discussion concedes the estimator captures only linearly readable random-feature structure. The three demonstrations therefore do not currently establish that a single universal quantity drives the three phenomena; they show that one reservoir-based score can rank ECA rules and shape systems under hand-set hyperparameters.

major comments (4)
  1. [Section 3, Eqs. (5)-(9)] The central identification of S_phi with the epiplexity M*_phi of Eq. (4) is not established. The model class M_phi is a frozen reservoir plus a linear readout, and C(W) in Eq. (6) is a spectral log-determinant, not a program length. The ridge solution is obtained via a small-W Taylor approximation of the MDL objective, and no theorem or synthetic ground-truth experiment shows that W_lambda approximates the true minimizer of Eq. (3). Appendix D shows that the exact minimizer of the surrogate J_MDL ranks systems almost identically, but J_MDL is still not shown to be epiplexity. The Discussion admits the estimator 'captures only the shallow structure that is linearly readable from random features.' This is a load-bearing gap: the abstract's claim that different 'projections of intelligence' emerge from ascent on one quantity is stronger than the evidence. I recommend either providing a for
  2. [Appendix B, Figure 7] The ECA ranking—the main validation of the score as a measure—is observer-relative in a way that undercuts the 'reproduces the classical complexity classification' claim in Section 4.1. Rule 110 ranks first across most hyperparameter sweeps, but at reservoir depth 2 it falls behind rule 25 (13.5 vs 10.4 bits), and at depth 1 it falls to rank 40. More strikingly, the relative order of rule 54 and rule 30 reverses when the receptive field grows beyond radius two (depth 4 or kernel 5), despite rule 54 being class IV and rule 30 class III. If the score is an observer-dependent quantity, then the 'recovery' of Wolfram classes is not a property of the ECA rules but of one chosen convolutional reservoir. The paper needs to justify why the reference observer is the principled one, or explicitly limit the claim to 'under this particular observer.' Otherwise the measurement claim is circular in th
  3. [Table 3 and Section 4] The 'single quantity' is not evaluated by a single observer across experiments: the ECA uses a circular 1D convolutional reservoir, the continuous-time flows use an MLP, the MNIST encoder uses a wide MLP with lambda=3 and eta=30, and RL uses a narrow MLP with lambda=0.3; the RL bonus weight beta is also calibrated per environment. This undermines the unified interpretation in Section 5 that 'all three were produced by a single quantity evaluated by a fixed observer of a single construction.' At minimum, the paper should show that the NCA and RL results are not qualitatively sensitive to the observer architecture and hyperparameters, as Appendix G does for MNIST and Appendix B does for ECA; without that, each demonstration is a separate task-specific surrogate objective rather than evidence for one common quantity.
  4. [Section 4.1, Figure 3] The NCA experiments show that gradient ascent on S_phi produces localized traveling structures, and the text connects these to the solitons used by rule 110. However, no evidence is provided that the learned NCA is computationally universal, that its collisions transmit or combine information, or that the dynamics are in a regime comparable to rule 110 in any formal sense. The phrase 'solitons ... by which rule 110 computes' suggests a stronger connection than the experiments establish. Either add quantitative analysis (e.g., collision-based information transfer or a comparison to known soliton CA) or soften the claim to 'traveling structures reminiscent of rule 110's solitons.'
minor comments (4)
  1. [Figure 1(c)] The illustrative bit counts are inconsistent with Eq. (6) with alpha=1/2 and eta=1: for s=256, S = 0.5 log2(1+256^2) ≈ 8.0 bits, not 9 bits; for s=16, S ≈ 4.0 bits, not 5 bits. Please correct the caption or the formula.
  2. [Abstract and Section 4.1] The abstract and main text state that rule 110 'ranks highest' without noting that this holds only under the reference observer; Appendix B shows failure at depth <=2 and tau=4. A qualifier such as 'under the reference reservoir' would be more precise.
  3. [Section 2, Eq. (2)] The text says the approximation in Eq. (2) 'errs only in lower-order terms'; for a general model class this is not guaranteed unless the class is sufficiently regular (e.g., exponential family with well-specified priors). Consider stating the regularity assumption explicitly.
  4. [Section 4.2, Appendix G] The choice lambda=3 is described as being set 'precisely so that the relevance criterion admits only this smooth structure.' This is a post hoc rationale; it would be helpful to state it as a design choice in the main text, since it is essential to the MNIST result.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Eq. (9) is an explicit ridge approximation within a declared model class, and all three demonstrations are checked against independent external targets.

full rationale

The paper's derivation chain is not circular. The theoretical object is the MDL-optimal program length S_phi = |M*_phi| (Eqs. 3-4); Section 3 instantiates the bounded model class as a frozen random reservoir plus linear readout, replaces the program-length term with the spectral log-determinant (Eq. 6), approximates the intractable minimization by ridge regression (Eq. 8), and defines the closed-form estimator in Eq. 9. Appendix D shows the ridge approximation ranks the ECA nearly identically (Spearman rho = 0.997) to the exact minimizer of the same MDL objective, so the reported score is a faithful approximation of the chosen objective, not an arbitrary fit. The step from Eq. 9 to 'epiplexity' is an interpretive identification, and the Discussion explicitly limits it: 'In its present form the estimator captures only the shallow structure that is linearly readable from random features.' That is a scope limitation, located in Section 5 (Discussion), not a circularity. Each headline result is tested against an external benchmark or held-out signal: ECA ranks against Wolfram/Cook complexity classes (with rule 110 as the target), MNIST labels are held out and used only for visualization/probing, and the RL bonus is evaluated against task returns with a state-magnitude control. No fitted parameter is renamed a prediction; no uniqueness theorem from the authors is invoked; and the load-bearing citations (Finzi et al., Musat) are not self-citations. Hyperparameter sensitivity of the ECA ranking at reservoir depth <= 2 (Appendix B) is a robustness/correctness concern, not definitional equivalence.

Assumptions & free parameters 7 free parameters · 5 assumptions · 1 invented entities

The central claim rests on treating a random reservoir plus ridge readout as the bounded observer, and on treating a spectral log-determinant of its readout as a description length. These are modeling choices with several free hyperparameters (α, η, λ, u_Y, architecture, τ, β). The ECA benchmark provides some external grounding, but the estimator's connection to true epiplexity is not validated against the original definition.

free parameters (7)
  • α (description-length scale) = 1/2 in all experiments
    Overall multiplier on the spectral description length (Eq. 6). The paper says it does not affect ECA ranking or gradient direction, but it scales the reward magnitude in RL and the objective in MNIST.
  • η (resolution) = 1 (all except MNIST: 30)
    Sets the bit cost per singular value. Low η (≤0.1) drives MNIST linear-probe accuracy below 0.5 (Fig. 10).
  • λ (ridge penalty) = 0.03 (ECA), 0.3 (NCA/RL), 0.1 (flows), 3 (MNIST)
    Controls the smoothness/learnability of functions the readout can express; the MNIST clustering claim depends on the unusually high λ=3.
  • u_Y target scale = not specified; posited in advance
    Target magnitude is priced through u_Y (Eq. 7); no principled default is given.
  • reservoir architecture (depth, width, kernel) = per data geometry (Table 3)
    ECA ranking collapses for depth 1–2 (Fig. 7); the observer's locality controls which rules are scored as complex.
  • RL bonus weight β = calibrated per environment
    Set so the bonus's whole-episode contribution is 0.1× random-policy task-return scale; a free tuning knob per task.
  • target window τ = 32 (ECA), 8 (NCA), 8–48 (RL per task)
    Hand-picked; τ=4 makes rule 110 lose its top rank (Fig. 7).
assumptions (5)
  • standard math Minimum description length decomposition L = min_M [|M| − log p(Y|X,M)] with only lower-order error (Eq. 2)
    Prequential/MDL framework from Rissanen and Grünwald; assumed without proof.
  • domain assumption A bounded observer can be represented by a frozen random reservoir plus a linear readout
    This identification is the core of the estimator (Section 3), but no theorem shows the reservoir is a faithful instantiation of the halting-program class M_φ.
  • ad hoc to paper The spectral log-determinant ½ log det(I + ηWWᵀ) is a valid description length for the readout
    Motivated by a hierarchical Gaussian prior (Appendix C) with α=(a+D)/2, then α is arbitrarily set to 1/2, discarding the D-dependence.
  • domain assumption Maximizing learnable novelty drives systems toward the edge of chaos where universal computation lives
    Used to interpret NCA solitons (Section 4.1, Discussion); inherited from Wolfram/Langton, not proven for this estimator.
  • ad hoc to paper The target scale u_Y and standardization of reservoir features do not distort the epiplexity ranking
    Eq. 7 introduces a posited scale; no invariance analysis is given.
invented entities (1)
  • Learnable novelty (the learnable part of cumulative surprise, operationalized as S_φ) independent evidence
    purpose: Central quantity of the theory; used as both measure and objective across all experiments.
    It has falsifiable handles (ECA ordering, MNIST cluster accuracy, RL exploration stability) even though the operational estimator is heuristic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Intelligence from Learnable Novelty." pith.science (2026). https://pith.science/paper/WNVT3I7E

@misc{pith2026260718433,
  author       = {Pith},
  title        = {Pith review of: Intelligence from Learnable Novelty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WNVT3I7E}},
  note         = {Machine review of arXiv:2607.18433}
}
read the original abstract

Intelligence appears under different names in different fields: as data compression in statistics and machine learning, as universal computation in dynamical systems, and as adaptive behavior in agents. Each field carries its own objective, and the two most influential drives often fail in mirror image: novelty search, which seeks surprise, is transfixed by a noisy television screen, while the free-energy principle, which avoids surprise, is most content in a dark room. Both failures have a single cause: each objective treats as one quantity the surprise a learner can convert into knowledge and the surprise it never can. Here we show that the learnable part of that information, which we call learnable novelty, yields the seemingly disparate projections of intelligence, and we give a closed-form estimator of it built on a cheap and differentiable reservoir computer. Used as a measure, with no supervision of any kind, the estimator recovers decades of complexity classification, ranking the Turing-complete rule~110 highest among the elementary cellular automata. Used as an objective, its gradient carries a neural cellular automaton from simple dynamics into a regime of solitons, the traveling, colliding structures by which rule~110 computes, as well as organizes the representation of an image encoder around the ten digit classes of MNIST, fully unsupervised: no label ever enters training. Handed to a reinforcement-learning agent as an intrinsic reward, it supplies the exploration that task rewards lack, improving on the task baseline in nine of ten environments and collapsing in none. Complexity generation, abstraction, and exploration, ordinarily pursued with unrelated objectives in separate fields, thus emerge from ascent on one differentiable quantity, and the projections of intelligence gain a common quantitative footing.

Figures

Figures reproduced from arXiv: 2607.18433 by the authors.

Figure 1
Figure 1. (a) An observer receives a stream one item at a time. It predicts each item before seeing it, is surprised by the difference, and updates itself, so as the stream’s regularities are absorbed the surprise falls, though never below the floor of irreducible noise. The learnable part of the accumulated surprise, in blue, is the epiplexity S ϕ . A noisy TV is all noise, and a dark room offers no surprise at all. (b) The … view at source ↗
Figure 2
Figure 2. S ϕ over all 88 locally unique elementary cellular automata (top fourteen shown, together with the reference rules). Bars are colored by Wolfram class [Wolfram, 2002]: II periodic, III chaotic, IV complex (legend); insets show the space-time diagrams (time downward) of the six reference rules 1, 2, 3, 30, 54, and 110. Each bar is the mean over ten independent draws of the random reservoir and the input ensemble, and… view at source ↗
Figure 3
Figure 3. Inverse design of a one-dimensional NCA by gradient ascent on a single epiplexity scalar. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Unsupervised MNIST encoder trained solely to maximize reservoir epiplexity. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: A pre-activation normalization holds a random reservoir at the edge of chaos. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Final NCA evolution under the direct and residual update rules, each over nine random seeds, with the [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: One-at-a-time robustness of the ECA ranking. In both panels each row varies one estimator hyperparameter [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: The ridge readout against the exact minimizer of the description length, on identical reservoirs and data. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: MLP-reservoir epiplexity on continuous-time systems. [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: One-at-a-time robustness of the MNIST representation experiment. Each cell gives the varied hyperparam [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 1 canonical work pages

  1. [1]

    Understanding intermediate layers using linear classifier probes

    Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016

    arXiv 2016

  2. [2]

    Predictability, complexity, and learning

    William Bialek, Ilya Nemenman, and Naftali Tishby. Predictability, complexity, and learning. Neural Computation, 13 0 (11): 0 2409--2463, 2001

    2001

  3. [3]

    The description length of deep learning models

    L \'e onard Blier and Yann Ollivier. The description length of deep learning models. In Advances in Neural Information Processing Systems, 2018

    2018

  4. [4]

    Exploration by random network distillation

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. In International Conference on Learning Representations, 2019

    2019

  5. [5]

    Semi-Supervised Learning

    Olivier Chapelle, Bernhard Sch \"o lkopf, and Alexander Zien, editors. Semi-Supervised Learning. MIT Press, 2006

    2006

  6. [6]

    Universality in elementary cellular automata

    Matthew Cook. Universality in elementary cellular automata. Complex Systems, 15 0 (1): 0 1--40, 2004

    2004

  7. [7]

    Crutchfield and Karl Young

    James P. Crutchfield and Karl Young. Inferring statistical complexity. Physical Review Letters, 63 0 (2): 0 105--108, 1989

    1989

  8. [8]

    Philip Dawid

    A. Philip Dawid. Present position and potential developments: Some personal views. statistical theory: The prequential approach. Journal of the Royal Statistical Society, Series A, 147 0 (2): 0 278--292, 1984

    1984

Show all 45 references
  1. [9]

    Language modeling is compression

    Gr \'e goire Del \'e tang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hutter, and Joel Veness. Language modeling is compression. In International Conference ...

    2024

  2. [10]

    Daniel C. Dennett. The Intentional Stance. MIT Press, Cambridge, MA, 1987

    1987

  3. [11]

    Zico Kolter, and Andrew Gordon Wilson

    Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, and Andrew Gordon Wilson. From entropy to epiplexity: Rethinking information for computationally bounded intelligence. arXiv preprint arXiv:2601.03220, 2026

    2026

  4. [12]

    The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11 0 (2): 0 127--138, 2010

    Karl Friston. The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11 0 (2): 0 127--138, 2010

    2010

  5. [13]

    Gr \"u nwald

    Peter D. Gr \"u nwald. The Minimum Description Length Principle. MIT Press, 2007

    2007

  6. [14]

    Hinton and Drew van Camp

    Geoffrey E. Hinton and Drew van Camp. Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the Sixth Annual Conference on Computational Learning Theory, pages 5--13, 1993

    1993

  7. [15]

    Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability

    Marcus Hutter. Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer, 2005

    2005

  8. [16]

    Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication

    Herbert Jaeger and Harald Haas. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, 304 0 (5667): 0 78--80, 2004

    2004

  9. [17]

    Klyubin, Daniel Polani, and Chrystopher L

    Alexander S. Klyubin, Daniel Polani, and Chrystopher L. Nehaniv. Empowerment: A universal agent-centric measure of control. In Proceedings of the IEEE Congress on Evolutionary Computation, 2005

    2005

  10. [18]

    Christopher G. Langton. Computation at the edge of chaos: Phase transitions and emergent computation. Physica D: Nonlinear Phenomena, 42 0 (1-3): 0 12--37, 1990

    1990

  11. [19]

    Gradient-based learning applied to document recognition

    Yann LeCun, L \'e on Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998

    1998

  12. [20]

    Joel Lehman and Kenneth O. Stanley. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation, 19 0 (2): 0 189--223, 2011

    2011

  13. [21]

    Real-time computing without stable states: A new framework for neural computation based on perturbations

    Wolfgang Maass, Thomas Natschl \"a ger, and Henry Markram. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14 0 (11): 0 2531--2560, 2002

    2002

  14. [22]

    Growing neural cellular automata

    Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, and Michael Levin. Growing neural cellular automata. Distill, 2020. doi:10.23915/distill.00023

    2020 doi

  15. [23]

    Neural weight norm = K olmogorov complexity

    Tiberiu Musat. Neural weight norm = K olmogorov complexity. arXiv preprint arXiv:2605.10878, 2026

    2026 arXiv

  16. [24]

    Norman H. Packard. Adaptation toward the edge of chaos. In Dynamic Patterns in Complex Systems, pages 293--301. World Scientific, 1988

    1988

  17. [25]

    Efros, and Trevor Darrell

    Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International Conference on Machine Learning, pages 2778--2787, 2017

    2017

  18. [26]

    Howard H. Pattee. The physics of symbols: bridging the epistemic cut. BioSystems, 60 0 (1--3): 0 5--21, 2001

    2001

  19. [27]

    Exponential expressivity in deep neural networks through transient chaos

    Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli. Exponential expressivity in deep neural networks through transient chaos. In Advances in Neural Information Processing Systems, volume 29, 2016

    2016

  20. [28]

    Stable-baselines3: Reliable reinforcement learning implementations

    Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22 0 (268): 0 1--8, 2021

    2021

  21. [29]

    Modeling by shortest data description

    Jorma Rissanen. Modeling by shortest data description. Automatica, 14 0 (5): 0 465--471, 1978

    1978

  22. [30]

    Life Itself: A Comprehensive Inquiry into the Nature, Origin, and Fabrication of Life

    Robert Rosen. Life Itself: A Comprehensive Inquiry into the Nature, Origin, and Fabrication of Life. Columbia University Press, New York, 1991

    1991

  23. [31]

    Empowerment: An introduction

    Christoph Salge, Cornelius Glackin, and Daniel Polani. Empowerment: An introduction. In Guided Self-Organization: Inception, pages 67--114. Springer, 2014

    2014

  24. [32]

    u rgen Schmidhuber. Ultimate cognition \`a la G \

    J \"u rgen Schmidhuber. Ultimate cognition \`a la G \"o del. Cognitive Computation, 1 0 (2): 0 177--193, 2009

    2009

  25. [33]

    Formal theory of creativity, fun, and intrinsic motivation (1990--2010)

    J \"u rgen Schmidhuber. Formal theory of creativity, fun, and intrinsic motivation (1990--2010). IEEE Transactions on Autonomous Mental Development, 2 0 (3): 0 230--247, 2010

    1990

  26. [34]

    Deep information propagation

    Samuel S Schoenholz, Justin Gilmer, Surya Ganguli, and Jascha Sohl-Dickstein. Deep information propagation. In International Conference on Learning Representations, 2017

    2017

  27. [35]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  28. [36]

    Solomonoff

    Ray J. Solomonoff. A formal theory of inductive inference. part i. Information and Control, 7 0 (1): 0 1--22, 1964

    1964

  29. [37]

    The dark room problem

    Zekun Sun and Chaz Firestone. The dark room problem. Trends in Cognitive Sciences, 24 0 (5): 0 346--348, 2020. doi:10.1016/j.tics.2020.02.006

    2020 doi

  30. [38]

    A. N. Tikhonov and V. Y. Arsenin. Solutions of Ill-Posed Problems. Winston, 1977

    1977

  31. [39]

    Pereira, and William Bialek

    Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method. In Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing, pages 368--377, 1999

    1999

  32. [40]

    Visualizing data using t- SNE

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t- SNE . Journal of Machine Learning Research, 9: 0 2579--2605, 2008

    2008

  33. [41]

    Observing Systems

    Heinz von Foerster. Observing Systems. Intersystems Publications, Seaside, CA, 1981

    1981

  34. [42]

    Wissner-Gross and Cameron E

    Alexander D. Wissner-Gross and Cameron E. Freer. Causal entropic forces. Physical Review Letters, 110 0 (16): 0 168702, 2013

    2013

  35. [43]

    Universality and complexity in cellular automata

    Stephen Wolfram. Universality and complexity in cellular automata. Physica D: Nonlinear Phenomena, 10 0 (1-2): 0 1--35, 1984

    1984

  36. [44]

    A New Kind of Science

    Stephen Wolfram. A New Kind of Science. Wolfram Media, 2002

    2002

  37. [45]

    Learning diverse and discriminative representations via the principle of maximal coding rate reduction

    Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Yi Ma. Learning diverse and discriminative representations via the principle of maximal coding rate reduction. In Advances in Neural Information Processing Systems, volume 33, 2020

    2020

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.