Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvement

Bing Yuan; Jiang Zhang; Qian Zhang

REVIEW 3 major objections 4 minor 16 references

Sustainable recursive self-improvement requires a system that can simulate and rewrite itself, not just tweak its outputs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 20:26 UTC pith:MULTMHZQ

load-bearing objection Clean recursion-theory packaging of an RSI barrier that is useful as framing but overstates necessity as a theorem. the 3 major comments →

arxiv 2607.04277 v1 pith:MULTMHZQ submitted 2026-07-05 physics.soc-ph cs.AI

Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvement

Jiang Zhang , Bing Yuan , Qian Zhang This is my paper

classification physics.soc-ph cs.AI

keywords self-referencelarge language modelsrecursive self-improvementintrospectionKleene’s second recursion theoremvon Neumann complexity thresholdmetacognitionAI self-evolution

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Self-evolving AI systems keep plateauing or collapsing after a few rounds of self-tinkering. This paper argues that the missing ingredient is the same kind of self-reference that lets a von Neumann automaton reproduce without getting simpler: a program that can simulate its own operations, evaluate them, and rewrite its own code. Kleene’s second recursion theorem guarantees that such “introspective” programs exist as fixed points. Current large language models only show quasi-introspection—partial self-knowledge and shallow self-critique—because they cannot fully access their own weights, run unbounded self-simulation, or close a true fixed-point loop. Crossing that introspection threshold would turn temporary self-improvement into a sustained recursive sequence; failing to cross it leaves systems dependent on external benchmarks and prone to degeneration.

Core claim

A system can sustain recursive self-improvement—producing a sequence of successors with monotonically rising performance—only if its starting program is an introspective self-improver: a Kleene fixed point of the composite operator that simulates the program for a finite horizon, evaluates the result, and rewrites the code. Systems without that capacity are limited to blind self-modification that saturates or degrades. Current LLMs remain below this introspection threshold.

What carries the argument

The introspection threshold (and the introspective self-improvement program): a Kleene fixed-point program that embeds a simulator of itself for T steps, an evaluator, and a modifier, so that running the program is equivalent to simulating, scoring, and rewriting its own source.

Load-bearing premise

That the mathematical existence of a self-simulating, self-rewriting fixed-point program is the same barrier that actually causes today’s LLM self-improvement loops to saturate and collapse—rather than a useful but non-necessary analogy.

What would settle it

Build or observe an LLM-scale system that, without full self-access or unbounded self-simulation, still produces a long open-ended sequence of genuine capability gains under fixed external evaluation; if such a sequence reliably continues past known saturation points, the necessity claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Clean recursion-theory packaging of an RSI barrier that is useful as framing but overstates necessity as a theorem.

read the letter

This paper’s real move is packaging: it takes Kleene’s second recursion theorem, Cutland’s introspection programs, von Neumann’s self-reproducing automata, and Schmidhuber’s Gödel Machine, then names an “introspection threshold” as the analogue of von Neumann’s critical size for LLM recursive self-improvement. Existence of a fixed-point program that simulates itself for finite T, evaluates, and rewrites is standard and correctly applied. The literature survey on self-evolving agents and LLM metacognition is broad and mostly fair; the four functional properties and the structural-gap section (no weight access, feedforward/TC0 limits, self-report without privileged access) line up with the cited work. The appendix mapping to the Gödel Machine is careful rather than hand-wavy.

What is new is the claim that this capacity is the complexity barrier that explains empirical saturation and model collapse, plus the quasi-introspection label for current systems. That necessity half is analogy, not proof. The thesis is written as if-and-only-if, but the construction only shows sufficiency. External verifiers, population archives, co-evolving evaluators, and approximate external self-models are acknowledged later and never ruled out. Saturation data are consistent with the story; they do not discriminate it from weaker explanations. Free parameters (T, design of M∘V) stay free. No experiments, no formal verification, no code.

Soft spots are real but proportional: overclaim on necessity, interpretive leap from recursion theory to Transformer practice, and invented terminology that does most of the rhetorical work. The math that is there is checkable and not circular relative to Kleene. Citation pattern is appropriate; self-citation is light.

This is for people who care about foundations of RSI, agent architecture, or alignment invariants—not for someone looking for a new theorem or an empirical result. I would bring it to a reading group if the group is already in that conversation. A serious editor should send it to referees as a framing/position paper, with the expectation that the threshold thesis gets softened from barrier theorem to sufficient-condition hypothesis. Worth engaging if you work on self-improving agents; not a must-cite for everyone.

Referee Report

3 major / 4 minor

Summary. The paper argues that sustainable recursive self-improvement (RSI) in LLMs requires crossing an 'introspection threshold'—a functional analogue of von Neumann’s critical complexity for non-degenerative self-reproduction. Drawing on Kleene’s Second Recursion Theorem, it constructs an introspective program (a fixed point of a total function f_T that simulates its own code for horizon T) and then an introspective self-improver ω as the fixed point of M∘V∘f_T. Current LLMs are shown via literature survey to exhibit only quasi-introspection (partial self-modeling, unreliable self-evaluation, shallow self-modification) due to missing complete self-access, feed-forward Transformer depth, and TC^{0} limits on fixed-point iteration. Architectural paths (externalized self-models, recurrent nets, approximate reflexivity) and safety implications are outlined.

Significance. If the necessity half of the Introspection Threshold Thesis holds, the paper supplies a recursion-theoretic criterion that cleanly separates degenerative from compounding self-improvement and explains empirical saturation and model-collapse phenomena. The constructive existence proof via Kleene is standard yet cleanly applied, the four functional properties of introspection are operationally useful for evaluation, and the safety discussion correctly flags the alignment risk of agents that can rewrite their own evaluation machinery. Even if necessity is weakened to sufficiency, the formalization and the quasi-introspection survey remain valuable interdisciplinary contributions linking computability, complex systems, and LLM self-evolution.

major comments (3)

[§3.4.3 Introspection Threshold Thesis] §3.4.3 (Introspection Threshold Thesis) and the abstract claim that sustained RSI occurs only if S0 is an introspective Kleene fixed point of M∘V∘f_T. Existence follows immediately once f_T is total (§3.3–3.4) and is uncontroversial. Necessity, however, is established solely by analogy to von Neumann’s A+B+C+ϕ architecture and by observing that current LLMs lack complete self-access and recurrent depth (§5.1). No argument rules out alternative non-degenerative routes—external verifiers, population archives (DGM), co-evolving evaluators (RQGM), or approximate externalized self-models (§5.2.1)—that never embed a complete functional self-description inside a single fixed-point program. Soften the thesis to a sufficient condition, or supply a concrete impossibility argument showing that any monotonically improving sequence must eventually construct such a fixed point.
[§3.4 / Appendix] The free parameters T (simulation horizon) and the design of the composite operator M∘V are left unconstrained (§3.3–3.4 and Appendix). Because Rice’s theorem already precludes a total, unrestricted V, the paper correctly restricts evaluation to a finite horizon; yet the claim that the resulting sequence is ‘sustained’ and ‘monotonically increasing’ then depends on an unstated assumption that successive finite-horizon improvements compound rather than cycle or plateau. A short formal statement of the conditions on M and V that guarantee monotonicity (or an explicit counter-example when they fail) is needed for the threshold to be well-defined.
[§5.1.1] §5.1.1 asserts that continuous parameter spaces preclude stable reflexivity because representation errors diverge under infinite recursion. This is plausible but not demonstrated; continuous approximations (neural Quines, synergistic cores) are later offered as possible paths (§5.2.3). Either prove that any continuous self-model is unstable under the required nesting depth, or qualify the claim so that it does not contradict the constructive suggestions that follow.

minor comments (4)

[throughout] Several typographical slips: ‘reflextive’ (p. 6), ‘man-made automata’ repeated, and future-dated references (June 2026, arXiv 2607) that will need updating for archival publication.
[Figures 1, 3] Figure 1 and Figure 3 are referenced as conceptual maps but never described in sufficient detail for a reader to reconstruct the claimed mappings without the accompanying text; a short caption expansion would help.
[§3.5.2 / §4] The four functional properties in §3.5.2 are useful, yet the subsequent empirical survey (§4) maps them only loosely; a short table explicitly scoring each cited paper against the four properties would tighten the ‘quasi-introspection’ diagnosis.
[§5.1.2] Citation of Merrill & Sabharwal (2023) on TC^{0} is correct for log-precision Transformers, but the paper should note that chain-of-thought / autoregressive unrolling already supplies a limited form of recurrence at inference time; the architectural claim in §5.1.2 therefore needs a one-sentence qualification.

Circularity Check

1 steps flagged

Mild definitional packaging of necessity; Kleene existence proof is non-circular constructive math, not a fitted or self-citing prediction.

specific steps

self definitional [Section 3.4.3 Introspection Threshold Thesis (and 3.4.2 construction)]
"Introspection Threshold Thesis. A computational system S can achieve sustained recursive self-improvement (i.e., a sequence of self-modifications S0, S1, S2, . . . with monotonically increasing performance on a given objective) if S0 is introspective self-improved. Systems lacking introspection are confined to blind self-modification, which converges to suboptimal fixed points or degrades."

Sufficiency is obtained by defining f ≡ M∘V∘f_T so that the Kleene fixed point ω evaluates and rewrites itself by construction (3.4.2); that is valid constructive existence, not a prediction. Necessity is packaged definitionally: 'lacking introspection' is equated with confinement to 'blind self-modification,' so the only-if direction largely renames the complement of the defined capacity rather than ruling out alternative RSI routes. Mild framing, not a full circular reduction.

full rationale

The paper's load-bearing formal step is a standard constructive application of Kleene's Second Recursion Theorem: define a total f that includes simulation, evaluation, and modification (f ≡ M∘V∘f_T), then obtain a fixed point ω that self-simulates and rewrites. That existence claim is independent of the LLM empirical material and is not circular relative to Kleene/Cutland; it is how fixed-point constructions work. The sufficiency half of the Introspection Threshold Thesis is therefore true by construction of f, which is legitimate mathematics rather than a disguised fit or self-citation. The necessity half (systems lacking introspection are confined to 'blind' self-modification that degrades) is asserted by analogy to von Neumann and by defining the complement of introspection as 'blind,' without an independent proof that no non-reflexive architecture (external verifiers, population archives, co-evolving evaluators) can produce a monotonic RSI sequence. That is definitional framing and overclaim of necessity, not a reduction of a prediction to fitted inputs or a load-bearing self-citation chain. No uniqueness theorem is imported from the authors; no parameter is fitted and re-presented as prediction; citations to Merrill–Sabharwal, Schmidhuber, Cutland, etc. are external. Score 2 reflects only the mild tautological packaging of the necessity clause, not a circular derivation of the central existence result.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 3 invented entities

The paper rests on standard recursion theory plus a strong domain analogy. Free parameters are mainly the simulation bound T and design choices for M and V. Invented entities are conceptual labels (introspection threshold, quasi-introspection) rather than physical objects. The load-bearing non-standard move is treating the Kleene fixed-point construction as the necessary complexity threshold for practical LLM RSI.

free parameters (2)

simulation horizon T
Finite T is required so f_T is total and Kleene applies; choice of T is free and determines how much of the program’s behavior is visible to V/M. No unique value is derived.
composite operator M∘V design
Evaluation and modification operators are left as arbitrary total computable functions (annealing, GA, RL, etc.); their form is not fixed by the theory and must be chosen by the designer.

axioms (5)

standard math Kleene’s Second Recursion Theorem: for any total computable f there exists e with φ_e ≃ φ_f(e).
Used in §3.2–3.4 to guarantee introspective and self-improving fixed points.
ad hoc to paper Von Neumann’s critical size of complication for non-degenerative self-reproduction has a functional analogue for AI self-improvement.
Authors invent the label ‘complexity threshold’ and assert the analogy carries over from physical constructors to symbolic self-simulation (§1, §3.1, §3.4).
domain assumption V (evaluation) and M (modification) can be taken as total computable functions over a bounded simulation horizon.
Required to apply the recursion theorem; Rice’s theorem is sidestepped only by bounding T (§3.4, §6.2).
domain assumption Log-precision Transformers are bounded by uniform TC0 and cannot perform open-ended fixed-point iteration.
Cited from Merrill & Sabharwal 2023; used in §4.1 and §5.1.2 as a structural barrier to true introspection.
ad hoc to paper Complete parameter-level self-access (or an equivalent faithful functional self-model) is required for true introspection in LLMs.
Stated in §5.1.1; continuous weights cannot be embedded as a discrete Quine-style description without architectural change.

invented entities (3)

introspection threshold no independent evidence
purpose: Name the critical complexity above which recursive self-improvement is sustainable rather than degenerative.
Central coined concept of the paper; analogous to von Neumann’s critical size but defined for functional self-simulation.
quasi-introspection no independent evidence
purpose: Label fragmentary LLM metacognition that falls short of the formal four-property definition.
Organizes the empirical survey in §4; not independently measured outside the paper’s criteria.
introspective self-improvement program (ω / S0) no independent evidence
purpose: Fixed-point program that simulates, evaluates, and modifies its own source within horizon T.
Constructed via Kleene in §3.4; existence is mathematical, but no concrete implementation is given.

pith-pipeline@v1.1.0-grok45 · 24552 in / 3647 out tokens · 44325 ms · 2026-07-11T20:26:10.580193+00:00 · methodology

0 comments

read the original abstract

The pursuit of self-evolving AI raises a critical question: when is autonomous self-improvement sustainable rather than degenerative? Drawing an analogy to von Neumann's complexity threshold for self-reproducing automata, we argue that sustainable recursive self-improvement in Large Language Models (LLMs) requires a functional analogue: introspection -- the system's capacity to simulate its own operations and target modifications. Grounded in Kleene's Second Recursion Theorem, we demonstrate the theoretical existence of such introspective programs. However, an empirical review reveals that while current LLMs exhibit quasi-introspection (e.g., partial metacognition), they fall short of true introspection due to structural bottlenecks: a lack of complete self-access, the feedforward nature of the Transformer, and computational class constraints that prevent fixed-point iteration. We conclude by outlining architectural paths to cross this complexity threshold and discussing the associated safety implications.

Figures

Figures reproduced from arXiv: 2607.04277 by Bing Yuan, Jiang Zhang, Qian Zhang.

**Figure 2.** Figure 2: The Architecture and Replication Mechanisms of von Neumann’s Self-Reproducing Automata [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Figure 3: Self-reference: from Quine to introspective Self-improvement [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

**Figure 4.** Figure 4: Introspective Self-Improvement: Construction and Recursive Sequence [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

16 extracted references · 1 canonical work pages

[1]

17 Swarma ResearchA PREPRINT Eliezer Yudkowsky

doi:10.1016/S0065-2458(08)60418-0. 17 Swarma ResearchA PREPRINT Eliezer Yudkowsky. Levels of organization in general intelligence. In Ben Goertzel and Cassio Pennachin, editors, Artificial General Intelligence. Springer,

doi:10.1016/s0065-2458(08)60418-0
[2]

Reflexion: Language agents with verbal reinforcement learning

Noah Shinn, Federico Cassano, Edward Berman, et al. Reflexion: Language agents with verbal reinforcement learning. InNeurIPS 2023,

2023
[3]

Self-refine: Iterative refinement with self-feedback

Aman Madaan, Niket Tandon, Prakhar Gupta, et al. Self-refine: Iterative refinement with self-feedback. InNeurIPS 2023,

2023
[4]

Large language models can self-improve

Jiaxin Huang, Shixiang Shane Gu, Le Hou, et al. Large language models can self-improve. InEMNLP 2023,

2023
[5]

Elvis Dohmatob et al

URL https://www.nature.com/articles/s41586-024-07566-y. Elvis Dohmatob et al. Model collapse demystified. InICLR 2025, 2025a. URL https://proceedings.iclr.cc/ paper_files/paper/2025/file/284afdc2309f9667d2d4fb9290235b0c-Paper-Conference.pdf. Daron Acemoglu, Dingwen Kong, and Asuman Ozdaglar. AI, human cognition and knowledge collapse, February

2025
[6]

Eric Zelikman et al

Introduces the GENOME / GENOME+ framework. Eric Zelikman et al. Self-taught optimizer (stop): Recursively self-improving code generation. InCOLM 2024,

2024
[7]

Recursive introspection: Teaching language model agents how to self-improve

Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. Recursive introspection: Teaching language model agents how to self-improve. InAdvances in Neural Information Processing Systems 37 (NeurIPS 2024),

2024
[8]

Hao Lu, Yong Liu, Jiachen Xu, Guoshun Nan, et al

URLhttps://arxiv.org/abs/2606.26294. Hao Lu, Yong Liu, Jiachen Xu, Guoshun Nan, et al. Auditing meta-cognitive hallucinations in reasoning large language models. InNeurIPS 2025,

Pith/arXiv arXiv 2025
[9]

Model collapse demystified: The case of regression

18 Swarma ResearchA PREPRINT Elvis Dohmatob et al. Model collapse demystified: The case of regression. InICLR 2025, 2025b. James P. Crutchfield. Space-time dynamics in video feedback.Physica D: Nonlinear Phenomena, 10(1):229–245,

2025
[10]

doi:https://doi.org/10.1016/0167-2789(84)90264-1

ISSN 0167-2789. doi:https://doi.org/10.1016/0167-2789(84)90264-1. URL https://www.sciencedirect.com/ science/article/pii/0167278984902641. Stephen Cole Kleene. On notation for ordinal numbers.Journal of Symbolic Logic, 3(4):150–155,

work page doi:10.1016/0167-2789(84)90264-1
[11]

Do i know this entity? knowledge awareness and hallucinations in language models

Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan, and Neel Nanda. Do i know this entity? knowledge awareness and hallucinations in language models. InICLR 2025,

2025
[12]

Universal transformers

Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. In ICLR 2019,

2019
[13]

Perceiver: General perception with iterative attention

Andrew Jaegle et al. Perceiver: General perception with iterative attention. InICML 2021,

2021
[14]

arXiv:1803.05859. A. I. Luppi, P. A. M. Mediano, F. E. Rosas, et al. A synergistic core for human brain evolution and cognition.Nature Neuroscience, 25:771–782,

Pith/arXiv arXiv
[15]

Thórisson, et al

Eric Nivel, Kristinn R. Thórisson, et al. Bounded recursive self-improvement. InAGI 2013,

2013
[16]

8 Appendix 8.1 Informal Alignment with Gödel Machine In this appendix, we establish an informal argument demonstrating how the recursive self-improvement sequenceS can be aligned with the Gödel Machine (GM) framework, an architecture whose global optimality has been formally guaranteed [Schmidhuber, 2003]. First, the program index (source code) ω, which e...

2003

[1] [1]

17 Swarma ResearchA PREPRINT Eliezer Yudkowsky

doi:10.1016/S0065-2458(08)60418-0. 17 Swarma ResearchA PREPRINT Eliezer Yudkowsky. Levels of organization in general intelligence. In Ben Goertzel and Cassio Pennachin, editors, Artificial General Intelligence. Springer,

doi:10.1016/s0065-2458(08)60418-0

[2] [2]

Reflexion: Language agents with verbal reinforcement learning

Noah Shinn, Federico Cassano, Edward Berman, et al. Reflexion: Language agents with verbal reinforcement learning. InNeurIPS 2023,

2023

[3] [3]

Self-refine: Iterative refinement with self-feedback

Aman Madaan, Niket Tandon, Prakhar Gupta, et al. Self-refine: Iterative refinement with self-feedback. InNeurIPS 2023,

2023

[4] [4]

Large language models can self-improve

Jiaxin Huang, Shixiang Shane Gu, Le Hou, et al. Large language models can self-improve. InEMNLP 2023,

2023

[5] [5]

Elvis Dohmatob et al

URL https://www.nature.com/articles/s41586-024-07566-y. Elvis Dohmatob et al. Model collapse demystified. InICLR 2025, 2025a. URL https://proceedings.iclr.cc/ paper_files/paper/2025/file/284afdc2309f9667d2d4fb9290235b0c-Paper-Conference.pdf. Daron Acemoglu, Dingwen Kong, and Asuman Ozdaglar. AI, human cognition and knowledge collapse, February

2025

[6] [6]

Eric Zelikman et al

Introduces the GENOME / GENOME+ framework. Eric Zelikman et al. Self-taught optimizer (stop): Recursively self-improving code generation. InCOLM 2024,

2024

[7] [7]

Recursive introspection: Teaching language model agents how to self-improve

Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. Recursive introspection: Teaching language model agents how to self-improve. InAdvances in Neural Information Processing Systems 37 (NeurIPS 2024),

2024

[8] [8]

Hao Lu, Yong Liu, Jiachen Xu, Guoshun Nan, et al

URLhttps://arxiv.org/abs/2606.26294. Hao Lu, Yong Liu, Jiachen Xu, Guoshun Nan, et al. Auditing meta-cognitive hallucinations in reasoning large language models. InNeurIPS 2025,

Pith/arXiv arXiv 2025

[9] [9]

Model collapse demystified: The case of regression

18 Swarma ResearchA PREPRINT Elvis Dohmatob et al. Model collapse demystified: The case of regression. InICLR 2025, 2025b. James P. Crutchfield. Space-time dynamics in video feedback.Physica D: Nonlinear Phenomena, 10(1):229–245,

2025

[10] [10]

doi:https://doi.org/10.1016/0167-2789(84)90264-1

ISSN 0167-2789. doi:https://doi.org/10.1016/0167-2789(84)90264-1. URL https://www.sciencedirect.com/ science/article/pii/0167278984902641. Stephen Cole Kleene. On notation for ordinal numbers.Journal of Symbolic Logic, 3(4):150–155,

work page doi:10.1016/0167-2789(84)90264-1

[11] [11]

Do i know this entity? knowledge awareness and hallucinations in language models

Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan, and Neel Nanda. Do i know this entity? knowledge awareness and hallucinations in language models. InICLR 2025,

2025

[12] [12]

Universal transformers

Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. In ICLR 2019,

2019

[13] [13]

Perceiver: General perception with iterative attention

Andrew Jaegle et al. Perceiver: General perception with iterative attention. InICML 2021,

2021

[14] [14]

arXiv:1803.05859. A. I. Luppi, P. A. M. Mediano, F. E. Rosas, et al. A synergistic core for human brain evolution and cognition.Nature Neuroscience, 25:771–782,

Pith/arXiv arXiv

[15] [15]

Thórisson, et al

Eric Nivel, Kristinn R. Thórisson, et al. Bounded recursive self-improvement. InAGI 2013,

2013

[16] [16]

8 Appendix 8.1 Informal Alignment with Gödel Machine In this appendix, we establish an informal argument demonstrating how the recursive self-improvement sequenceS can be aligned with the Gödel Machine (GM) framework, an architecture whose global optimality has been formally guaranteed [Schmidhuber, 2003]. First, the program index (source code) ω, which e...

2003