Pith. sign in

REVIEW 3 major objections 6 minor 120 references

Strong play on interactive abstraction games depends on deciding when to build and use a model, not only on how accurately that model replays the past.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 12:13 UTC pith:N6TRYE45

load-bearing objection Strong systems paper with a real negative result: transition-perfect repair loses to selective model use, but the policy ranking sits on n=1 public runs that include development games. the 3 major comments →

arxiv 2607.28287 v1 pith:N6TRYE45 submitted 2026-07-30 cs.AI cs.CVcs.SC

Tycho: Active Abstraction with Programmatic World Models for ARC-AGI-3

Jens Lehmann , Andrei Aioanei , Sahar Vahdati This is my paper
classification cs.AI cs.CVcs.SC
keywords active abstractionprogrammatic world modelsARC-AGI-3coding agentsskill acquisitionmodel-based agentsinteractive reasoningmetareasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

ARC-AGI-3 turns skill acquisition into a scored interaction problem: an agent must infer an unfamiliar game’s rules, hidden state, and goal while every move counts. This paper claims the joint hard problem is active abstraction—building a testable model from costly interaction and deciding when constructing, repairing, consulting, or bypassing it is worth the cost. It formalizes each game as a parameterized rendered deterministic Moore machine and introduces Tycho, a coding-agent system that keeps typed interaction history and can write free-form executable Python hypotheses with state, dynamics, rendering, and outcomes. In a matched comparison of four model-maintenance policies, actor-requested delegation to a specialist builder scored highest; with that policy, two frontier models completed all 183 public levels at 100 Relative Human Action Efficiency. Automatic repair after verification failures made simulators much better at replaying observed transitions, yet played worse—showing that transition match is not the same as identifying the objective or improving the next action.

Core claim

The paper establishes that strong ARC-AGI-3 play requires active abstraction, not transition accuracy alone. Under matched public-set runs, actor-requested delegation to a model builder reached the highest mean Relative Human Action Efficiency among four policies (88.49). Automatic verification-triggered repair produced far higher accepted transition match but only 83.07. With the selected policy, two frontier models both reached 100.00 RHAE and finished all 183 levels, one using 61% fewer scored actions than the aggregate official human baselines.

What carries the argument

Active abstraction, implemented by Tycho’s optional programmatic world models over rendered deterministic Moore machines. A free-form executable hypothesis exposes initialization, transition, partial rendering, and outcome classification; verification and planning thread state from typed history; four policies vary who builds or repairs the model and when, while only the actor may commit environment actions.

Load-bearing premise

The policy ranking and perfect scores rest on one run of each setup on the same 25 public games, several of which helped shape the system during development.

What would settle it

Freeze the harness, then run repeated independent evaluations on newly released games never used in development; if actor-requested delegation no longer beats automatic repair, or perfect public-set scores collapse off that set, the central system claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Transition match is an incomplete success metric when goals are hidden and every exploratory action is scored.
  • Making model construction optional and actor-gated can outperform always-on repair under the same inference budget.
  • Free-form executable hypotheses can support planning without a fixed object schema if history, verification, and partial rendering are explicit.
  • Action-efficiency headlines can hide large unscored inference costs that still dominate total resource use.
  • The same limited-feedback loop can be aimed one level up: from task models to revisions of the agent harness itself.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Metareasoning over whether to invoke a simulator may matter as much as simulator fidelity in other scarce-interaction settings such as UI automation or lab protocols.
  • Renderers that abstain on unknown pixels offer a practical middle path between full observation models and pure latent planners.
  • If repair overfits dynamics while missing objectives, agents may need a separate budget for goal hypotheses rather than one world-model loop.
  • Public-set saturation at 100 RHAE will force evaluation toward held-out games and multi-run statistics sooner than architecture claims alone would.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper formalizes ARC-AGI-3 games as parameterized rendered deterministic Moore machines with typed interaction history, and presents Tycho, a coding-agent harness that can construct, verify, plan with, repair, or bypass free-form Python world models during scored interaction. In a matched four-policy comparison on all 25 public games with Claude Opus 4.8, actor-requested builder delegation (Orchestrator) achieves the highest observed mean RHAE (88.49), while automatic verification-triggered repair (Trigger) yields much higher accepted transition match but only 83.07 RHAE. With Orchestrator fixed, GPT-5.6 Sol and Opus 5 both reach 100.00 RHAE and complete all 183 levels; Opus 5 uses substantially fewer scored actions than official human baselines. The authors frame the joint problem as active abstraction: building a testable model from costly interaction and deciding when modeling is worth its cost.

Significance. If the results hold, this is a substantial contribution to interactive abstraction and agent world-model research. Strengths include a clean formal setting that separates hidden state, transient emissions, and terminal outcomes; an explicit, auditable comparison of model-allocation policies rather than only model synthesis; careful separation of transition match from gameplay; public competition-replay scorecards; rich pre-action and end-of-run diagnostics; human-replay midranks; cost normalization against contemporaneous systems; and strong qualitative case studies (cd82, tu93, sk48, tr87, ka59, ls20) that make the active-abstraction thesis concrete. The dissociation between high transition fidelity and weaker play is an important, falsifiable systems finding for the world-model agenda. Open-source release and protocol transparency further raise the work’s value as a reference point for ARC-AGI-3 agent evaluation.

major comments (3)
  1. [§5.1, Table 3, Figure 8] §5.1 and Table 3: The load-bearing scientific claim is not only that frontier Orchestrator runs hit 100 RHAE, but that actor-requested delegation is the best of four matched policies and that this gap, against Trigger’s much higher transition match, supports active abstraction over transition accuracy alone. That ranking rests on one stochastic trajectory per policy on the same 25 public games. The Orchestrator−Trigger paired delta is +5.43 with a game-bootstrap interval that includes zero ([−6.73, 16.44]), many games tie at saturation, and some reverse the ranking (e.g., sk48 in Fig. 8, where Trigger completes the game and Orchestrator does not). The observed ordering is still informative, but the manuscript should not treat policy selection as settled without either (i) repeated runs on at least a substantial subset under the same budgets, or (ii) a clearly demoted claim language plus
  2. [§5, Limitations] §5 opening and Limitations: Five games (tr87, vc33, r11l, bp35, ft09) informed harness development and remain inside the 25-game scorecards used both to select Orchestrator and to headline 100 RHAE. The paper correctly flags that results characterize a fixed public testbed, not unseen-game generalization. For the central policy-comparison claim this is load-bearing: please report full matched-policy and selected-policy aggregates with the five development games excluded (and, if sample size allows, with the three early-preview games excluded as a contamination sensitivity). Without those numbers in the main results, it is hard to judge how much of the Orchestrator advantage and ceiling performance is carried by games that shaped the harness.
  3. [§4.4, Appendix C.2, Table 7] §4.4 / Appendix C.2 and Table 7: Trigger’s verifier gates (accepted match < 0.999, coverage < 0.75, lifecycle builder calls) are free parameters fixed during development and not ablated. Because Trigger is the foil that supplies high transition match with lower RHAE, the active-abstraction argument depends on Trigger being a reasonable automatic-repair policy rather than an over-calling straw man. A minimal sensitivity—e.g., fewer lifecycle invocations or a looser match threshold on a subset—or an explicit argument that the chosen gates match the intended ‘repair when unusable’ semantics would make the Orchestrator vs Trigger contrast more convincing.
minor comments (6)
  1. [Abstract, Table 4] Abstract and §5.1: ‘midranks of 98.5 and 100.0’ vs Table 4’s 98.47 and 100.00—harmonize rounding and state once that midrank is game-balanced over uneven per-game human sample sizes (10–54).
  2. [§2] Related Work cost comparisons are useful but dense; a short table of protocol tags (single-pass vs conditional rerun, action limit, model) next to RHAE/midrank/cost would reduce the risk of readers equating headline numbers across incompatible protocols.
  3. [§3.1–3.2] Definition 1 / Figure 2: The non-Markov scrolling example is excellent; consider explicitly cross-referencing Proposition 1 there so the formal insufficiency claim is tied to the figure on first reading.
  4. [Table 2, Table 10] Table 2: Clarify that the $750 / $1,500 inference ceilings and 3500 LM-call budgets are harness stops, and that several matched-study failures are budget-censored (already partly in Table 10)—a one-sentence pointer in the main text would help.
  5. [§5.2] §5.2 sk48 case: Strong counterexample to ‘accurate dynamics ⇒ good play’; add the Trigger action count on sk48 beside Orchestrator’s 314 failed actions for a sharper paired contrast.
  6. [Abstract / front matter] Minor polish: spacing in ‘at88.49’, ‘reaches100.00RHAE’, and similar run-on number/unit strings in the front matter; fix for camera-ready.

Circularity Check

0 steps flagged

No derivation circularity: RHAE, completion, and policy ranking are external empirical outcomes, not quantities forced by Tycho’s definitions or self-citation.

full rationale

Tycho is an empirical systems paper. Its load-bearing claims are (i) a matched four-policy comparison on the 25 public ARC-AGI-3 games and (ii) the observation that higher accepted transition match (Trigger) does not yield higher RHAE than actor-requested delegation (Orchestrator). Official RHAE, level completion, scored actions, and human-replay midranks are defined by the ARC Prize protocol and external human baselines, not by Tycho’s world-model loss, verifier thresholds, or fitted objectives. The rendered Moore-machine formalization is a modeling contract for evidence and verification; it does not algebraically entail any policy’s score. “Active abstraction” names the empirical allocation problem (when to build/use/bypass), not a self-definitional identity between transition match and play quality—the paper’s point is precisely that those come apart. Related-work self-citations (WorldCoder, TheoryCoder, etc.) are background, not uniqueness theorems that force the Orchestrator ranking. Development on five public games and n=1 runs are generalization/reliability threats, not circular reductions of prediction to fitted input. No step reduces Eq./metric Y to input X by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

This is an empirical systems paper. Load-bearing background is the ARC-AGI-3 environment/scoring contract and the assumption that discrete programmatic simulators are an adequate hypothesis class. Free parameters are engineering thresholds and evaluation choices (verifier gates, budgets, single-run protocol), not physical constants. Invented entities are definitional frameworks (active abstraction, rendered Moore-machine game model, Tycho policies), not new physical objects.

free parameters (4)
  • Trigger verifier gates (accepted match < 0.999; coverage < 0.75) = 0.999 match; 0.75 coverage
    Fixed during harness development and used unchanged; not swept. Directly controls when automatic repair fires and thus the Trigger policy’s compute/behavior.
  • LM-call budget per game and inference dollar ceilings = 3500 calls; $750 / $1500
    Matched study uses 3500 LM calls/game and $750; selected-policy eval allows $1500. Stopping rules censor unfinished tails and affect RHAE comparisons.
  • Policy-selection backbone (single Opus 4.8 run per policy) = n=1 per policy
    Orchestrator is chosen from one matched pass, then frozen for GPT-5.6 Sol and Opus 5. Selection noise is unquantified.
  • Development-game subset inside public eval = 5/25 games
    tr87, vc33, r11l, bp35, ft09 informed harness development while remaining in the reported 25-game score.
axioms (5)
  • domain assumption ARC-AGI-3 levels behave as deterministic rendered environments for fixed version and initial configuration; legal action + full latent state yields one successor.
    Section 3 uses transition functions rather than kernels; justified by turn-based design and benchmark replay qualification, but is an environment assumption.
  • domain assumption Official RHAE (human upper-median baselines, per-level cap 115, later-level weights) is the primary measure of skill-acquisition efficiency.
    All headline comparisons adopt ARC Prize scoring; action count ignores inference cost by benchmark design.
  • ad hoc to paper A free-form Python program with init/transition/render/outcome is an adequate executable hypothesis language for these games.
    Section 4 argues suitability for discrete algorithmic grids; alternative neural latent models are acknowledged but not evaluated.
  • ad hoc to paper Only the actor may commit environment actions; builders only edit models and advise.
    Core experimental control in Table 1 and architecture figure; defines what ‘delegation’ means.
  • standard math Standard deterministic automata / Moore-machine semantics and ordinary planning search are valid background math for the formalization.
    Definitions 1–4 and Appendix B route-sufficiency proofs use standard transition-system reasoning.
invented entities (3)
  • Active abstraction no independent evidence
    purpose: Name the joint problem of inducing a testable model from costly interaction and deciding when building/using it is worth the action and inference cost.
    Central conceptual contribution; operationalized via four policies and diagnostics, not an external natural kind with independent measurement outside this framing.
  • Parameterized rendered deterministic Moore machine with transition emissions (game formalization) independent evidence
    purpose: Make hidden state, non-injective grids, transient frames, and outcomes explicit for verification and planning.
    Definitional model of ARC-AGI-3 in Section 3; useful and checkable against traces, but postulated as the right abstraction rather than uniquely identified from data.
  • Tycho four model-maintenance policies (No-WM, Single, Orchestrator, Trigger) independent evidence
    purpose: Turn model ownership and repair timing into an experimental object.
    Harness-level inventions; compared empirically on the public set.

pith-pipeline@v1.2.0-daily-grok45 · 44749 in / 4040 out tokens · 86350 ms · 2026-07-31T12:13:49.517867+00:00 · methodology

0 comments
read the original abstract

ARC-AGI-3 turns abstraction into an interactive problem of skill acquisition. A player must infer an unfamiliar game's rules, hidden state, and goal while maintaining action efficiency because every move counts. We formalize these environments as parameterized rendered deterministic Moore machines and introduce Tycho, a coding-agent system that constructs and uses game-specific models during interaction. Tycho separates actionable observations from intermediate animation, level-completion, and game-over frames. From this structured history, an agent can model, test, plan with, repair, or bypass a free-form executable hypothesis. In one matched public-set run per policy, we compare four orchestration policies on all 25 public games using Claude Opus 4.8 under matched inference budgets. Actor-requested delegation to a model builder obtains the highest observed mean Relative Human Action Efficiency (RHAE), 88.49. With this selected policy, GPT-5.6 Sol and Opus 5 both reach 100.00 RHAE and complete all 183 levels. Their game-balanced first-run human-replay midranks are 98.5 and 100.0. Opus 5 uses 61% fewer scored actions than the aggregate official human baselines. Automatic repair after verification failures produces models that reproduce observed transitions much more accurately, yet reaches only 83.07 RHAE. Transition match indicates whether a simulator reproduces observed dynamics, not whether it has identified the objective or improves the next action. Strong play also requires deciding when to construct, repair, use, or bypass a model. We call this joint problem active abstraction: generating a testable model from costly interaction and deciding when acquiring or using it is worth its cost.

Figures

Figures reproduced from arXiv: 2607.28287 by Andrei Aioanei, Jens Lehmann, Sahar Vahdati.

Figure 1
Figure 1. Figure 1: Tycho’s shared task-time architecture. The actor reasons over the interaction record and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A non-Markov scrolling viewport. Identical grids at two world offsets have different [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Tycho’s two mechanisms for uncertain output. In (a), scrolling exposes cells the model [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Querying accumulated evidence in the GPT-5.6 Sol/Orchestrator run on [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Synthetic illustration of verification and planning. Panel (a) connects the learned [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: RHAE in the two-stage evaluation. Four matched Opus 4.8 runs select the orchestrator [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Action use and score saturation. Left: actions assigned to completed levels and to [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Per-game public-set RHAE. The first four rows are the matched Opus policy study; the [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Paired public-game RHAE deltas. The no-world-model comparisons show large wins [PITH_FULL_IMAGE:figures/full_fig_p020_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Compute–performance tradeoffs; labels give policy and model. [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: World-model diagnostics for the matched Opus runs. Left: builder calls versus orches [PITH_FULL_IMAGE:figures/full_fig_p025_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Planning with an Executable Model in cd82, level 4. From left to right, the initial observation identifies the target and mutable canvas, the canvas is enlarged, and successive modeled states apply the planned actions until the final state reaches the target and returns level_complete. The diagonally offset cards behind the modeled states show the corresponding observed crops, with green shading marking t… view at source ↗
Figure 13
Figure 13. Figure 13: Planning over a reduced state representation in [PITH_FULL_IMAGE:figures/full_fig_p027_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Transition mismatch, repair, and replanning in [PITH_FULL_IMAGE:figures/full_fig_p028_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Outcome-identification failure in sk48, level 0. The initial puzzle is followed by three simulator-reachable configurations suggested by the HUD; all remained nonterminal in the environment despite accurate modeled dynamics. Together, the cases show both the flexibility and the risk of the model interface. One model preserves a nearly pixel-level UI state. Another discards display resolution in favor of a… view at source ↗
Figure 16
Figure 16. Figure 16: Within-game recombination in tr87, level 6. Earlier levels separately introduce two-stage composition and editable rule panels; level 6 combines them. The agent predicts the intermediate string and searches for a minimum-cost panel assignment, completing the level in 24 actions. piston that can be moved between shafts (Figure 17a). Under the inherited assumption that every piston extends downward, exhaust… view at source ↗
Figure 17
Figure 17. Figure 17: Hypothesis testing through useful actions in [PITH_FULL_IMAGE:figures/full_fig_p030_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Two coupled adaptation loops. During a frozen task run, the model and harness turn [PITH_FULL_IMAGE:figures/full_fig_p032_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

120 extracted references · 5 canonical work pages

  1. [1]

    L. A. Agrawal, S. Tan, D. Soylu, N. Ziems, R. Khare, K. Opsahl-Ong, A. Singhvi, H. Shandilya, M. J. Ryan, M. Jiang, C. Potts, K. Sen, A. G. Dimakis, I. Stoica, D. Klein, M. Zaharia, and O. Khattab. GEPA : Reflective prompt evolution can outperform reinforcement learning. In International Conference on Learning Representations, 2026. URL https://openreview...

    2026

  2. [2]

    Ahmed, J

    Z. Ahmed, J. B. Tenenbaum, C. Bates, and S. J. Gershman. Synthesizing world models for bilevel planning. Transactions on Machine Learning Research, 2025. URL https://openreview.net/forum?id=m9V4JHLJrD

    2025

  3. [4]

    D. Angluin. Learning regular sets from queries and counterexamples. Information and Computation, 75 0 (2): 0 87--106, 1987. doi:10.1016/0890-5401(87)90052-6

    doi:10.1016/0890-5401(87)90052-6 1987

  4. [5]

    Models overview

    Anthropic . Models overview. https://platform.claude.com/docs/en/about-claude/models/overview, 2026 a . Accessed 2026-07-29

    2026

  5. [6]

    Claude pricing

    Anthropic . Claude pricing. https://platform.claude.com/docs/en/about-claude/pricing, 2026 b . Accessed 2026-07-22

    2026

  6. [7]

    ARC-AGI-3 scoring methodology

    ARC Prize Foundation . ARC-AGI-3 scoring methodology. https://docs.arcprize.org/methodology, 2026 a . Accessed 2026-07-09

    2026

  7. [9]

    ARC-AGI community leaderboard

    ARC Prize Foundation . ARC-AGI community leaderboard. https://arcprize.org/leaderboard/community, 2026 c . Accessed 2026-07-22

    2026

  8. [11]

    Bessis, J

    H. Bessis, J. Cottaar, I. Pressman, A. Smit, M. Te s nar, and S. Viel. Duck harness: Winning solution for ARC-AGI-3 milestone 1. https://tufalabs.ai/research/duck-harness/, 2026. Published 2026-07-01; accessed 2026-07-22

    2026

  9. [12]

    Bonet and H

    B. Bonet and H. Geffner. Planning with incomplete information as heuristic search in belief space. In Proceedings of the Fifth International Conference on Artificial Intelligence Planning and Scheduling (AIPS), pages 52--61. AAAI Press, 2000

    2000

  10. [13]

    M. Chen, J. Wang, Z. Liu, Y. Wang, H. Zheng, and Q. Wang. From failed trajectories to reliable LLM agents: Diagnosing and repairing harness flaws. arXiv preprint arXiv:2606.06324, 2026

    Pith/arXiv arXiv 2026

  11. [15]

    Courtis, W

    D. Courtis, W. Li, and S. Sanner. OPINE-World : Programmatic world modeling with ontology-error-prioritized interactive exploration for ARC-AGI-3 . arXiv preprint arXiv:2607.01531, 2026

    Pith/arXiv arXiv 2026

  12. [16]

    Dainese, M

    N. Dainese, M. Merler, M. Alakuijala, and P. Marttinen. Generating code world models with large language models guided by monte carlo tree search. In Advances in Neural Information Processing Systems, volume 37, pages 60429--60474, 2024. doi:10.52202/079017-1933

    doi:10.52202/079017-1933 2024

  13. [17]

    Elsworth, K

    C. Elsworth, K. Huang, D. Patterson, I. Schneider, R. Sedivy, S. Goodman, B. Townsend, P. Ranganathan, J. Dean, A. Vahdat, B. Gomes, and J. Manyika. Measuring the environmental impact of delivering AI at Google scale. arXiv preprint arXiv:2508.15734, 2025

    Pith/arXiv arXiv 2025

  14. [18]

    A. Fox, J. Wang, P. Rosu, and B. Dhingra. PRO-LONG : Programmatic memory enables long-horizon reasoning. arXiv preprint arXiv:2607.20064, 2026 a

    Pith/arXiv arXiv 2026

  15. [19]

    A. Fox, J. Wang, P. Rosu, and B. Dhingra. PRO-LONG source code, scorecards, and run logs. https://github.com/alexisfox7/PRO-LONG, 2026 b . Repository revision 7268de0 ; accessed 2026-07-28

    2026

  16. [21]

    Giantamidis, S

    G. Giantamidis, S. Tripakis, and S. Basagiannis. Learning Moore machines from input-output traces. International Journal on Software Tools for Technology Transfer, 23 0 (1): 0 1--29, 2021. doi:10.1007/s10009-019-00544-0

    work page doi:10.1007/s10009-019-00544-0 2021

  17. [22]

    Grimm, A

    C. Grimm, A. Barreto, S. P. Singh, and D. Silver. The value equivalence principle for model-based reinforcement learning. In Advances in Neural Information Processing Systems, volume 33, pages 5541--5552, 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/3bb585ea00014b0e3ebe4c6dd165a358-Abstract.html

    2020

  18. [23]

    Ha and J

    D. Ha and J. Schmidhuber. Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems, volume 31, pages 2450--2462, 2018

    2018

  19. [24]

    Hafner, T

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi. Dream to control: Learning behaviors by latent imagination. In International Conference on Learning Representations (ICLR), 2020. URL https://openreview.net/forum?id=S1lOTC4tDS

    2020

  20. [25]

    L. K. Han. AERA ARC-AGI-3 evaluation code. GitHub, https://github.com/farmountain/aera-arc3-paper, 2026 a . Repository revision https://github.com/farmountain/aera-arc3-paper/tree/53ba4db1d715cd75c28640ca1a2ddd444cd02e89 53ba4db . Accessed 2026-07-30

    2026

  21. [26]

    L. K. Han. Explore before you solve: The speed--depth trade-off in epistemic agents for ARC-AGI-3 . arXiv preprint arXiv:2605.25931, 2026 b

    Pith/arXiv arXiv 2026

  22. [27]

    R. A. Howard. Information value theory. IEEE Transactions on Systems Science and Cybernetics, 2 0 (1): 0 22--26, 1966. doi:10.1109/TSSC.1966.300074

    arXiv 1966

  23. [28]

    S. Hu, C. Lu, and J. Clune. Automated design of agentic systems. In International Conference on Learning Representations, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/hash/36b7acf6f6010652b3f2a433774a66fe-Abstract-Conference.html

    2025

  24. [29]

    L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101 0 (1--2): 0 99--134, 1998. doi:10.1016/S0004-3702(98)00023-X

    doi:10.1016/s0004-3702(98)00023-x 1998

  25. [30]

    G. Kamradt. Analyzing GPT-5.5 & Opus 4.7 with ARC-AGI-3 . ARC Prize Foundation, https://arcprize.org/blog/arc-agi-3-gpt-5-5-opus-4-7-analysis, 2026 a . Published 2026-05-01; accessed 2026-07-22

    2026

  26. [31]

    G. Kamradt. Measuring human performance on ARC-AGI-3 . https://arcprize.org/blog/arc-agi-3-human-dataset, 2026 b . Published 2026-04-14; accessed 2026-07-22

    2026

  27. [32]

    Kapoor, B

    S. Kapoor, B. Stroebl, Z. S. Siegel, N. Nadgir, and A. Narayanan. AI agents that matter. Transactions on Machine Learning Research, 2025. URL https://openreview.net/forum?id=Zy4uFzMviZ

    2025

  28. [34]

    E. D. Klenske and P. Hennig. Dual control for approximate bayesian reinforcement learning. Journal of Machine Learning Research, 17 0 (127): 0 1--30, 2016

    2016

  29. [35]

    Lambert, B

    N. Lambert, B. Amos, O. Yadan, and R. Calandra. Objective mismatch in model-based reinforcement learning. In Proceedings of the 2nd Conference on Learning for Dynamics and Control, volume 120 of Proceedings of Machine Learning Research, pages 761--770, 2020

    2020

  30. [36]

    W.-D. Li, K. Hu, C. Larsen, Y. Wu, S. Alford, C. Woo, S. M. Dunn, H. Tang, W.-L. Zheng, Y. Pu, and K. Ellis. Combining induction and transduction for abstract reasoning. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=UmdotAAVDe

    2025

  31. [37]

    Locatello, D

    F. Locatello, D. Weissenborn, T. Unterthiner, A. Mahendran, G. Heigold, J. Uszkoreit, A. Dosovitskiy, and T. Kipf. Object-centric learning with slot attention. In Advances in Neural Information Processing Systems, volume 33, pages 11594--11604, 2020

    2020

  32. [38]

    D. J. C. MacKay. Information-based objective functions for active data selection. Neural Computation, 4 0 (4): 0 590--604, 1992. doi:10.1162/neco.1992.4.4.590

    doi:10.1162/neco.1992.4.4.590 1992

  33. [39]

    Menaged, G

    R. Menaged, G. Lior, S. Ravfogel, R. Aharoni, and G. Stanovsky. Can LLM agents infer world models? evidence from agentic automata learning. arXiv preprint arXiv:2606.16576, 2026

    arXiv 2026

  34. [40]

    GPT-5.6 Sol Model

    OpenAI . GPT-5.6 Sol Model . OpenAI API documentation, 2026. URL https://developers.openai.com/api/docs/models/gpt-5.6-sol. Accessed 2026-07-22

    2026

  35. [41]

    Oviedo, F

    F. Oviedo, F. Kazhamiaka, E. Choukse, A. Kim, A. Luers, M. Nakagawa, R. Bianchini, and J. M. Lavista Ferres. Energy use of AI inference, efficiency pathways, and test-time scaling. Joule, page 102430, 2026. doi:10.1016/j.joule.2026.102430

    arXiv 2026

  36. [43]

    W. T. Piriyakulkij, Y. Liang, H. Tang, A. Weller, M. Kryven, and K. Ellis. PoE-World : Compositional world modeling with products of programmatic experts. In Advances in Neural Information Processing Systems, volume 38, 2025. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/262dd62fd1bbb30d6a6b4d578f5e65ff-Abstract-Conference.html

    2025

  37. [45]

    R. L. Rivest and R. E. Schapire. Inference of finite automata using homing sequences. Information and Computation, 103 0 (2): 0 299--347, 1993. doi:10.1006/inco.1993.1021

    arXiv 1993

  38. [47]

    Rodionov

    S. Rodionov. Do coding agents need executable world models, simplification, and verification to solve ARC-AGI-3 ? arXiv preprint arXiv:2607.15439, 2026 b

    Pith/arXiv arXiv 2026

  39. [48]

    Rodionov

    S. Rodionov. Executable world models for ARC-AGI-3 in the era of coding agents. arXiv preprint arXiv:2605.05138, 2026 c . Accepted at AGI-2026

    Pith/arXiv arXiv 2026

  40. [49]

    Rudakov, J

    E. Rudakov, J. Shock, and B. U. Cowley. Graph-based exploration for ARC-AGI-3 interactive reasoning tasks. arXiv preprint arXiv:2512.24156, 2025

    arXiv 2025

  41. [50]

    Russell and E

    S. Russell and E. Wefald. Principles of metareasoning. Artificial Intelligence, 49 0 (1--3): 0 361--395, 1991. doi:10.1016/0004-3702(91)90015-C

    work page doi:10.1016/0004-3702(91)90015-c 1991

  42. [52]

    Sch \"o lkopf, F

    B. Sch \"o lkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio. Toward causal representation learning. Proceedings of the IEEE, 109 0 (5): 0 612--634, 2021. doi:10.1109/JPROC.2021.3058954

    arXiv 2021

  43. [53]

    Schrittwieser, I

    J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, T. Lillicrap, and D. Silver. Mastering Atari , Go , chess and shogi by planning with a learned model. Nature, 588: 0 604--609, 2020. doi:10.1038/s41586-020-03051-4

    doi:10.1038/s41586-020-03051-4 2020

  44. [54]

    Shinn, F

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 8634--8652, 2023. doi:10.52202/075280-0377

    doi:10.52202/075280-0377 2023

  45. [55]

    H. Tang, D. Key, and K. Ellis. WorldCoder , a model-based LLM agent: Building world models by writing code and interacting with the environment. In Advances in Neural Information Processing Systems, volume 37, pages 70148--70212, 2024. doi:10.52202/079017-2243

    doi:10.52202/079017-2243 2024

  46. [56]

    Tsividis, J

    P. Tsividis, J. Loula, J. Burga, J. P. Rodriguez, S. Arnaud, N. Foss, A. Campero, A. Subramanian, T. Pouncy, S. J. Gershman, and J. B. Tenenbaum. Human-level learning of complex novel tasks as theory-based modelling, exploration and planning. Philosophical Transactions of the Royal Society A, 384 0 (2320): 0 20240529, 2026. doi:10.1098/rsta.2024.0529

    arXiv 2026

  47. [57]

    Vaandrager, B

    F. Vaandrager, B. Garhewal, J. Rot, and T. Wi mann. A new approach for active automata learning based on apartness. In Tools and Algorithms for the Construction and Analysis of Systems, volume 13243 of Lecture Notes in Computer Science, pages 223--243. Springer, 2022. doi:10.1007/978-3-030-99524-9_12

    doi:10.1007/978-3-030-99524-9_12 2022

  48. [58]

    Vahdati, A

    S. Vahdati, A. Aioanei, H. Suresh, and J. Lehmann. The ARC of progress towards AGI : A living survey of abstraction and reasoning. arXiv preprint arXiv:2603.13372, 2026

    arXiv 2026

  49. [59]

    G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research, 2024. URL https://openreview.net/forum?id=ehfRiF0R3a

    2024

  50. [60]

    Warrier, T

    A. Warrier, T. D. Nguyen, M. Naim, M. Jain, Y. Liang, K. Schroeder, C. Yang, J. B. Tenenbaum, S. Vollmer, K. Ellis, and Z. Tavares. Benchmarking world-model learning with environment-level queries. In Proceedings of the Forty-third International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, 2026. URL https://openr...

    2026

  51. [61]

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press. SWE-agent : Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems, volume 37, pages 50528--50652, 2024. doi:10.52202/079017-1601

    doi:10.52202/079017-1601 2024

  52. [62]

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023. URL https://openreview.net/forum?id=WE_vluYUL-X

    2023

  53. [64]

    Y. Yu, P. Herman, D. L. Rothman, D. Agarwal, and F. Hyder. Evaluating the gray and white matter energy budgets of human brain function. Journal of Cerebral Blood Flow & Metabolism, 38 0 (8): 0 1339--1353, 2018. doi:10.1177/0271678X17708691

    work page doi:10.1177/0271678x17708691 2018

  54. [65]

    G. Zeng, J. Wang, W. Ma, S. Yin, C. Wang, S. Liu, A. Kanazawa, W. Ni, X. Li, A. Zanette, and H. Feng. Frontier models with our harness achieve 99\ public--- Schema . Impossible Research, https://schema-harness.github.io/, 2026. Project report; retained traces at https://huggingface.co/datasets/schema-harness/arc-agi-3-schema-traces; accessed 2026-07-22

    2026

  55. [66]

    Zhang, J

    Y. Zhang, J. Wang, Y. Ge, W. Xu, J. Hamm, and C. K. Reddy. Stop comparing LLM agents without disclosing the harness. Preprints, 2026. doi:10.20944/preprints202605.0711.v1. URL https://www.preprints.org/manuscript/202605.0711

    arXiv 2026

  56. [67]

    arXiv preprint arXiv:1911.01547 , year=

    On the Measure of Intelligence , author=. arXiv preprint arXiv:1911.01547 , year=

    Pith/arXiv arXiv 1911

  57. [68]

    The Thirteenth International Conference on Learning Representations , year=

    Combining Induction and Transduction for Abstract Reasoning , author=. The Thirteenth International Conference on Learning Representations , year=

  58. [69]

    Artificial Intelligence , volume=

    Planning and acting in partially observable stochastic domains , author=. Artificial Intelligence , volume=. 1998 , publisher=

    1998

  59. [70]

    Information and Computation , volume=

    Learning regular sets from queries and counterexamples , author=. Information and Computation , volume=. 1987 , publisher=

    1987

  60. [71]

    Information and Computation , volume=

    Inference of finite automata using homing sequences , author=. Information and Computation , volume=. 1993 , publisher=

    1993

  61. [72]

    arXiv preprint arXiv:2603.24621 , year=

    Pith/arXiv arXiv

  62. [73]

    Vahdati, Sahar and Aioanei, Andrei and Suresh, Haridhra and Lehmann, Jens , journal=. The

  63. [74]

    2026 , howpublished=

    2026

  64. [75]

    Measuring Human Performance on

    Kamradt, Greg , year=. Measuring Human Performance on

  65. [76]

    Journal of Cerebral Blood Flow & Metabolism , volume=

    Evaluating the Gray and White Matter Energy Budgets of Human Brain Function , author=. Journal of Cerebral Blood Flow & Metabolism , volume=. 2018 , doi=

    2018

  66. [77]

    Measuring the Environmental Impact of Delivering

    Elsworth, Cooper and Huang, Keguo and Patterson, David and Schneider, Ian and Sedivy, Robert and Goodman, Savannah and Townsend, Ben and Ranganathan, Parthasarathy and Dean, Jeff and Vahdat, Amin and Gomes, Ben and Manyika, James , journal=. Measuring the Environmental Impact of Delivering

  67. [78]

    , journal=

    Oviedo, Felipe and Kazhamiaka, Fiodar and Choukse, Esha and Kim, Allen and Luers, Amy and Nakagawa, Melanie and Bianchini, Ricardo and Lavista Ferres, Juan M. , journal=. Energy Use of. 2026 , doi=

    2026

  68. [79]

    2026 , howpublished=

    Claude Pricing , author=. 2026 , howpublished=

    2026

  69. [80]

    2026 , howpublished=

    Models Overview , author=. 2026 , howpublished=

    2026

  70. [81]

    Executable World Models for

    Rodionov, Sergey , journal=. Executable World Models for

  71. [82]

    Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve

    Rodionov, Sergey , journal=. Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve

  72. [83]

    Supporting Data and Full Run Artifacts for ``Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve

    Rodionov, Sergey , year=. Supporting Data and Full Run Artifacts for ``Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve. doi:10.5281/zenodo.21412274 , howpublished=

    work page doi:10.5281/zenodo.21412274

  73. [84]

    arXiv preprint arXiv:2507.12821 , year=

    Assessing Adaptive World Models in Machines with Novel Games , author=. arXiv preprint arXiv:2507.12821 , year=

    Pith/arXiv arXiv

  74. [85]

    arXiv preprint arXiv:2605.09650 , year=

    Workspace Optimization: How to Train Your Agent , author=. arXiv preprint arXiv:2605.09650 , year=

    Pith/arXiv arXiv

  75. [86]

    arXiv preprint arXiv:2605.09998 , year=

    Continual Harness: Online Adaptation for Self-Improving Foundation Agents , author=. arXiv preprint arXiv:2605.09998 , year=

    Pith/arXiv arXiv

  76. [87]

    arXiv preprint arXiv:2607.20709 , year=

    Furgale, Paul and Klingler, Severin and Nolan, James and Staats, Matt and Di Lorenzo, Gaia and Martinez Abad, Elisa and Sch. arXiv preprint arXiv:2607.20709 , year=

    Pith/arXiv arXiv

  77. [88]

    Duck Harness: Winning Solution for

    Bessis, Harold and Cottaar, Jeroen and Pressman, Isaiah and Smit, Andries and Te. Duck Harness: Winning Solution for. 2026 , howpublished=

    2026

  78. [89]

    Frontier Models with Our Harness Achieve

    Zeng, Guanning and Wang, Jiani and Ma, Wenjie and Yin, Shaofeng and Wang, Chenyang and Liu, Shichen and Kanazawa, Angjoo and Ni, Wode and Li, Xiuyu and Zanette, Andrea and Feng, Haiwen , year=. Frontier Models with Our Harness Achieve

  79. [90]

    Fox, Alexis and Wang, Junlin and Rosu, Paul and Dhingra, Bhuwan , journal=

  80. [91]

    Fox, Alexis and Wang, Junlin and Rosu, Paul and Dhingra, Bhuwan , year=

Showing first 80 references.