From Next Token Prediction to (STRIPS) World Models

Carlos N\'u\~nez-Molina; Hector Geffner; Vicen\c{c} G\'omez

arxiv: 2509.13389 · v6 · submitted 2025-09-16 · 💻 cs.AI

From Next Token Prediction to (STRIPS) World Models

Carlos N\'u\~nez-Molina , Vicen\c{c} G\'omez , Hector Geffner This is my paper

Pith reviewed 2026-05-18 16:17 UTC · model grok-4.3

classification 💻 cs.AI

keywords next token predictionSTRIPS planningworld modelstransformersgeneralizationaction modelssymbolic planningplanning domains

0 comments

The pith

Next-token prediction on action traces yields STRIPS world models accurate enough for planning on unseen states and goals.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether training transformers via next-token prediction on action traces alone can produce world models that support genuine planning in symbolic STRIPS domains. Two architectures are compared: a STRIPS Transformer built with explicit symbolic alignment to the formal structure of STRIPS, and standard transformers using different positional encodings and attention mechanisms. Both approaches generate models that off-the-shelf STRIPS planners can use to solve problems involving exponentially many initial states and goals never encountered in training. The standard transformer with stick-breaking attention reaches near-perfect accuracy and generalizes reliably, while the STRIPS Transformer demands larger datasets despite its stronger inductive bias.

Core claim

In a controlled symbolic setting where propositional STRIPS action models are learned solely from action traces and correctness can be checked exactly, next-token prediction produces models that serve as drop-in replacements for hand-written STRIPS operators. These models enable classical planners to solve planning tasks over exponentially many unseen initial states and goals. The STRIPS Transformer incorporates theoretical links to the formal language of STRIPS domains, whereas a standard transformer augmented with stick-breaking attention achieves comparable or better results in accuracy and generalization without explicit symbolic structure.

What carries the argument

Next-token prediction on sequences of STRIPS actions and states, used to learn implicit action models that are then extracted and fed to off-the-shelf symbolic planners.

If this is right

Both architectures support planning across five classical domains and varying problem sizes without retraining.
Generalization holds for exponentially many new initial states and goals drawn from the same domain.
Stick-breaking attention enables reliable performance on long traces where standard attention fails.
A symbolic STRIPS model can be extracted from a transformer trained only on short traces to restore planning capability.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

This suggests data-driven sequence models can replace manual STRIPS engineering in domains where action traces are cheap to collect.
The approach may extend to domains with partial observability if trace data includes observations.
It opens the possibility of using large language models directly as implicit planners without separate symbolic extraction.
Testable extension: apply the same training regime to domains with numeric fluents or durative actions.

Load-bearing premise

The learned next-token predictors are accurate and complete enough to serve as correct STRIPS action models for arbitrary unseen states and goals.

What would settle it

A held-out planning problem whose true solution requires an action sequence that the learned model either forbids or fails to reach within the planner's search budget.

Figures

Figures reproduced from arXiv: 2509.13389 by Carlos N\'u\~nez-Molina, Hector Geffner, Vicen\c{c} G\'omez.

**Figure 1.** Figure 1: Vectors produced by the B-RASP program f B-RASP M for the two traces τ + and τ − shown on top, drawn from the model M shown on the left, named simple. The value of f B-RASP M (τ ) is given by the last entry of the last vector, i.e., Z(n = 6) (marked in bold), and for both traces, f B-RASP M (τ ) = fM(τ ). (a) The hidden STRIPS domain simple. (b) Vectors produced for computing f B-RASP M (τ +). (c) Vectors … view at source ↗

**Figure 2.** Figure 2: Examples of domains learned in simple for the training dataset with 200 samples. For comparison purposes, we renamed the propositions to p, q, r as in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Figure 3: Self-attention computations for (a) the valid trace [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

read the original abstract

We study whether next-token prediction can yield world models that truly support planning, in a controlled symbolic setting where propositional STRIPS action models are learned from action traces alone and correctness can be evaluated exactly. We introduce two architectures. The first is the STRIPS Transformer, a symbolically aligned model grounded in theoretical results linking transformers and the formal language structure of STRIPS domains. The second is a standard transformer architecture without explicit symbolic structure built in, for which we study different positional encoding schemes and attention aggregation mechanisms. We evaluate both architectures on five classical planning domains, measuring training accuracy, generalization, and planning performance across domains and problem sizes. Interestingly, both approaches can be used to produce models that support planning with off-the-shelf STRIPS planners over exponentially many unseen initial states and goals. Although the STRIPS Transformer incorporates a strong symbolic inductive bias, it is harder to optimize and requires larger datasets to generalize reliably. In contrast, a standard transformer with stick-breaking attention achieves near-perfect training accuracy and strong generalization. Finally, standard transformers without stick-breaking attention do not generalize to long traces, whereas a symbolic STRIPS model extracted from a transformer trained on shorter traces does.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Next-token prediction on action traces can produce STRIPS models that support planning on unseen states and goals, with a standard transformer plus stick-breaking attention working better than the custom symbolic version.

read the letter

The paper's core result is that you can train a transformer on planning traces alone and extract a model that functions as a usable STRIPS domain for off-the-shelf planners. Both their STRIPS Transformer and a plain transformer with stick-breaking attention achieve this on five classical domains, with planning success reported over exponentially many held-out initial states and goals. The evaluation stays in a fully symbolic setting, so correctness can be checked exactly rather than through simulation or approximation. That is the practical bridge they are after, and the experiments back it up with direct planning runs rather than just prediction accuracy. The comparisons are the clearest contribution. They test a symbolically aligned architecture grounded in formal language results against standard transformers with different positional encodings and attention mechanisms. The symbolic version incorporates a strong inductive bias but turns out harder to optimize and needs larger datasets to generalize reliably. In contrast, the standard transformer with stick-breaking attention reaches near-perfect training accuracy and strong generalization, while versions without that attention fail on longer traces. Even a model trained on short traces can be turned into a symbolic model that still supports planning. The setup is clean and the results are reported with enough detail to see the trade-offs. The main soft spot is that the symbolic model underperforms despite the built-in structure, which raises questions about how much the inductive bias actually helps in practice. The stress-test concern about completeness on states reached during search is reasonable to raise, but the direct planning evaluations on unseen problems should catch many soundness issues that would otherwise break the planner. Still, more explicit checks on planner-generated states would strengthen the claim. This is useful for anyone working on learning world models or neuro-symbolic planning. The experiments are controlled, the metrics are appropriate, and the thinking is straightforward, so it deserves peer review.

Referee Report

1 major / 2 minor

Summary. The paper studies whether next-token prediction can produce world models supporting planning in a controlled symbolic setting. It introduces the STRIPS Transformer, grounded in theoretical links between transformers and STRIPS formal language, and compares it to standard transformers using different positional encodings and attention mechanisms (including stick-breaking). Both are trained on action traces from five classical planning domains and evaluated on training accuracy, generalization to unseen initial states and goals, and planning performance with off-the-shelf STRIPS planners.

Significance. If the results hold, the work shows that next-token predictors can yield extractable STRIPS operators that enable exact planning over exponentially many unseen states and goals, bridging neural sequence modeling and symbolic planning. The finding that a standard transformer with stick-breaking attention achieves strong generalization while the symbolically biased STRIPS Transformer is harder to optimize is a useful empirical contribution, as is the observation that standard transformers without stick-breaking fail to generalize to long traces.

major comments (1)

[Evaluation / planning performance] Evaluation section (planning performance results): The central claim that the learned models support planning over unseen states and goals requires that the extracted STRIPS operators remain sound and complete for states generated during search. While planning success is reported, the manuscript does not describe an explicit check (e.g., enumeration or verification of predicted effects against ground-truth semantics) for states encountered by the planner itself, which may lie outside the training trace distribution. This leaves open the possibility of silent failures that could invalidate plans or dead-end detection.

minor comments (2)

[Abstract] The abstract states that both architectures 'can be used to produce models that support planning,' but the text should clarify whether this holds uniformly across all five domains or if there are domain-specific exceptions in the reported results.
[Method] Notation for the extracted STRIPS operators (preconditions and effects) should be introduced earlier and used consistently when describing how next-token outputs are converted to formal action models.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their careful reading and constructive feedback. We address the single major comment below and will revise the manuscript to incorporate an explicit verification procedure as suggested.

read point-by-point responses

Referee: Evaluation section (planning performance results): The central claim that the learned models support planning over unseen states and goals requires that the extracted STRIPS operators remain sound and complete for states generated during search. While planning success is reported, the manuscript does not describe an explicit check (e.g., enumeration or verification of predicted effects against ground-truth semantics) for states encountered by the planner itself, which may lie outside the training trace distribution. This leaves open the possibility of silent failures that could invalidate plans or dead-end detection.

Authors: We agree that an explicit verification step strengthens the central claim. In the current experiments, planning success was defined as the off-the-shelf STRIPS planner returning a plan that, when executed in the ground-truth simulator, reaches the goal without violating any preconditions or producing incorrect effects. Because the planner only applies operators that are deemed applicable and because success is measured by actual execution in the true environment, any unsound operator would have produced either an invalid plan or a failure to reach the goal. Nevertheless, this verification was performed only at the level of final plan execution rather than for every intermediate state visited during search. To address the referee's concern directly, we will add a new paragraph in Section 5 (Evaluation) that describes an additional post-hoc check: for every solved planning instance we record all states generated by the planner and verify that the predicted add and delete effects of each applied operator exactly match the ground-truth transition function. We will report the fraction of states for which this holds and will include the verification code in the supplementary material. This change will be made in the revised version. revision: yes

Circularity Check

0 steps flagged

No significant circularity: derivation relies on external empirical validation against held-out states and planners

full rationale

The paper trains next-token models on action traces, extracts or uses them as STRIPS operators, and evaluates training accuracy, generalization to unseen initial states/goals, and planning success with off-the-shelf external STRIPS planners. No equation or claim reduces a reported prediction to a fitted quantity by construction, nor does any load-bearing step invoke a self-citation chain or uniqueness theorem that collapses to the paper's own inputs. Results are measured against independent benchmarks outside the training distribution, making the central claims empirically falsifiable rather than tautological.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the assumption that next-token accuracy on traces translates to correct action models usable by planners, plus standard transformer training assumptions. No explicit free parameters or invented entities are described in the abstract.

axioms (1)

domain assumption Next-token prediction on action traces is sufficient to recover the full propositional STRIPS transition function.
Invoked by the decision to train only on traces and then extract or use the model for planning.

pith-pipeline@v0.9.0 · 5741 in / 1259 out tokens · 32335 ms · 2026-05-18T16:17:00.156670+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Differentiable Learning of Lifted Action Schemas for Classical Planning
cs.AI 2026-05 unverdicted novelty 7.0

A differentiable neural model recovers ground-truth lifted action schemas from state traces by jointly learning schemas and inferring unobserved action arguments.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages · cited by 1 Pith paper

[1]

J.; and Onaindia, E

Aineto, D.; Celorrio, S. J.; and Onaindia, E. 2019. Learning action models with minimal observability. Artificial Intelligence, 275: 104--137

work page 2019
[2]

Asai, M.; Kajino, H.; Fukunaga, A.; and Muise, C. 2022. Classical planning in deep latent space. Journal of Artificial Intelligence Research, 74: 1599--1686

work page 2022
[3]

Bonet, B.; and Geffner, H. 2020. Learning First-Order Symbolic Representations for Planning from the Structure of the State Space. In ECAI 2020 - 24th European Conference on Artificial Intelligence , volume 325, 2322--2329. IOS Press

work page 2020
[4]

N.; McCluskey, T

Cresswell, S. N.; McCluskey, T. L.; and West, M. M. 2013. Acquiring planning domain models using LOCM. The Knowledge Engineering Review, 28(2): 195--213

work page 2013
[5]

Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations

work page 2021
[6]

G \"o sgens, J.; Jansen, N.; and Geffner, H. 2024. Learning Lifted STRIPS Models from Action Traces Alone: A Simple, General, and Scalable Solution. arXiv preprint arXiv:2411.14995

work page arXiv 2024
[7]

E.; and Traverso, P

Lamanna, L.; Serafini, L.; Saetti, A.; Gerevini, A. E.; and Traverso, P. 2025. Lifted action models learning from partial traces. Artificial Intelligence, 339: 104256

work page 2025
[8]

K.; Bau, D.; Vi \'e gas, F.; Pfister, H.; and Wattenberg, M

Li, K.; Hopkins, A. K.; Bau, D.; Vi \'e gas, F.; Pfister, H.; and Wattenberg, M. 2023. Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task. In The Eleventh International Conference on Learning Representations

work page 2023
[9]

Lin, S.; and Bercher, P. 2022. On the Expressive Power of Planning Formalisms in Conjunction with LTL. Proceedings of the International Conference on Automated Planning and Scheduling, 32(1): 231--240

work page 2022
[10]

Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; and Doll \'a r, P. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2980--2988

work page 2017
[11]

Liu, L.; Jiang, H.; He, P.; Chen, W.; Liu, X.; Gao, J.; and Han, J. 2019. On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265

work page arXiv 2019
[12]

D.; Bonet, B.; Romero, J.; and Geffner, H

Rodriguez, I. D.; Bonet, B.; Romero, J.; and Geffner, H. 2021. Learning First-Order Representations for Planning from Black Box States: New Results . In Proceedings of the 18th International Conference on Principles of Knowledge Representation and Reasoning , 539--548

work page 2021
[13]

Tan, S.; Yang, S.; Courville, A.; Panda, R.; and Shen, Y. 2025. Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study. In The Thirteenth International Conference on Learning Representations

work page 2025
[14]

Y.; Rambachan, A.; Kleinberg, J.; and Mullainathan, S

Vafa, K.; Chen, J. Y.; Rambachan, A.; Kleinberg, J.; and Mullainathan, S. 2024 a . Evaluating the world model implicit in a generative model. Advances in Neural Information Processing Systems, 37: 26941--26975

work page 2024
[15]

Y.; Rambachan, A.; Kleinberg, J.; and Mullainathan, S

Vafa, K.; Chen, J. Y.; Rambachan, A.; Kleinberg, J.; and Mullainathan, S. 2024 b . Evaluating the world model implicit in a generative model. Advances in Neural Information Processing Systems, 37: 26941--26975

work page 2024
[16]

N.; Kaiser, .; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

work page 2017
[17]

Weiss, G.; Goldberg, Y.; and Yahav, E. 2021. Thinking like transformers. In International Conference on Machine Learning, 11080--11090. PMLR

work page 2021
[18]

Xi, K.; Gould, S.; and Thi \'e baux, S. 2024. Neuro-Symbolic Learning of Lifted Action Models from Visual Traces. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 34, 653--662

work page 2024
[19]

Yang, A.; Chiang, D.; and Angluin, D. 2024. Masked Hard-Attention Transformers Recognize Exactly the Star-Free Languages. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

work page 2024
[20]

Yildirim, I.; and Paul, L. 2024. From task structures to world models: what do LLMs know? Trends in Cognitive Sciences, 28(5): 404--415

work page 2024
[21]

H.; and Kambhampati, S

Zhuo, H. H.; and Kambhampati, S. 2013. Action-model acquisition from noisy plan traces. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, 2444–2450. AAAI Press

work page 2013
[22]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

work page
[23]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

work page

[1] [1]

J.; and Onaindia, E

Aineto, D.; Celorrio, S. J.; and Onaindia, E. 2019. Learning action models with minimal observability. Artificial Intelligence, 275: 104--137

work page 2019

[2] [2]

Asai, M.; Kajino, H.; Fukunaga, A.; and Muise, C. 2022. Classical planning in deep latent space. Journal of Artificial Intelligence Research, 74: 1599--1686

work page 2022

[3] [3]

Bonet, B.; and Geffner, H. 2020. Learning First-Order Symbolic Representations for Planning from the Structure of the State Space. In ECAI 2020 - 24th European Conference on Artificial Intelligence , volume 325, 2322--2329. IOS Press

work page 2020

[4] [4]

N.; McCluskey, T

Cresswell, S. N.; McCluskey, T. L.; and West, M. M. 2013. Acquiring planning domain models using LOCM. The Knowledge Engineering Review, 28(2): 195--213

work page 2013

[5] [5]

Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations

work page 2021

[6] [6]

G \"o sgens, J.; Jansen, N.; and Geffner, H. 2024. Learning Lifted STRIPS Models from Action Traces Alone: A Simple, General, and Scalable Solution. arXiv preprint arXiv:2411.14995

work page arXiv 2024

[7] [7]

E.; and Traverso, P

Lamanna, L.; Serafini, L.; Saetti, A.; Gerevini, A. E.; and Traverso, P. 2025. Lifted action models learning from partial traces. Artificial Intelligence, 339: 104256

work page 2025

[8] [8]

K.; Bau, D.; Vi \'e gas, F.; Pfister, H.; and Wattenberg, M

Li, K.; Hopkins, A. K.; Bau, D.; Vi \'e gas, F.; Pfister, H.; and Wattenberg, M. 2023. Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task. In The Eleventh International Conference on Learning Representations

work page 2023

[9] [9]

Lin, S.; and Bercher, P. 2022. On the Expressive Power of Planning Formalisms in Conjunction with LTL. Proceedings of the International Conference on Automated Planning and Scheduling, 32(1): 231--240

work page 2022

[10] [10]

Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; and Doll \'a r, P. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2980--2988

work page 2017

[11] [11]

Liu, L.; Jiang, H.; He, P.; Chen, W.; Liu, X.; Gao, J.; and Han, J. 2019. On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265

work page arXiv 2019

[12] [12]

D.; Bonet, B.; Romero, J.; and Geffner, H

Rodriguez, I. D.; Bonet, B.; Romero, J.; and Geffner, H. 2021. Learning First-Order Representations for Planning from Black Box States: New Results . In Proceedings of the 18th International Conference on Principles of Knowledge Representation and Reasoning , 539--548

work page 2021

[13] [13]

Tan, S.; Yang, S.; Courville, A.; Panda, R.; and Shen, Y. 2025. Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study. In The Thirteenth International Conference on Learning Representations

work page 2025

[14] [14]

Y.; Rambachan, A.; Kleinberg, J.; and Mullainathan, S

Vafa, K.; Chen, J. Y.; Rambachan, A.; Kleinberg, J.; and Mullainathan, S. 2024 a . Evaluating the world model implicit in a generative model. Advances in Neural Information Processing Systems, 37: 26941--26975

work page 2024

[15] [15]

Y.; Rambachan, A.; Kleinberg, J.; and Mullainathan, S

Vafa, K.; Chen, J. Y.; Rambachan, A.; Kleinberg, J.; and Mullainathan, S. 2024 b . Evaluating the world model implicit in a generative model. Advances in Neural Information Processing Systems, 37: 26941--26975

work page 2024

[16] [16]

N.; Kaiser, .; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

work page 2017

[17] [17]

Weiss, G.; Goldberg, Y.; and Yahav, E. 2021. Thinking like transformers. In International Conference on Machine Learning, 11080--11090. PMLR

work page 2021

[18] [18]

Xi, K.; Gould, S.; and Thi \'e baux, S. 2024. Neuro-Symbolic Learning of Lifted Action Models from Visual Traces. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 34, 653--662

work page 2024

[19] [19]

Yang, A.; Chiang, D.; and Angluin, D. 2024. Masked Hard-Attention Transformers Recognize Exactly the Star-Free Languages. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

work page 2024

[20] [20]

Yildirim, I.; and Paul, L. 2024. From task structures to world models: what do LLMs know? Trends in Cognitive Sciences, 28(5): 404--415

work page 2024

[21] [21]

H.; and Kambhampati, S

Zhuo, H. H.; and Kambhampati, S. 2013. Action-model acquisition from noisy plan traces. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, 2444–2450. AAAI Press

work page 2013

[22] [22]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

work page

[23] [23]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

work page