Pith. sign in

REVIEW 3 major objections 5 minor 121 references

Make Planning Research Rigorous Again!

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM-based planning research should use the formal languages, validators, and competition protocols the planning community built over six decades.

desk verdict A sound, useful position paper on bringing classical planning rigor to LLM planning evaluation, with a few evidence gaps that should be fixed. read the letter →

arxiv 2505.21674 v1 pith:LDQRI7M2 submitted 2025-05-27 cs.AI

Michael Katz , Harsha Kokel , Christian Muise , Shirin Sohrabi , Sarath Sreedharan This is my paper
classification cs.AI
keywords LLMplanningPDDLplanvalidationbenchmarkdesignevaluationmethodologyInternationalCompetitiondatacontaminationcomputationalcomplexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Automated planning grew into a mature field by insisting on formal problem definitions, standard representation languages, independent validation of proposed plans, competition-style benchmarks on unseen instances, and attention to computational complexity. The paper's position is that research on planning with large language models should now be held to that same standard. Concretely, it recommends that LLM-based planners be designed and evaluated with PDDL, the field's standard factored planning language, with validator-backed correctness checks, freshly generated benchmark instances rather than scraped or memorized corpora, and baselines that solve the same computational problem. The reason to care is that much current LLM planning work repeats pitfalls the planning community already identified and solved, so adopting these practices would let the field measure real progress instead of reproducing old mistakes.

What carries the argument

The machinery carrying the argument is PDDL, a standardized factored representation language that separates domain knowledge, such as types, predicates, actions, preconditions, and effects, from problem instances, such as objects, the initial state, and goals, together with the validator, generator, and competition infrastructure built around it. That infrastructure does the work: independent validators decide in polynomial time whether an action sequence actually reaches the goal; PDDL generators produce new problem instances of controlled difficulty so tests are not contaminated by memorization; and the International Planning Competition supplies the protocol of running planners on previously unseen domains under fixed time and memory limits. Around this core, the paper also leans on computational complexity results, such as PSPACE-hardness of plan existence and domain-specific results like BlocksWorld being NP-complete for cost-optimal planning but polynomial for satisficing planning, to argue that comparisons across different computational problems are meaningless.

What would settle it

A concrete disconfirmation would be an LLM planner that passes PDDL-generated benchmark validation but fails on naturally worded planning tasks from the same domains without PDDL scaffolding, or the reverse; either outcome would show the recommended protocols measure a different ability than the one being claimed.

Watch

Extended reading notes

Core claim

The discovery this paper asserts is methodological rather than empirical: the rigor that made automated planning reliable is transferable, and it should be applied to LLM-based planners. The paper contends that recent LLM planning work repeatedly reinvents flawed evaluation, using internet-scraped benchmarks whose answers may be in training data, asking language models to grade plans, comparing planners that solve different computational problems, and claiming optimality without soundness guarantees. Its remedy is to make PDDL the interface: represent the task in a formal domain and problem, generate fresh task instances with domain generators, validate every returned plan with an independent validator, and report complexity, soundness, completeness, and resource usage in the style of the International Planning Competition. If the paper is right, a large fraction of current LLM planning results are inflated or uninterpretable, and the field's own tools can fix that.

Load-bearing premise

The load-bearing premise is that performance on PDDL-defined benchmark instances, checked by classical validators, reflects an LLM planner's real ability to solve the natural-language planning problems people care about.

Editorial extensions

If this is right

  • LLM planning papers would be expected to translate tasks into PDDL or a formally defined equivalent and publish the translation, so that claims become checkable.
  • Every proposed plan would need to pass an independent validator before being counted as solved, and unsound planners would have to be paired with such validation.
  • Evaluations would move from static scraped benchmarks to generator-produced instances drawn at evaluation time, reducing memorization and allowing controlled scaling of difficulty.
  • Comparisons would be restricted to planners solving the same computational problem, with explicit statements of soundness, completeness, and optimality.
  • Reviewers would have a concrete checklist for rejecting work that claims better planning without matching protocol, validator use, or data provenance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves open is whether PDDL-mediated success transfers to natural-language planning tasks that never pass through PDDL; that transfer question deserves its own benchmark comparing planners with and without formal scaffolding on the same real-world tasks.
  • Validator feedback could become a training signal rather than an afterthought: an LLM planner that is asked to re-plan until its output validates may converge on more reliable plans, a prediction that can be tested by ablation.
  • The same validator-first discipline could generalize to other LLM domains with checkable outputs, such as code generation or theorem proving, where independent checkers already exist but are not always enforced.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper is a position paper arguing that research on planning with large language models should adopt the formalisms, benchmark protocols, validators, and complexity awareness of the classical automated planning community. It provides a tutorial-style overview of planning terminology, computational problems, benchmark-construction practices, available tools, and evaluation pitfalls, and it makes concrete recommendations such as using PDDL generators, validators, and IPC-style metrics. The central thesis is that applying these established practices would improve the reliability, comparability, and interpretability of LLM-based planning research.

Significance. If accepted, the paper would serve as a useful checklist for LLM-planning researchers and reviewers, addressing a real and growing evaluation crisis in a fast-moving area. Its main strengths are that it grounds its recommendations in decades of planning-community practice, it gives a concise map of planning formalisms and computational problems, and it calls for transparency and validation in concrete terms that go beyond general exhortation. The paper makes no quantitative derivations, so there is no risk of circular fitting; the central position is independently grounded in the external planning literature. The remaining risks are factual accuracy in a few supporting claims and the operational coherence of the proposed contamination remedy, both of which are addressable in revision.

major comments (3)
  1. [Section 4, Common Pitfall 2] The generator-based remedy proposed in Common Pitfall 2 does not deliver the uncontaminated evaluation that the same paragraph declares ideal. The text first states that evaluation should be conducted on novel domains and problem instances that are guaranteed to be unseen during training, then dismisses mystery domains because random names may introduce ambiguity for LLMs, and finally recommends generators for existing PDDL domains. New instances of a known domain do not prevent an LLM that has memorized the domain's action schemas and solution strategies from exploiting domain-level knowledge, which is exactly the kind of memorization the paragraph is concerned about. The paper should either provide a concrete method for constructing novel domains that are suitable for LLMs (for example, systematically generated new domains with controlled vocabulary, or human-authored domains following IPC practice), or explicitly state that generator-based evaluation addresses only instance-level leakage and leaves domain-level contamination unresolved.
  2. [Section 4, Common Pitfall 2; reference [58]] The contamination claim attributed to Hu et al. [58] is not supported by the cited reference. Reference [58] is Text2World: Benchmarking Large Language Models for Symbolic World Model Generation, which is not a contamination study and, as described in the manuscript itself, addresses symbolic world-model generation rather than memorization of PDDL. Because this citation is the only evidence offered for the load-bearing assertion that frontier models have memorized the PDDL, it should be replaced with a genuine contamination or memorization study, or the assertion should be qualified as unsupported.
  3. [Section 3, first paragraph] The statement that all the problems described previously are PSPACE-hard is inaccurate for bounded plan existence: for propositional STRIPS, deciding whether a plan of length or cost at most k exists is NP-complete, not PSPACE-hard. This matters in a paper whose central recommendation is that LLM-planning researchers should report the correct computational problem and its complexity. The sentence should be narrowed to plan existence and cost-optimal variants, with bounded plan existence stated separately.
minor comments (5)
  1. [Section 4 vs Section 5] The number of available PDDL generators is given as over 60 in Section 4, Common Pitfall 2 and as over 50 in Section 5; the count should be unified or the source of the discrepancy explained.
  2. [Section 4, Common Pitfall 3] The phrase the unreliability of the data is most prolonged appears to be a typo for most pronounced; please rephrase it.
  3. [Section 6, Common Pitfall 3] The list of LLM-based planning works cited for lacking soundness guarantees includes methods with different aims, such as prompting strategies, search algorithms, and agent frameworks; the blanket statement that all lack soundness guarantees should be qualified so that each cited work is only used as support for the specific property being claimed.
  4. [Section 2, References to Some Tutorials] The tutorial pointers in Section 2 are given without formal citations or stable URLs; for a published version, full bibliographic entries should be provided so that readers can actually find the tutorials.
  5. [Abstract and Section 1] The claim of an abundance of recent works that replicate and propagate the same pitfalls is asserted repeatedly without concrete examples; adding a small table or pointer to the specific papers discussed later would make the motivation easier to verify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper's central position is grounded in external planning-community practice, not in self-referential derivation.

full rationale

This is a position paper with no quantitative derivation, fitted parameters, or empirical prediction that could reduce to its own inputs by construction. The central claim—that LLM-based planning research should adopt the planning community's formalisms, validators, benchmark protocols, and comparison standards—is supported by decades of external, community-established tools and practices (PDDL, IPC, Fast Downward, VAL, planning.domains, PDDL generators), not by the authors' own results. The self-citations that appear (e.g., Thought of Search, ACPBench, Planutils, PlanBench) are used as pointers to tools, datasets, or examples; they are not load-bearing premises that the argument depends on, and removing them would not collapse the position. The skeptical concern about Section 4—that the 'novel domains' ideal is not fully delivered by generators that instantiate existing domains—is a legitimate correctness/operational gap in the proposal, but it is not circularity: the paper offers a practical remedy without claiming that remedy is equivalent to the ideal, and no quoted passage exhibits a conclusion that is assumed as a premise. No circular step meeting the required quoted-evidence bar was found.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

No numerical fits or invented entities. The paper relies on the value judgment that classical planning rigor transfers to LLM evaluation, and on external contamination studies.

assumptions (2)
  • domain assumption The IPC and classical planning evaluation protocols (standardized unseen domains, validators, complexity-aware comparison) constitute a gold standard of rigor that should govern LLM planning research.
    Invoked throughout Sections 4-6 as the model to follow; the paper does not defend this against alternative evaluation philosophies.
  • domain assumption The empirical claims about benchmark contamination and memorization (e.g., Hu et al. 2025) are sufficient to generalize concerns to the named datasets.
    Section 4, Common Pitfall 2, builds the recommendation for generator-based evaluation on these external findings without independent verification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Make Planning Research Rigorous Again!." pith.science (2026). https://pith.science/paper/LDQRI7M2

@misc{pith2026250521674,
  author       = {Pith},
  title        = {Pith review of: Make Planning Research Rigorous Again!},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LDQRI7M2}},
  note         = {Machine review of arXiv:2505.21674}
}
read the original abstract

In over sixty years since its inception, the field of planning has made significant contributions to both the theory and practice of building planning software that can solve a never-before-seen planning problem. This was done through established practices of rigorous design and evaluation of planning systems. It is our position that this rigor should be applied to the current trend of work on planning with large language models. One way to do so is by correctly incorporating the insights, tools, and data from the automated planning community into the design and evaluation of LLM-based planners. The experience and expertise of the planning community are not just important from a historical perspective; the lessons learned could play a crucial role in accelerating the development of LLM-based planners. This position is particularly important in light of the abundance of recent works that replicate and propagate the same pitfalls that the planning community has encountered and learned from. We believe that avoiding such known pitfalls will contribute greatly to the progress in building LLM-based planners and to planning in general.

Figures

Figures reproduced from arXiv: 2505.21674 by the authors.

Figure 1
Figure 1. Benchmarks used in the literature for LLM-based Planning evaluations [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

121 extracted references · 61 canonical work pages

  1. [58]

    Text2world: Benchmarking large language models for symbolic world model generation, 2025

    Mengkang Hu, Tianxing Chen, Yude Zou, Yuheng Lei, Qiguang Chen, Ming Li, Yao Mu, Hongyuan Zhang, Wenqi Shao, and Ping Luo. Text2world: Benchmarking large language models for symbolic world model generation, 2025. URL https://arxiv.org/abs/ 2502.13092

    arXiv 2025

  2. [1]

    Oversubscription planning: Complexity and compilabil- ity

    Meysam Aghighi and Peter Jonsson. Oversubscription planning: Complexity and compilabil- ity. In Carla E. Brodley and Peter Stone, editors,Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI 2014), pages 2221–2227. AAAI Press, 2014

    2014

  3. [2]

    Translating HTNs to PDDL: A small amount of domain knowledge can go a long way

    Ron Alford, Ugur Kuter, and Dana Nau. Translating HTNs to PDDL: A small amount of domain knowledge can go a long way. In Craig Boutilier, editor,Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI 2009), pages 1629–1634. AAAI Press, 2009

    2009

  4. [3]

    The AIPS’00 planning competition.AI Magazine, 22(3):47–56, 2001

    Fahiem Bacchus. The AIPS’00 planning competition.AI Magazine, 22(3):47–56, 2001

    2001

  5. [4]

    Planning for temporally extended goals.Annals of Mathematics and Artificial Intelligence, 22(1,1):5–27, 1998

    Fahiem Bacchus and Froduald Kabanza. Planning for temporally extended goals.Annals of Mathematics and Artificial Intelligence, 22(1,1):5–27, 1998

    1998

  6. [5]

    Planning in polynomial time: the SAS-PUBS class

    Christer Bäckström and Inger Klein. Planning in polynomial time: the SAS-PUBS class. Computational Intelligence, 7(3):181–197, 1991

    1991

  7. [6]

    Complexity results for SAS + planning.Computa- tional Intelligence, 11(4):625–655, 1995

    Christer Bäckström and Bernhard Nebel. Complexity results for SAS + planning.Computa- tional Intelligence, 11(4):625–655, 1995

    1995

  8. [7]

    Baier and Sheila A

    Jorge A. Baier and Sheila A. McIlraith. Planning with temporally extended goals using heuristic search. In Derek Long, Stephen F. Smith, Daniel Borrajo, and Lee McCluskey, editors,Proceedings of the Sixteenth International Conference on Automated Planning and Scheduling (ICAPS 2006), pages 342–345. AAAI Press, 2006

    2006

Show all 121 references
  1. [8]

    Baier and Sheila A

    Jorge A. Baier and Sheila A. McIlraith. Planning with preferences.AI Magazine, 29(4):25–36, 2008

    2008

  2. [9]

    Avrim Blum and Merrick L. Furst. Fast planning through planning graph analysis. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI 1995), pages 1636–1642. Morgan Kaufmann, 1995

    1995

  3. [10]

    Planning with incomplete information as heuristic search in belief space

    Blai Bonet and Héctor Geffner. Planning with incomplete information as heuristic search in belief space. In Steve Chien, Subbarao Kambhampati, and Craig A. Knoblock, editors, Proceedings of the Fifth International Conference on Artificial Intelligence Planning and Scheduling (...

    2000

  4. [11]

    Benchagents: Automated benchmark creation with agent interaction.arXiv preprint arXiv:2410.22584, 2024

    Natasha Butt, Varun Chandrasekaran, Neel Joshi, Besmira Nushi, and Vidhisha Balachan- dran. Benchagents: Automated benchmark creation with agent interaction.arXiv preprint arXiv:2410.22584, 2024

    2024

  5. [12]

    The computational complexity of propositional STRIPS planning.Artificial Intelligence, 69(1–2):165–204, 1994

    Tom Bylander. The computational complexity of propositional STRIPS planning.Artificial Intelligence, 69(1–2):165–204, 1994

    1994

  6. [13]

    Act local, think global: Width notions for tractable planning

    Hubie Chen and Omer Giménez. Act local, think global: Width notions for tractable planning. In Mark Boddy, Maria Fox, and Sylvie Thiébaux, editors,Proceedings of the Seventeenth International Conference on Automated Planning and Scheduling (ICAPS 2007), pages 73–80. AAAI Press, 2007

    2007

  7. [14]

    Completeness and optimality preserving reduction for planning

    Yixin Chen and Guohui Yao. Completeness and optimality preserving reduction for planning. In Craig Boutilier, editor,Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI 2009), pages 1659–1664. AAAI Press, 2009. 10

    2009

  8. [15]

    Planning via model checking: A decision procedure forAR

    Alessandro Cimatti, Fausto Giunchiglia, Enrico Giunchiglia, and Paolo Traverso. Planning via model checking: A decision procedure forAR. In Sam Steel and Rachid Alami, editors, Recent Advances in AI Planning. 4th European Conference on Planning (ECP 1997), volume 1348 ofLectur...

    1997

  9. [16]

    Stephen A. Cook. The complexity of theorem-proving procedures. In Michael A. Harrison, Ranan B. Banerji, and Jeffrey D. Ullman, editors,Proceedings of the 3rd Annual ACM Symposium on the Theory of Computing (STOC 1971), pages 151–158. ACM, 1971

    1971

  10. [17]

    Textworld: A learning environment for text-based games

    Marc-Alexandre Côté, Akos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, Wendy Tay, and Adam Trischler. Textworld: A learning environment for text-based games. InComputer Games: 7th Workshop, CGW 2018...

    2018

  11. [18]

    Culberson

    Joseph C. Culberson. Sokoban is PSPACE-complete. Technical Report TR 97-02, Department of Computing Science, The University of Alberta, Edmonton, Alberta, Canada, 1997

    1997

  12. [19]

    Giuseppe De Giacomo and Moshe Y . Vardi. Linear temporal logic and linear dynamic logic on finite traces. In Francesca Rossi, editor,Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI 2013), pages 854–860. AAAI Press, 2013

    2013

  13. [20]

    Generalized best-first search strategies and the optimality of A∗.Journal of the ACM, 32(3):505–536, 1985

    Rina Dechter and Judea Pearl. Generalized best-first search strategies and the optimality of A∗.Journal of the ACM, 32(3):505–536, 1985

    1985

  14. [21]

    Planning as a service

    Yi Ding, Cam Cunningham, Christian Muise, and Nir Lipovetzky. Planning as a service. In ICAPS 2023 System Demonstrations and Exhibits, 2023

    2023

  15. [22]

    Novelty vs

    Simon Dold and Malte Helmert. Novelty vs. potential heuristics: A comparison of hardness measures for satisficing planning. In Jennifer Dy and Sriraam Natarajan, editors,Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI 2024), pages 20692– 20699...

    2024

  16. [23]

    Probabilistic planning via heuristic forward search and weighted model counting.Journal of Artificial Intelligence Research, 30:565–620, 2007

    Carmel Domshlak and Jörg Hoffmann. Probabilistic planning via heuristic forward search and weighted model counting.Journal of Artificial Intelligence Research, 30:565–620, 2007

    2007

  17. [24]

    Deterministic oversubscription planning as heuristic search: Abstractions and reformulations.Journal of Artificial Intelligence Research, 52: 97–169, 2015

    Carmel Domshlak and Vitaly Mirkis. Deterministic oversubscription planning as heuristic search: Abstractions and reformulations.Journal of Artificial Intelligence Research, 52: 97–169, 2015

    2015

  18. [25]

    Red-black planning: A new systematic approach to partial delete relaxation.Artificial Intelligence, 221:73–114, 2015

    Carmel Domshlak, Jörg Hoffmann, and Michael Katz. Red-black planning: A new systematic approach to partial delete relaxation.Artificial Intelligence, 221:73–114, 2015

    2015

  19. [26]

    SOKOBAN and other motion planning problems.Computational Geometry, 13:215–228, 1999

    Dorit Dor and Uri Zwick. SOKOBAN and other motion planning problems.Computational Geometry, 13:215–228, 1999

    1999

  20. [27]

    Planning with pattern databases

    Stefan Edelkamp. Planning with pattern databases. In Amedeo Cesta and Daniel Borrajo, editors,Proceedings of the Sixth European Conference on Planning (ECP 2001), pages 84–90. AAAI Press, 2001

    2001

  21. [28]

    PDDL2.2: The language for the classical part of the 4th International Planning Competition

    Stefan Edelkamp and Jörg Hoffmann. PDDL2.2: The language for the classical part of the 4th International Planning Competition. Technical Report 195, University of Freiburg, Department of Computer Science, 2004

    2004

  22. [29]

    Hendler, and Dana S

    Kutluhan Erol, James A. Hendler, and Dana S. Nau. HTN planning: Complexity and expres- sivity. InProceedings of the Twelfth National Conference on Artificial Intelligence (AAAI 1994), pages 1123–1128. AAAI Press, 1994

    1994

  23. [30]

    Fikes and Nils J

    Richard E. Fikes and Nils J. Nilsson. STRIPS: A new approach to the application of theorem proving to problem solving.Artificial Intelligence, 2:189–208, 1971

    1971

  24. [31]

    The detection and exploitation of symmetry in planning problems

    Maria Fox and Derek Long. The detection and exploitation of symmetry in planning problems. In Thomas Dean, editor,Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI 1999), pages 956–961. Morgan Kaufmann, 1999

    1999

  25. [32]

    PDDL2.1: An extension to PDDL for expressing temporal planning domains.Journal of Artificial Intelligence Research, 20:61–124, 2003

    Maria Fox and Derek Long. PDDL2.1: An extension to PDDL for expressing temporal planning domains.Journal of Artificial Intelligence Research, 20:61–124, 2003

    2003

  26. [33]

    Tarski: An AI planning modeling framework.https://github.com/aig-upf/tarski, 2018

    Guillem Francés, Miquel Ramirez, and Collaborators. Tarski: An AI planning modeling framework.https://github.com/aig-upf/tarski, 2018. 11

    2018

  27. [34]

    Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah D. Goodman. Stream of Search (SoS): Learning to search in language. arXiv:2404.03683 [cs.LG], 2024

    2024 arXiv

  28. [35]

    Functional Strips: A more flexible language for planning and problem solving

    Héctor Geffner. Functional Strips: A more flexible language for planning and problem solving. In Jack Minker, editor,Logic-Based Artificial Intelligence, volume 597 ofKluwer International Series In Engineering And Computer Science, chapter 9, pages 187–209. Kluwer, Dordrecht, 2000

    2000

  29. [36]

    Gerevini, Patrik Haslum, Derek Long, Alessandro Saetti, and Yannis Dimopou- los

    Alfonso E. Gerevini, Patrik Haslum, Derek Long, Alessandro Saetti, and Yannis Dimopou- los. Deterministic planning in the fifth international planning competition: PDDL3 and experimental evaluation of the planners.Artificial Intelligence, 173(5–6):619–668, 2009

    2009

  30. [37]

    Gerevini, Alessandro Saetti, and Ivan Serina

    Alfonso E. Gerevini, Alessandro Saetti, and Ivan Serina. Temporal planning with problems requiring concurrency through action graphs and local search. In Ronen Brafman, Héctor Geffner, Jörg Hoffmann, and Henry Kautz, editors,Proceedings of the Twentieth International Conferenc...

    2010

  31. [38]

    NL2Plan: Robust LLM-driven planning from minimal text descriptions

    Elliot Gestrin, Marco Kuhlmann, and Jendrik Seipp. NL2Plan: Robust LLM-driven planning from minimal text descriptions. InICAPS 2024 Workshop on Human-Aware and Explainable Planning (HAXP), 2024

    2024

  32. [39]

    Nau, and Paolo Traverso.Automated planning - theory and practice

    Malik Ghallab, Dana S. Nau, and Paolo Traverso.Automated planning - theory and practice. Elsevier, 2004. ISBN 978-1-55860-856-6

    2004

  33. [40]

    Nau, and Paolo Traverso.Automated Planning and Acting

    Malik Ghallab, Dana S. Nau, and Paolo Traverso.Automated Planning and Acting. Cambridge University Press, 2016. ISBN 978-1-107-03727-4

    2016

  34. [41]

    Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati. Leveraging pre-trained large language models to construct and utilize world models for model-based task planning.Advances in Neural Information Processing Systems, 36:79081–79094, 2023

    2023

  35. [42]

    Actionrea- soningbench: Reasoning about actions with and without ramification constraints

    Divij Handa, Pavel Dolin, Shrinidhi Kumbhar, Chitta Baral, and Tran Cao Son. Actionrea- soningbench: Reasoning about actions with and without ramification constraints. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  36. [43]

    Hart, Nils J

    Peter E. Hart, Nils J. Nilsson, and Bertram Raphael. A formal basis for the heuristic determi- nation of minimum cost paths.IEEE Transactions on Systems Science and Cybernetics, 4(2): 100–107, 1968

    1968

  37. [44]

    INV AL: the Independent PDDL plan Validator.https://github.com/ patrikhaslum/INVAL, 2016

    Patrik Haslum. INV AL: the Independent PDDL plan Validator.https://github.com/ patrikhaslum/INVAL, 2016. Accessed May 2, 2025

    2016

  38. [45]

    Morgan & Claypool, 2019

    Patrik Haslum, Nir Lipovetzky, Daniele Magazzeni, and Christian Muise.An Introduction to the Planning Domain Definition Language, volume 13 ofSynthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool, 2019

    2019

  39. [46]

    Exploring the capacity of pretrained language models for reasoning about actions and change

    Weinan He, Canming Huang, Zhanhao Xiao, and Yongmei Liu. Exploring the capacity of pretrained language models for reasoning about actions and change. InACL. Association for Computational Linguistics, 2023

    2023

  40. [47]

    Hearn and Erik D

    Robert A. Hearn and Erik D. Demaine. PSPACE-completeness of sliding-block puzzles and other problems through the nondeterministic constraint logic model of computation. Theoretical Computer Science, 343(1–2):72–96, 2005

    2005

  41. [48]

    Decidability and undecidability results for planning with numerical state variables

    Malte Helmert. Decidability and undecidability results for planning with numerical state variables. In Malik Ghallab, Joachim Hertzberg, and Paolo Traverso, editors,Proceedings of the Sixth International Conference on Artificial Intelligence Planning and Scheduling (AIPS 2002)...

    2002

  42. [49]

    Complexity results for standard benchmark domains in planning.Artificial Intelligence, 143(2):219–262, 2003

    Malte Helmert. Complexity results for standard benchmark domains in planning.Artificial Intelligence, 143(2):219–262, 2003

    2003

  43. [50]

    New complexity results for classical planning benchmarks

    Malte Helmert. New complexity results for classical planning benchmarks. In Derek Long, Stephen F. Smith, Daniel Borrajo, and Lee McCluskey, editors,Proceedings of the Sixteenth International Conference on Automated Planning and Scheduling (ICAPS 2006), pages 52–61. AAAI Press...

    2006

  44. [51]

    The Fast Downward planning system.Journal of Artificial Intelligence Research, 26:191–246, 2006

    Malte Helmert. The Fast Downward planning system.Journal of Artificial Intelligence Research, 26:191–246, 2006

    2006

  45. [52]

    Concise finite-domain representations for PDDL planning tasks.Artificial Intelligence, 173:503–535, 2009

    Malte Helmert. Concise finite-domain representations for PDDL planning tasks.Artificial Intelligence, 173:503–535, 2009

    2009

  46. [53]

    The FF planning system: Fast plan generation through heuristic search.Journal of Artificial Intelligence Research, 14:253–302, 2001

    Jörg Hoffmann and Bernhard Nebel. The FF planning system: Fast plan generation through heuristic search.Journal of Artificial Intelligence Research, 14:253–302, 2001

    2001

  47. [54]

    HTN-MAKER: Learning HTNs with minimal additional knowledge engineering required

    Chad Hogg, Héctor Muñoz-Avila, and Ugur Kuter. HTN-MAKER: Learning HTNs with minimal additional knowledge engineering required. InProceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI 2008), pages 950–956. AAAI Press, 2008

    2008

  48. [55]

    Learning hierarchical task networks for nondeterministic planning domains

    Chad Hogg, Ugur Kuter, and Héctor Muñoz-Avila. Learning hierarchical task networks for nondeterministic planning domains. In Craig Boutilier, editor,Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI 2009), pages 1708–1714. AAAI Press, 2009

    2009

  49. [56]

    Learning methods to generate good plans: Integrating HTN learning and reinforcement learning

    Chad Hogg, Ugur Kuter, and Héctor Muñoz-Avila. Learning methods to generate good plans: Integrating HTN learning and reinforcement learning. In Maria Fox and David Poole, editors, Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), pages 15...

    2010

  50. [57]

    V AL’s progress: The automatic validation tool for PDDL2.1 used in the International Planning Competition

    Richard Howey and Derek Long. V AL’s progress: The automatic validation tool for PDDL2.1 used in the International Planning Competition. In Stefan Edelkamp and Jörg Hoffmann, editors,Proceedings of the ICAPS 2003 Workshop on the Competition: Impact, Organisation, Evaluation, B...

    2003

  51. [59]

    Tractable plan existence does not imply tractable plan generation.Annals of Mathematics and Artificial Intelligence, 22(3,4):281–296, 1998

    Peter Jonsson and Christer Bäckström. Tractable plan existence does not imply tractable plan generation.Annals of Mathematics and Artificial Intelligence, 22(3,4):281–296, 1998

    1998

  52. [60]

    Corrêa, Florian Pommerening, and Gabriele Röger

    Lucas Galery Käser, Clemens Büchner, Augusto B. Corrêa, Florian Pommerening, and Gabriele Röger. Machetli: Simplifying input files for debugging. InICAPS 2022 System Demonstrations and Exhibits, 2022

    2022

  53. [61]

    New islands of tractability of cost-optimal planning

    Michael Katz and Carmel Domshlak. New islands of tractability of cost-optimal planning. Journal of Artificial Intelligence Research, 32:203–288, 2008

    2008

  54. [62]

    Implicit abstraction heuristics.Journal of Artificial Intelligence Research, 39:51–126, 2010

    Michael Katz and Carmel Domshlak. Implicit abstraction heuristics.Journal of Artificial Intelligence Research, 39:51–126, 2010

    2010

  55. [63]

    In search of tractability for partial satisfaction planning

    Michael Katz and Vitaly Mirkis. In search of tractability for partial satisfaction planning. In Subbarao Kambhampati, editor,Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI 2016), pages 3154–3160. AAAI Press, 2016

    2016

  56. [64]

    Thought of search: Planning with language models through the lens of efficiency

    Michael Katz, Harsha Kokel, Kavitha Srinivas, and Shirin Sohrabi. Thought of search: Planning with language models through the lens of efficiency. InProceedings of the Thirty- Eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024), pages 138491–138568, 2024

    2024

  57. [65]

    Pushing the envelope: Planning, propositional logic, and stochastic search

    Henry Kautz and Bart Selman. Pushing the envelope: Planning, propositional logic, and stochastic search. InProceedings of the Thirteenth National Conference on Artificial Intelli- gence (AAAI 1996), pages 1194–1201. AAAI Press, 1996

    1996

  58. [66]

    Soft goals can be compiled away.Journal of Artificial Intelligence Research, 36:547–556, 2009

    Emil Keyder and Héctor Geffner. Soft goals can be compiled away.Journal of Artificial Intelligence Research, 36:547–556, 2009

    2009

  59. [67]

    ACPBench: Reasoning about action, change, and planning

    Harsha Kokel, Michael Katz, Kavitha Srinivas, and Shirin Sohrabi. ACPBench: Reasoning about action, change, and planning. In Julie Shah and Zico Kolter, editors,Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI 2025). AAAI Press, 2025

    2025

  60. [68]

    Nau, Elnatan Reisner, and Robert P

    Ugur Kuter, Dana S. Nau, Elnatan Reisner, and Robert P. Goldman. Using classical planners to solve nondeterministic planning problems. In Jussi Rintanen, Bernhard Nebel, J. Christopher Beck, and Eric Hansen, editors,Proceedings of the Eighteenth International Conference on Aut...

    2008

  61. [69]

    Beyond a ∗: Better planning with transformers via search dynamics bootstrapping

    Lucas Lehnert, Sainbayar Sukhbaatar, DiJia Su, Qinqing Zheng, Paul McVay, Michael Rabbat, and Yuandong Tian. Beyond a ∗: Better planning with transformers via search dynamics bootstrapping. InProceedings of the First Conference on Language Modeling (COLM 2024), 2024

    2024

  62. [70]

    The deterministic part of the seventh international planning competition.Artificial Intelligence, 223:82–119, 2015

    Carlos Linares López, Sergio Jiménez Celorrio, and Angel García Olaya. The deterministic part of the seventh international planning competition.Artificial Intelligence, 223:82–119, 2015

    2015

  63. [71]

    Width and serialization of classical planning problems

    Nir Lipovetzky and Hector Geffner. Width and serialization of classical planning problems. In Luc De Raedt, Christian Bessiere, Didier Dubois, Patrick Doherty, Paolo Frasconi, Fredrik Heintz, and Peter Lucas, editors,Proceedings of the 20th European Conference on Artificial In...

    2012

  64. [72]

    Michael L. Littman. Probabilistic propositional planning: Representations and complexity. InProceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI 1997), pages 748–754. AAAI Press, 1997

    1997

  65. [73]

    LLM+P: empowering large language models with optimal planning proficiency.CoRR, abs/2304.11477, 2023

    Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. LLM+P: empowering large language models with optimal planning proficiency.CoRR, abs/2304.11477, 2023

    2023 arXiv

  66. [74]

    Agentbench: Evaluating llms as agents.CoRR, abs/2308.03688, 2023

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. Agentbench:...

    2023 arXiv

  67. [75]

    Exploiting a graphplan framework in temporal planning

    Derek Long and Maria Fox. Exploiting a graphplan framework in temporal planning. In Enrico Giunchiglia, Nicola Muscettola, and Dana Nau, editors,Proceedings of the Thirteenth International Conference on Automated Planning and Scheduling (ICAPS 2003), pages 51–62. AAAI Press, 2003

    2003

  68. [76]

    Anderson, Daniel S

    Derek Long, Henry Kautz, Bart Selman, Blai Bonet, Héctor Geffner, Jana Koehler, Michael Brenner, Jörg Hoffmann, Frank Rittinger, Corin R. Anderson, Daniel S. Weld, David E. Smith, and Maria Fox. The AIPS-98 planning competition.AI Magazine, 21(2):13–33, 2000

    2000

  69. [77]

    Sudoku as a sat problem

    Inês Lynce and Joël Ouaknine. Sudoku as a sat problem. InAI&M. Citeseer, 2006

    2006

  70. [78]

    Agentboard: An analytical evaluation board of multi-turn LLM agents.CoRR, abs/2401.13178, 2024

    Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn LLM agents.CoRR, abs/2401.13178, 2024

    2024 arXiv

  71. [79]

    Tenenbaum, and Leslie Pack Kaelbing

    Jiayuan Mao, Tomas Lozano-Perez, Joshua B. Tenenbaum, and Leslie Pack Kaelbing. What planning problems can a relational neural network solve? InProceedings of the Thirty- Seventh Annual Conference on Neural Information Processing Systems (NeurIPS 2023), pages 59522–59542, 2023

    2023

  72. [80]

    Pattern database heuristics for fully observable nondeterministic planning

    Robert Mattmüller, Manuela Ortlieb, Malte Helmert, and Pascal Bercher. Pattern database heuristics for fully observable nondeterministic planning. In Ronen Brafman, Héctor Geffner, Jörg Hoffmann, and Henry Kautz, editors,Proceedings of the Twentieth International Con- ference ...

    2010

  73. [81]

    Synthesis Lectures on Artificial Intelligence and Machine Learning

    Mausam and Andrey Kolobov.Planning with Markov Decision Processes: An AI Perspective. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool, 2012

    2012

  74. [82]

    PDDL – The Planning Domain Definition Language – Version 1.2

    Drew McDermott, Malik Ghallab, Adele Howe, Craig Knoblock, Ashwin Ram, Manuela Veloso, Daniel Weld, and David Wilkins. PDDL – The Planning Domain Definition Language – Version 1.2. Technical Report CVC TR-98-003/DCS TR-1165, Yale Center for Computational Vision and Control, Ya...

    1998

  75. [83]

    Planning.Domains

    Christian Muise. Planning.Domains. InICAPS 2016 System Demonstrations and Exhibits,

    2016

  76. [84]

    Planutils: Bringing planning to the masses

    Christian Muise, Florian Pommerening, Jendrik Seipp, and Michael Katz. Planutils: Bringing planning to the masses. InICAPS 2022 System Demonstrations and Exhibits, 2022. 14

    2022

  77. [85]

    Muise, Sheila A

    Christian J. Muise, Sheila A. McIlraith, and J. Christopher Beck. Improved non-deterministic planning by exploiting state relevance. In Lee McCluskey, Brian Williams, José Reinaldo Silva, and Blai Bonet, editors,Proceedings of the Twenty-Second International Conference on Auto...

    2012

  78. [86]

    Muise, Sheila A

    Christian J. Muise, Sheila A. McIlraith, and Vaishak Belle. Non-deterministic planning with conditional effects. In Steve Chien, Alan Fern, Wheeler Ruml, and Minh Do, editors, Proceedings of the Twenty-Fourth International Conference on Automated Planning and Scheduling (ICAPS...

    2014

  79. [87]

    Nau, Tsz-Chiu Au, Okhtay Ilghami, Ugur Kuter, J

    Dana S. Nau, Tsz-Chiu Au, Okhtay Ilghami, Ugur Kuter, J. William Murdock, Dan Wu, and Fusun Yaman. SHOP2: An HTN planning system.Journal of Artificial Intelligence Research, 20:379–404, 2003

    2003

  80. [88]

    Large language models as planning domain generators

    James Oswald, Kavitha Srinivas, Harsha Kokel, Junkyu Lee, Michael Katz, and Shirin Sohrabi. Large language models as planning domain generators. In Sara Bernardini and Christian Muise, editors,Proceedings of the Thirty-Fourth International Conference on Automated Planning and ...

    2024

  81. [89]

    Compiling uncertainty away in conformant planning problems with bounded width.Journal of Artificial Intelligence Research, 35:623–675, 2009

    Hector Palacios and Hector Geffner. Compiling uncertainty away in conformant planning problems with bounded width.Journal of Artificial Intelligence Research, 35:623–675, 2009

    2009

  82. [90]

    Planning with numeric timed initial fluents

    Chiara Piacentini, Maria Fox, and Derek Long. Planning with numeric timed initial fluents. In Blai Bonet and Sven Koenig, editors,Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI 2015), pages 4196–4197. AAAI Press, 2015

    2015

  83. [91]

    Rosenschein

    Nir Pochter, Aviv Zohar, and Jeffrey S. Rosenschein. Exploiting problem symmetries in state- based planners. In Wolfram Burgard and Dan Roth, editors,Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2011), pages 1004–1009. AAAI Press, 2011

    2011

  84. [92]

    MIT Press, 2001

    Raymond Reiter.Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems. MIT Press, 2001

    2001

  85. [93]

    System 1.x: Learning to balance fast and slow planning with language models

    Swarnadeep Saha, Archiki Prasad, Justin Chen, Peter Hase, Elias Stengel-Eskin, and Mohit Bansal. System 1.x: Learning to balance fast and slow planning with language models. In Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025). Open...

    2025

  86. [94]

    Relational dynamic influence diagram language (RDDL): Language description, 2010

    Scott Sanner. Relational dynamic influence diagram language (RDDL): Language description, 2010

    2010

  87. [95]

    PDDL generators

    Jendrik Seipp, Álvaro Torralba, and Jörg Hoffmann. PDDL generators. https://doi. org/10.5281/zenodo.6382173, 2022

    2022 doi

  88. [96]

    Al- gorithm of thoughts: Enhancing exploration of ideas in large language models.CoRR, abs/2308.10379, 2023

    Bilgehan Sel, Ahmad Al-Tawaha, Vanshaj Khattar, Lu Wang, Ruoxi Jia, and Ming Jin. Al- gorithm of thoughts: Enhancing exploration of ideas in large language models.CoRR, abs/2308.10379, 2023

    2023 arXiv

  89. [97]

    Alfworld: Aligning text and embodied environments for interactive learning

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Cote, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. InProceedings of the Nineth International Conference on Learning Representations (ICLR 2021). Ope...

    2021

  90. [98]

    Generalized planning in PDDL domains with pretrained large language mod- els

    Tom Silver, Soham Dan, Kavitha Srinivas, Josh Tenenbaum, Leslie Pack Kaelbling, and Michael Katz. Generalized planning in PDDL domains with pretrained large language mod- els. In Jennifer Dy and Sriraam Natarajan, editors,Proceedings of the Thirty-Eighth AAAI Conference on Art...

    2024

  91. [99]

    Sudoku as a constraint problem

    Helmut Simonis. Sudoku as a constraint problem. InCP Workshop on modeling and refor- mulating Constraint Satisfaction Problems, volume 12, pages 13–27. Citeseer Sitges, Spain, 2005

    2005

  92. [100]

    The complexity of propositional linear temporal logics.Journal of the ACM, 32(3):733–749, 1985

    Aravinda Prasad Sistla and Edmund Melson Clarke. The complexity of propositional linear temporal logics.Journal of the ACM, 32(3):733–749, 1985

    1985

  93. [101]

    David E. Smith. Choosing objectives in over-subscription planning. In Shlomo Zilberstein, Jana Koehler, and Sven Koenig, editors,Proceedings of the Fourteenth International Conference on Automated Planning and Scheduling (ICAPS 2004), pages 393–401. AAAI Press, 2004. 15

    2004

  94. [102]

    Smith and Daniel S

    David E. Smith and Daniel S. Weld. Conformant graphplan. In Charles Rich and Jack Mostow, editors,Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI 1998), pages 889–896. AAAI Press, 1998

    1998

  95. [103]

    Baier, and Sheila A

    Shirin Sohrabi, Jorge A. Baier, and Sheila A. McIlraith. HTN planning with preferences. In Craig Boutilier, editor,Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI 2009), pages 1790–1797. AAAI Press, 2009

    2009

  96. [104]

    Qualitative numeric planning

    Siddharth Srivastava, Shlomo Zilberstein, Neil Immerman, and Hector Geffner. Qualitative numeric planning. In Wolfram Burgard and Dan Roth, editors,Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2011), pages 1010–1016. AAAI Press, 2011

    2011

  97. [105]

    Automating the genera- tion of prompts for llm-based action choice in pddl planning

    Katharina Stein, Daniel Fišer, Jörg Hoffmann, and Alexander Koller. Automating the genera- tion of prompts for llm-based action choice in pddl planning. InProceedings of the Fourteenth International Conference on Automated Planning and Scheduling (ICAPS 2025). AAAI Press, 2025

    2025

  98. [106]

    The 2023 International Planning Competition.AI Magazine, 45(2):280–296,

    Ayal Taitler, Ron Alford, Joan Espasa, Gregor Behnke, Daniel Fišer, Michael Gimelfarb, Florian Pommerening, Scott Sanner, Enrico Scala, Dominik Schreiber, Javier Segovia-Aguas, and Jendrik Seipp. The 2023 International Planning Competition.AI Magazine, 45(2):280–296,

    2023

  99. [107]

    LLMs as planning modelers: A survey for leveraging large language models to construct automated planning models.CoRR, abs/2503.18971, 2025

    Marcus Tantakoun, Xiaodan Zhu, and Christian Muise. LLMs as planning modelers: A survey for leveraging large language models to construct automated planning models.CoRR, abs/2503.18971, 2025

    2025

  100. [108]

    McCluskey, editors.The Eighth International Planning Competition: Description of Participant Planners of the Deterministic Track, 2014

    Mauro Vallati, Lukáš Chrpa, and Thomas L. McCluskey, editors.The Eighth International Planning Competition: Description of Participant Planners of the Deterministic Track, 2014

    2014

  101. [109]

    Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. InProceedings of the Thirty-Seventh Annual Conference on Neural Inform...

    2023

  102. [110]

    Do, and Subbarao Kambhampati

    Menkes van den Briel, Romeo Sanchez, Minh B. Do, and Subbarao Kambhampati. Effec- tive approaches for partial satisfaction (over-subscription) planning. InProceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI 2004), pages 562–569. AAAI Press, 2004

    2004

  103. [111]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InProceedings of the Thirty-Sixth Annual Conference on Neural Information Processing Sy...

    2022

  104. [112]

    Travelplanner: A benchmark for real-world planning with language agents

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. Travelplanner: A benchmark for real-world planning with language agents. In ICML. OpenReview.net, 2024

    2024

  105. [113]

    Learning action models from plan examples using weighted MAX-SAT.Artificial Intelligence, 171:107–143, 2007

    Qiang Yang, Kangheng Wu, and Yunfei Jiang. Learning action models from plan examples using weighted MAX-SAT.Artificial Intelligence, 171:107–143, 2007

    2007

  106. [114]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Proceedings of the Thirty-Seventh Annual Conference on Neural Information Processing Systems (NeurIPS ...

    2023

  107. [115]

    ReAct: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InProceedings of the Eleventh International Conference on Learning Representations (ICLR 2023). OpenReview.net, 2023

    2023

  108. [116]

    Probabilistic planning via determinization in hindsight

    Sungwook Yoon, Alan Fern, Robert Givan, and Subbarao Kambhampati. Probabilistic planning via determinization in hindsight. InProceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI 2008), pages 1010–1016. AAAI Press, 2008

    2008

  109. [117]

    Håkan L. S. Younes, Michael L. Littman, David Weissman, and John Asmuth. The first probabilistic track of the international planning competition.Journal of Artificial Intelligence Research, 24:851–887, 2005. 16

    2005

  110. [118]

    Le, Ed H

    Huaixiu Steven Zheng, Swaroop Mishra, Hugh Zhang, Xinyun Chen, Minmin Chen, Azade Nova, Le Hou, Heng-Tze Cheng, Quoc V . Le, Ed H. Chi, and Denny Zhou. NATURAL PLAN: benchmarking llms on natural language planning.CoRR, abs/2406.04520, 2024

    2024 arXiv

  111. [119]

    Littman, and Stephen H

    Max Zuo, Francisco Piedrahita Velez, Xiaochen Li, Michael L. Littman, and Stephen H. Bach. Planetarium: A rigorous benchmark for translating text to structured planning languages. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association f...

    2025

  112. [2016]

    https://api.planning.domains

  113. [2024]

    doi: 10.1002/aaai.12169

    doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.