Pith. sign in

REVIEW 4 major objections 6 minor 42 references

MINDSTORES: Memory-Informed Neural Decision Synthesis for Task-Oriented Reinforcement in Embodied Systems

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MINDSTORES claims that retrieving past natural-language experiences lifts an LLM planner's Minecraft success rate by 9.4 percent.

desk verdict MINDSTORES has a sensible and clearly described experience-memory mechanism for LLM planners, but the headline 9.4% gain over existing methods is not established because the only comparison is against the authors' own weakened DEPS reimplementation. read the letter →

arxiv 2501.19318 v4 pith:3ZFFFB7T submitted 2025-01-31 cs.AI

Anirudh Chari , Suraj Reddy , Aditya Tiwari , Richard Lian , Brian Zhou This is my paper · ORCID
classification cs.AI
keywords experience-augmentedplanninglargelanguagemodelsembodiedagentsmentalMinecraftMineDojomemoryretrievalzero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that an LLM-based planner for embodied agents can learn continuously by keeping a retrievable record of its own past attempts, and that this record makes it plan better in open-world environments. The central proposal, MINDSTORES, stores each attempt as a natural-language (state, task, plan, outcome) tuple, retrieves the most similar past experiences before planning, and has the LLM reason over them to generate and refine plans. On MineDojo's eight complexity tiers, the paper reports a mean success-rate improvement of 9.4 percent over its own DEPS-style LLM-planning baseline, with the largest gains on intermediate tasks. If correct, this is a path to agents that build persistent mental models without giving up the flexibility of zero-shot LLM planning.

What carries the argument

The central object is the experience database $\mathcal{D}$ of tuples $(s,t,p,o)$, where $s$ is a natural-language state description, $t$ a task, $p$ a plan, and $o$ the observed outcome. Each component is embedded with a Sentence-BERT model, and retrieval uses cosine similarity weighted as $\lambda_s = 0.4$, $\lambda_t = 0.4$, $\lambda_p = 0.2$ over the top $k=5$ entries. The planning loop first retrieves similar experiences to generate insights, then retrieves similar plans to predict the outcome of the proposed plan, revises the plan if failure is predicted, executes it, and stores the new tuple. DEPS, the underlying zero-shot planner, is the iterative Describe-Explain-Plan-Select architecture that MINDSTORES extends with this memory loop.

What would settle it

Run MINDSTORES and the original DEPS controller on the same MineDojo tasks with the same random seeds and episode lengths, and compare success rates task by task. If the mean gap drops well below 9.4 percentage points, or if disabling only the experience database leaves performance unchanged, the central claim is not supported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that experiential memory, represented as plain-language tuples and retrieved by semantic similarity, lets an LLM planner improve across episodes in an open world. The paper reports that MINDSTORES outperforms its DEPS reimplementation across the MineDojo task table, with a mean gap of 9.4 percentage points; tasks where DEPS scores zero while MINDSTORES reaches double digits include Iron Helmet, Iron Chestplate, Iron Pickaxe, Minecart, and Hopper. It also reports that MINDSTORES reaches mastery in fewer learning iterations than Voyager or Reflexion on complex tasks, and that continuously building the database across tasks cuts the steps needed for later tasks. The paper does not claim to solve the hardest tier: both MINDSTORES and DEPS score 0 percent on MT8 diamond crafting.

Load-bearing premise

The paper's headline comparison uses the authors' own DEPS reimplementation, which they state performs worse than the original DEPS controller because of software version constraints, so the measured 9.4 percent advantage assumes that weakened reimplementation is a fair stand-in for existing methods.

Editorial extensions

If this is right

  • Any LLM planner that currently starts each episode from scratch could be augmented with a natural-language experience store without changing the underlying controller.
  • Intermediate-complexity tasks are where the memory pays off most, with an average gain of 11.0 percentage points over the DEPS baseline across MT3-MT5.
  • Continuous experience building lets later tasks reuse earlier lessons: in the reported run, the Minecart task needed about 200 new steps on top of the prior nine tasks instead of its fresh-environment 6,000 steps.
  • The hardest tier remains unsolved: MT8 diamond crafting is 0 percent for both MINDSTORES and DEPS, so the memory loop alone does not overcome very long-horizon tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's ablations do not separate the outcome-prediction step from the retrieval memory, so a natural next experiment is to disable prediction alone and measure how much of the 9.4 percent comes from rejecting bad plans before execution.
  • If the memory mechanism, not the particular controller, is the active ingredient, the same experience-retrieval wrapper should transfer to other open-world embodied benchmarks and to stronger controllers; the paper does not test that transfer.
  • Because the baseline is the authors' own weakened DEPS, the reported advantage over the published DEPS system may be smaller than 9.4 percent; an independent comparison using the original controller would size the true gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents MINDSTORES, an LLM-based planner for the MineDojo/Minecraft environment that augments a DEPS-style zero-shot planning loop with a persistent natural-language experience database. Each experience is stored as a (state, task, plan, outcome) tuple, embedded with Sentence-BERT, and retrieved by weighted cosine similarity. The LLM generates insights from retrieved experiences, produces a plan, predicts the plan outcome by retrieving similar past plans, and iteratively revises the plan until the predicted outcome is success. The authors evaluate on 53 tasks across eight complexity tiers (MT1–MT8), reporting success rates, a comparison with their own DEPS implementation, learning-efficiency comparisons with Voyager and Reflexion, retrieval-size sensitivity, and a continuous database-building experiment. The headline result is a 9.4% mean improvement in success rate over the authors' DEPS baseline.

Significance. If the central claim is established, the paper makes a useful contribution: it offers an interpretable, natural-language memory mechanism that transfers experience across tasks without retraining, directly addressing a known weakness of zero-shot LLM planners. The architecture is simple and the (state, task, plan, outcome) tuple representation is a sensible design choice. The paper also deserves credit for a transparent Algorithm 1, a substantial evaluation suite of 53 tasks, and an explicit acknowledgment in Section 4.1 that its controller is weaker than the original DEPS controller. However, as written, the empirical evidence does not yet support the claimed improvement over existing methods: the main comparison is against a self-implemented and deliberately weakened baseline, the promised component ablations are missing, and success rates are reported without variance or significance tests. The idea is promising, but the validation is currently insufficient to support the paper's central claim.

major comments (4)
  1. [§4.1, §5.1, Table 4] The headline claim of a "9.4% mean improvement over existing methods" is computed against the authors' own DEPS implementation, which Section 4.1 explicitly concedes achieves lower baseline performance than the original DEPS controller. Consequently, the reported margin may be an artifact of controller weakness rather than a benefit of the experience database. The authors should compare MINDSTORES against the original DEPS (or report the original DEPS numbers on the same tasks) and, at minimum, rephrase the contribution so that it claims improvement over their DEPS reimplementation rather than over existing methods.
  2. [§4.4, §5, Table 4] The paper promises "No Experience" and "Single-Shot" ablations in Section 4.4 but never reports them; Table 4 compares MINDSTORES with DEPS, which differs not only in the experience database but also in outcome prediction and possibly in the planner prompts. This confound prevents attribution of the observed gains to memory. I ask for a no-experience ablation that keeps the same planner, outcome-prediction mechanism, and controller, and for a Single-Shot ablation that disables iterative refinement while retaining retrieval.
  3. [§4.1, §4.4, Fig. 5] The retrieval hyperparameters (k=5, λs=0.4, λt=0.4, λp=0.2) were selected via ablation studies, but the paper does not state whether those ablations were performed on held-out tasks or on the same 53 tasks used for the reported success rates. If the latter, the headline numbers are tuned on the test set and the "improvement" is optimistic. In addition, all success rates come from 30 trials with a fixed seed and no confidence intervals or significance tests, so small differences (e.g., 3.3–6.7 points on several MT1 tasks) are within sampling noise.
  4. [§5.2, Table 3] The learning-efficiency comparison against Voyager and Reflexion lacks detail: the paper does not specify how these baselines were configured, whether they used the same behavior-cloning controller, or how the capped values (500) enter the comparison. Figure 4 and Table 3 also disagree on the quantity being reported ("Novel Learning Iterations" versus "Time steps"). This comparison therefore does not currently support the claim of superior learning efficiency.
minor comments (6)
  1. [Appendix B.3, §4.1] Appendix B.3 states that the SBERT model is "all-MiniLM-L6-v2" with output dimension 768, but this model outputs 384-dimensional embeddings; Section 4.1 also claims 768-dim embeddings. Please correct the dimension or the model name.
  2. [§4.1, Appendix B.2] Section 4.1 reports a fixed random seed of 42, but Appendix B.2 sets env seed=40; these should be reconciled.
  3. [Figure 4, Table 3] Table 3 is titled "Time steps required" while Figure 4's y-axis is "Novel Learning Iterations"; the two terms appear to be used interchangeably, which is confusing.
  4. [Figure 6] Figure 6 reports a 100% success rate for all tasks in the continuous database-building experiment, which appears inconsistent with the success rates for the same task names in Table 2; the protocol for Figure 6 should be clarified.
  5. [Algorithm 1] Algorithm 1's while-true loop has no iteration cap; if the outcome predictor never returns success, the agent can loop indefinitely. Please add a termination criterion or report its behavior in practice.
  6. [Abstract, §7] The abstract and conclusion claim "significant improvements" and "significantly better than existing memory-based LLM planners," but no statistical significance tests are reported; please either add significance tests or soften the wording.

Circularity Check

2 steps flagged · score 4.0 of 10

The headline 9.4% gain over existing methods is not independently established: retrieval hyperparameters were tuned on the evaluation tasks themselves, and the 'DEPS' comparison is against the authors' own memory-ablated, deliberately weakened reimplementation.

  1. fitted input called prediction [Section 4.1, Experimental Setup; Figure 5; Table 2]
    "Key parameters were determined through ablation studies: • Optimal k = 5 neighbors (tested k = 1, 3, 5, 10, 20) • Weighted similarity: λs = 0.4 (state), λt = 0.4 (task), λp = 0.2 (plan)"

    The 'optimal' hyperparameters are the ones that maximize success rate on the very benchmark used to report the headline results. Figure 5 plots 'Success rates vs. retrieval context size k' for Torch, Iron Boots, Iron Pickaxe, Minecart, and Diamond; the same tasks appear in Table 2 as evidence of MINDSTORES performance. After selecting k=5 for best success on those tasks, the success rates reported are not independent predictions but the fitted outcome of the selection itself, so the improvement over DEPS is partially forced by the choice of k.

  2. other [Section 4.1; Section 5.1, Figure 3 caption]
    "Due to software version constraints, our implementation of the controller achieves lower baseline performance than the original DEPS controller. Therefore, we use our implementation of DEPS without the experience database as the primary baseline for fair comparison."

    The paper's contribution claims a 9.4% mean improvement 'over existing methods' (Section 1). In the experiments, however, the DEPS baseline is not the existing DEPS system of Wang et al. (2024c) listed in Section 4.3; it is the authors' own implementation 'without the experience database,' which they admit is weaker than the original controller. Section 5.1 confirms the comparison is against 'our version of DEPS.' Consequently, the headline margin reduces to a self-ablated, deliberately weakened variant rather than an external method, so the claim about existing methods is not established by the reported numbers.

full rationale

This is an empirical systems paper rather than a mathematical derivation, so the circularity is confined to the evidence chain around the headline number. Two reductions are visible. First, Section 4.1 states the retrieval parameters (k=5, lambda weights) were set by ablation studies, and Figure 5 shows success-rate-vs-k curves for tasks (Torch, Iron Boots, Iron Pickaxe, Minecart, Diamond) whose success rates are later reported in Table 2 as MINDSTORES results. The reported success rates are therefore outcomes of a hyperparameter search on the evaluation metric, not an independent prediction. Second, Section 4.1 defines the primary baseline as 'our implementation of DEPS without the experience database' while conceding this implementation is weaker than the original DEPS controller; Section 5.1 then analyzes 'our version of DEPS,' and Figure 3 presents this internal ablation under the name 'DEPS.' The contribution claim of a 9.4% mean improvement 'over existing methods' thus rests on a comparison to a self-ablated, weakened variant rather than to the external DEPS system cited in Section 4.3. These issues are material to the central empirical claim but do not make the whole method vacuously circular: MINDSTORES could still provide a genuine memory benefit, and the MineDojo evaluations are real executions, so a moderate score of 4 is appropriate.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central results depend on a small set of tuned constants (k and lambda retrieval weights) and on assumptions about the baseline controller, the embedding model, the LLM's planning ability, and the environment reward signal. No new physical entities are introduced; the 'mental model' is a metaphorical name for the experience database with no independent falsifiable handle.

free parameters (5)
  • k (number of retrieved experiences) = 5
    Tuned via ablation over k=1, 3, 5, 10, 20 on the evaluation tasks (Section 4.1), not on a held-out validation set.
  • lambda_s (state similarity weight) = 0.4
    Chosen through ablation studies on the same tasks used for final evaluation (Section 4.1).
  • lambda_t (task similarity weight) = 0.4
    Chosen through ablation studies on the same tasks used for final evaluation (Section 4.1).
  • lambda_p (plan similarity weight) = 0.2
    Chosen through ablation studies on the same tasks used for final evaluation (Section 4.1).
  • Reflexion iteration cap = 500
    Learning-efficiency comparisons cap Reflexion at 500 iterations for later tasks (Figure 4 note and Table 3), which affects the reported learning-efficiency results.
assumptions (5)
  • domain assumption The behavior-cloning controller trained on human demonstrations can execute high-level plans with reliability comparable to the original DEPS controller.
    Section 4.1 states the authors' controller achieves lower baseline performance than the original DEPS controller, yet this reimplementation is the primary baseline for the 9.4% improvement claim.
  • domain assumption SBERT sentence embeddings capture planning-relevant semantic similarity between states, tasks, and plans.
    Equations 1-3 and Section 3.2 assume cosine similarity over embeddings is a useful retrieval signal for plan improvement.
  • domain assumption GPT-4 can generate valid insights, plans, and outcome predictions from retrieved natural-language experiences without additional grounding.
    Section 3.3 relies entirely on the LLM to analyze experiences, generate plans, and predict outcomes.
  • domain assumption The MineDojo reward and done signals correctly indicate task completion in all 53 tasks.
    Success rates in Section 5 are computed from these environment signals; Appendix A uses reward and done as outcome labels.
  • ad hoc to paper Ablation-tuned hyperparameters (k and lambda weights) transfer to the reported evaluation tasks.
    Section 4.1 reports optimal parameters from ablation studies on the same benchmark, not on a separate validation split.
invented entities (1)
  • Artificial mental model (persistent natural-language experience store)
    purpose: Framing device for the experience database that retrieves past (state, task, plan, outcome) tuples to inform planning.
    No independent evidence is provided that this store constitutes a cognitive mental model; it is a retrieval database with no falsifiable signature beyond the reported task success.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MINDSTORES: Memory-Informed Neural Decision Synthesis for Task-Oriented Reinforcement in Embodied Systems." pith.science (2026). https://pith.science/paper/3ZFFFB7T

@misc{pith2026250119318,
  author       = {Pith},
  title        = {Pith review of: MINDSTORES: Memory-Informed Neural Decision Synthesis for Task-Oriented Reinforcement in Embodied Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3ZFFFB7T}},
  note         = {Machine review of arXiv:2501.19318}
}
read the original abstract

While large language models (LLMs) have shown promising capabilities as zero-shot planners for embodied agents, their inability to learn from experience and build persistent mental models limits their robustness in complex open-world environments like Minecraft. We introduce MINDSTORES, an experience-augmented planning framework that enables embodied agents to build and leverage mental models through natural interaction with their environment. Drawing inspiration from how humans construct and refine cognitive mental models, our approach extends existing zero-shot LLM planning by maintaining a database of past experiences that informs future planning iterations. The key innovation is representing accumulated experiences as natural language embeddings of (state, task, plan, outcome) tuples, which can then be efficiently retrieved and reasoned over by an LLM planner to generate insights and guide plan refinement for novel states and tasks. Through extensive experiments in the MineDojo environment, a simulation environment for agents in Minecraft that provides low-level controls for Minecraft, we find that MINDSTORES learns and applies its knowledge significantly better than existing memory-based LLM planners while maintaining the flexibility and generalization benefits of zero-shot approaches, representing an important step toward more capable embodied AI systems that can learn continuously through natural experience.

Figures

Figures reproduced from arXiv: 2501.19318 by the authors.

Figure 1
Figure 1. Overview of the MINDSTORES planning architecture. The left shows the iterative [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Interactive planning process for crafting iron boots in Minecraft. The system initially plans [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison: MIND￾STORES consistently outperforms DEPS across tasks. Both systems show declining success rates with increasing complexity (MT1–MT8), with MT8 resulting in 0% success for both. Mean difference: 9.4%. Wood Cobblestone Coal Furnace Sword Iron 0 200 400 Novel Learning Iterations MINDSTORES Voyager Reflexion [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Success rates vs. retrieval context size k for different tasks. Simple tasks improve steadily with k, while more complex tasks re￾quire larger k values. Advanced tasks remain unachievable regardless of k. 1 2 3 4 5 6 7 8 9 10 0 0.2 0.4 0.6 0.8 1 ·104 Task Reference Num…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 13 canonical work pages

  1. [1]

    Grounding llms for robot task planning using closed-loop state feedback, 2024

    Vineet Bhat, Ali Umut Kaypak, Prashanth Krishnamurthy, Ramesh Karri, and Farshad Khorrami. Grounding llms for robot task planning using closed-loop state feedback, 2024. URL https://arxiv.org/abs/2402.08546

    arXiv 2024

  2. [2]

    Language Models are Few - Shot Learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    work page 1901

  3. [3]

    Modeling Transfer Learning in Human Categorization with the Hierarchical Dirichlet Process

    Kevin R Canini, Mikhail M Shashkov, and Thomas L Griffiths. Modeling Transfer Learning in Human Categorization with the Hierarchical Dirichlet Process . International Conference on Machine Learning, 2010

    work page 2010

  4. [4]

    The nature of explanation

    Kenneth James Williams Craik. The nature of explanation. Cambridge : University Press, 1952. URL http://archive.org/details/natureofexplanat0000crai

    work page 1952

  5. [5]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. PaLM - E : An Embodi...

    arXiv 2023

  6. [6]

    Dulac-Arnold, N

    G. Dulac-Arnold, N. Levine, D.J. Mankowitz, et al. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning, 110: 0 2419--2468, 2021. doi:10.1007/s10994-021-05961-4. URL https://doi.org/10.1007/s10994-021-05961-4

    doi:10.1007/s10994-021-05961-4 2021

  7. [7]

    MineDojo : Building Open - Ended Embodied Agents with Internet - Scale Knowledge

    Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. MineDojo : Building Open - Ended Embodied Agents with Internet - Scale Knowledge . Advances in Neural Information Processing Systems, 35: 0 18343--18362, December 2022. URL https://proceedings.neurips.cc/paper_files/pap...

    work page 2022

  8. [8]

    Artur d’Avila Garcez and Luís C. Lamb. Neurosymbolic AI : the 3rd wave. Artif. Intell. Rev., 56 0 (11): 0 12387--12406, March 2023. ISSN 0269-2821. doi:10.1007/s10462-023-10448-w. URL https://doi.org/10.1007/s10462-023-10448-w

    doi:10.1007/s10462-023-10448-w 2023

Show all 42 references
  1. [9]

    PDDLStream : Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning

    Caelan Reed Garrett, Tomás Lozano-Pérez, and Leslie Pack Kaelbling. PDDLStream : Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning . Proceedings of the International Conference on Automated Planning and Scheduling, 30: 0 440--448, June 2020. ...

    2020 doi

  2. [10]

    World Models

    David Ha and Jürgen Schmidhuber. World Models . arXiv, March 2018. doi:10.5281/zenodo.1207631. URL http://arxiv.org/abs/1803.10122. arXiv:1803.10122 [cs]

    2018 arXiv

  3. [11]

    Mastering diverse domains through world models, 2024

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models, 2024. URL https://arxiv.org/abs/2301.04104

    2024 arXiv

  4. [12]

    Reasoning with Language Model is Planning with World Model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with Language Model is Planning with World Model . In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process...

    2023 doi

  5. [13]

    Towards Reasoning in Large Language Models : A Survey

    Jie Huang and Kevin Chen-Chuan Chang. Towards Reasoning in Large Language Models : A Survey . In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics : ACL 2023 , pp.\ 1049--1065, Toronto, Canada, July 2023. Asso...

    2023 doi

  6. [14]

    Language Models as Zero - Shot Planners : Extracting Actionable Knowledge for Embodied Agents , March 2022 a

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language Models as Zero - Shot Planners : Extracting Actionable Knowledge for Embodied Agents , March 2022 a . URL http://arxiv.org/abs/2201.07207. arXiv:2201.07207 [cs]

    2022 arXiv

  7. [15]

    Inner Monologue : Embodied Reasoning through Planning with Language Models , July 2022 b

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Noah Brown, Tomas Jackson, Linda Luu, Sergey Levine, Karol Hausman, and Brian Ichter. Inner Monologue : Embodied Reasoning t...

    2022 arXiv

  8. [16]

    Making Large Language Models Better Planners with Reasoning - Decision Alignment , August 2024

    Zhijian Huang, Tao Tang, Shaoxiang Chen, Sihao Lin, Zequn Jie, Lin Ma, Guangrun Wang, and Xiaodan Liang. Making Large Language Models Better Planners with Reasoning - Decision Alignment , August 2024. URL http://arxiv.org/abs/2408.13890. arXiv:2408.13890 [cs]

    2024 arXiv

  9. [17]

    Playing nethack with llms: Potential & limitations as zero-shot agents, 2024

    Dominik Jeurissen, Diego Perez-Liebana, Jeremy Gow, Duygu Cakmak, and James Kwan. Playing nethack with llms: Potential & limitations as zero-shot agents, 2024. URL https://arxiv.org/abs/2403.00690

    2024 arXiv

  10. [18]

    Hierarchical task and motion planning in the now

    Leslie Pack Kaelbling and Tomas Lozano-Perez. Hierarchical task and motion planning in the now. In 2011 IEEE International Conference on Robotics and Automation , pp.\ 1470--1477, Shanghai, China, May 2011. IEEE. ISBN 978-1-61284-386-5. doi:10.1109/ICRA.2011.5980391. URL http:...

    2011

  11. [19]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, ...

    2015 doi

  12. [20]

    Reasoning with Large Language Models , a Survey , July 2024

    Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. Reasoning with Large Language Models , a Survey , July 2024. URL http://arxiv.org/abs/2407.11511. arXiv:2407.11511 [cs]

    2024

  13. [21]

    Rajesh P. N. Rao and Dana H. Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2 0 (1): 0 79--87, January 1999. ISSN 1546-1726. doi:10.1038/4580. URL https://www.nature.com/article...

    1999 doi

  14. [22]

    Proximal Policy Optimization Algorithms , August 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms , August 2017. URL http://arxiv.org/abs/1707.06347. arXiv:1707.06347 [cs]

    2017 arXiv

  15. [23]

    LLMs Can Plan Only If We Tell Them , January 2025

    Bilgehan Sel, Ruoxi Jia, and Ming Jin. LLMs Can Plan Only If We Tell Them , January 2025. URL http://arxiv.org/abs/2501.13545. arXiv:2501.13545 [cs]

    2025 arXiv

  16. [24]

    From llms to actions: Latent codes as bridges in hierarchical robot control, 2024

    Yide Shentu, Philipp Wu, Aravind Rajeswaran, and Pieter Abbeel. From llms to actions: Latent codes as bridges in hierarchical robot control, 2024. URL https://arxiv.org/abs/2405.04798

    2024 arXiv

  17. [25]

    Reflexion: language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36: 0 8634--8652, December 2023. URL https://proceedings.neurips.cc/paper_files/...

    2023

  18. [26]

    Progprompt: Generating situated robot task plans using large language models

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automatio...

    2023

  19. [27]

    AdaPlanner : Adaptive Planning from Feedback with Language Models

    Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, and Chao Zhang. AdaPlanner : Adaptive Planning from Feedback with Language Models . Advances in Neural Information Processing Systems, 36: 0 58202--58245, December 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023...

    2023

  20. [28]

    Sutton, Doina Precup, and Satinder Singh

    Richard S. Sutton, Doina Precup, and Satinder Singh. Between MDPs and semi- MDPs : A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112 0 (1): 0 181--211, August 1999. ISSN 0004-3702. doi:10.1016/S0004-3702(99)00052-1. URL https://www.sc...

    1999 doi

  21. [29]

    Voyager: An Open - Ended Embodied Agent with Large Language Models , October 2023

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An Open - Ended Embodied Agent with Large Language Models , October 2023. URL http://arxiv.org/abs/2305.16291. arXiv:2305.16291 [cs]

    2023 arXiv

  22. [30]

    E 2CL : Exploration -based Error Correction Learning for Embodied Agents

    Hanlin Wang, Chak Tou Leong, Jian Wang, and Wenjie Li. E 2CL : Exploration -based Error Correction Learning for Embodied Agents . In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics : EMNLP 2024 , pp.\ 7626--76...

    2024 doi

  23. [31]

    Large language models for robotics: Opportunities, challenges, and perspectives

    Jiaqi Wang, Enze Shi, Huawen Hu, Chong Ma, Yiheng Liu, Xuhui Wang, Yincheng Yao, Xuan Liu, Bao Ge, and Shu Zhang. Large language models for robotics: Opportunities, challenges, and perspectives. Journal of Automation and Intelligence, 2024 b . ISSN 2949-8554. doi:https://doi.o...

    2024 doi

  24. [32]

    Describe, Explain , Plan and Select : Interactive Planning with Large Language Models Enables Open - World Multi - Task Agents , July 2024 c

    Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, Explain , Plan and Select : Interactive Planning with Large Language Models Enables Open - World Multi - Task Agents , July 2024 c . URL http://arxiv.org/abs/2302.01560. arXiv:2302.01560 [cs]

    2024 arXiv

  25. [33]

    ReAct : Synergizing Reasoning and Acting in Language Models , March 2023

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct : Synergizing Reasoning and Acting in Language Models , March 2023. URL http://arxiv.org/abs/2210.03629. arXiv:2210.03629 [cs]

    2023 arXiv

  26. [34]

    Robotic control via embodied chain-of-thought reasoning, 2024

    Michał Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied chain-of-thought reasoning, 2024. URL https://arxiv.org/abs/2407.08693

    2024 arXiv

  27. [35]

    PIGLeT : Language Grounding Through Neuro - Symbolic Interaction in a 3D World

    Rowan Zellers, Ari Holtzman, Matthew Peters, Roozbeh Mottaghi, Aniruddha Kembhavi, Ali Farhadi, and Yejin Choi. PIGLeT : Language Grounding Through Neuro - Symbolic Interaction in a 3D World . In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of th...

    2021

  28. [36]

    ExpeL : LLM Agents Are Experiential Learners

    Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL : LLM Agents Are Experiential Learners . Proceedings of the AAAI Conference on Artificial Intelligence, 38 0 (17): 0 19632--19642, March 2024. ISSN 2374-3468. doi:10.1609/aaai.v38i17.29936....

    2024 doi

  29. [37]

    Wenqing Zheng, S. P. Sharan, Zhiwen Fan, Kevin Wang, Yihan Xi, and Zhangyang Wang. Symbolic visual reinforcement learning: A scalable framework with object-level abstraction and differentiable expression search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4...

    2025

  30. [38]

    Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models , June 2024

    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models , June 2024. URL http://arxiv.org/abs/2310.04406. arXiv:2310.04406 [cs]

    2024 arXiv

  31. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  32. [40]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  33. [41]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  34. [42]

    s 2vp ^7믿D+ cȊx Αǣ )(L4@ 4 h MMM3 HI_Ÿ &a!1* & > b` z :r&! A Mq1 t: ؊ @@oqK & v Lq `x &a7΂n Sb N-1@ l 6 7 : z \ [ h Jʏ r0ARbc oRub h ϟg yI !Ђ < #VNo WV: N '*ČJJ^ HvRK xM

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    1999

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.