Pith. sign in

REVIEW 4 major objections 5 minor 51 references

GraphThink: Graph-Enhanced LLM Thinking for Long-Horizon Embodied Task Planning

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper's central claim is that a task graph used as prompt, reward, and verifier lets a small LLM planner beat leading API-based LLMs on long-horizon embodied tasks.

desk verdict Strong ALFRED test-set result and a genuinely integrated graph-based planning pipeline, but the high-level planner comparisons use the authors' own task graph and an unspecified LLM judge as the oracle, so the claimed margin over API LLMs is not settled. read the letter →

arxiv 2608.07905 v1 pith:PCVGNG3S submitted 2026-08-08 cs.AI cs.RO

Chen Li , Sijie Cheng , Yuelin Zhang , Junxi Li , Maozhi Huang , Yang Liu , Wenbing Huang This is my paper · ORCID
classification cs.AIcs.RO
keywords embodiedtaskplanninggraphsceneGRPOlong-horizonALFREDbenchmarkevent-drivenreplanningLLMplanner
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GraphThink is an attempt to fix a specific failure mode: general-purpose language models used as embodied planners produce subtask sequences that look reasonable but violate physical preconditions, lose track of long instructions, and fail to adapt when the environment differs from the plan. The paper's proposal is to give the LLM a task graph — a directed graph of which subtasks can executably follow which — and to use that same graph in three places at once: as part of the prompt, as the reward signal in GRPO reinforcement learning, and as an external verifier that sends feedback for plan revision. A companion scene graph, assembled online from vision-language model outputs, serves as task-relevant memory and triggers replanning on low-level errors or at subtask checkpoints. The paper reports that the resulting planner reaches 96.22% seen, 96.56% unseen, and 90.04% long-horizon planning success on ALFRED, surpassing API-based LLMs, and that the full agent reaches 67.71% seen and 68.52% unseen success on the ALFRED test set. A sympathetic reader should take the central contribution to be the claim that structured graph constraints, rather than more data or bigger models, are what make long-horizon LLM planning reliable.

What carries the argument

The load-bearing object is the task graph $G=(V,E)$, whose nodes are the subtask vocabulary ($12$ high-level action types over object meta-classes such as pickupable objects, fixed receptacles, portable containers, and task-specific targets) and whose edges encode feasible transitions between subtasks. Edges are produced by an LLM-based transition compatibility analysis: each subtask's serialized policy is abstracted into a precondition set $P_i$ and an effect set $E_i$, and an edge $v_p\to v_q$ is added when $E_p$ satisfies $P_q$ and the transition is semantically valid. The same graph drives a graph-enhanced prompt, the four-term GRPO reward (format, node-level object validity, edge-level transition validity, and two-stage instruction following), and a verification loop that feeds node- and edge-level errors back to the LLM for up to three correction rounds. A second structure, the scene graph, is an online-maintained set of semantic triples over task-relevant objects; it acts as compact environmental memory and triggers replanning on low-level execution errors or at subtask completions.

What would settle it

Take GraphThink's trained planner and the leading retrieval-augmented baseline, and score both against an independent human-annotated executability oracle on the same long-horizon samples, rather than against the task-graph verifier that GraphThink was trained and prompted with; if the success-rate gap collapses to near zero, the claimed planner superiority is an artifact of self-evaluation.

Watch

Extended reading notes

Core claim

The central claim is that a structured task graph can serve simultaneously as a planning prompt, a reinforcement-learning reward source, and an inference-time verifier, and that this threefold use removes the physical hallucinations and long-horizon decay that currently limit LLM-based embodied planners. On ALFRED, the full hierarchical agent achieves 67.71% seen and 68.52% unseen success rates, and the high-level planner alone reaches 96.22% seen, 96.56% unseen, and 90.04% on a newly constructed 1,396-sample long-horizon dataset, outperforming leading API-based LLMs under zero-shot, chain-of-thought, few-shot, and retrieval-augmented settings. The paper also claims the approach generalizes out of distribution: it composes nine new action primitives in AI2-Thor with 80.50% success and transfers to VirtualHome tasks, both with the same 7B backbone, without task-specific expert long-horizon trajectories.

Load-bearing premise

The load-bearing premise is that the task graph — built by an LLM's own compatibility analysis — is a correct, complete, and neutral standard for which subtask transitions are executable and which plans satisfy the instruction, and that judging plans against that same graph is a fair way to compare planners.

Editorial extensions

If this is right

  • If the central claim holds, a small open 7B model trained with graph-based rewards can match or beat much larger API models on long-horizon task planning, so model scale is not the only route to reliable planning.
  • The task graph can be extended with new subtask nodes and edges through the same compatibility analysis, so adding a new skill does not require collecting new expert trajectories.
  • Replanning triggered at subtask completion catches plans that are executable but semantically misaligned with the instruction, a failure mode that low-level execution feedback alone misses.
  • Because the planner needs only high-level goal instructions, its training signal can come from unlabeled graph-valid data plus the instruction reward, reducing dependence on expensive step-by-step annotations.
  • Cross-environment transfer to VirtualHome suggests high-level planning priors learned in one embodied simulator can be reused in another through a semantic action translation layer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: Since the task graph's edges are generated by LLM compatibility analysis, the framework's ceiling is set by the quality of that analysis; a direct stress test is to build the graph from human demonstrations or a formal planner and measure whether long-horizon success changes.
  • Editorial inference: The verification-plus-LLM-judge protocol used for planner evaluation opens a natural audit: an independent human-annotated executability oracle on a random sample of long-horizon plans would show how much of the reported margin over retrieval-augmented baselines is genuine planning ability versus shared graph structure.
  • Editorial inference: The same graph-as-prompt/reward/verifier recipe could transfer to other structured generation tasks with typed preconditions, such as tool-use pipelines or browser automation, by replacing object meta-classes with typed resources.
  • Editorial inference: A practical deployment would benefit from measuring how scene-graph noise from VLM hallucinations propagates into replanning; one testable extension is to corrupt a fraction of relation triples and chart the resulting success-rate drop.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. GraphThink is a hierarchical framework for embodied instruction following. It builds a task graph over subtask transitions using an LLM-assisted compatibility analysis; the graph is injected into the LLM prompt, used to define GRPO rewards (node, edge, format, instruction-following), and used as a post-hoc verifier with iterative refinement. A scene-graph memory triggers replanning on low-level errors or subtask completion. The paper reports state-of-the-art ALFRED test SR (67.71% seen, 68.52% unseen), ablations on validation, a new long-horizon dataset of 1,396 samples on which the high-level planner reaches 90.04% versus 59.2% for GPT-5.2+RAG, and generalization results to AI2-Thor novel tasks and VirtualHome.

Significance. The paper's central contribution is a principled way to use a task graph to define dense, multi-path rewards for RL training of LLM planners, which is a genuine step toward avoiding single-trajectory overfitting in embodied planning. The full-agent ALFRED leaderboard results are strong external evidence, and the ablation study is reasonably complete. However, the headline claim that the high-level planner surpasses API LLMs rests on a self-defined metric whose oracle is the same task graph used to train and prompt the system; until that comparison is validated by execution or an independent oracle, the claim is not established. The long-horizon dataset and judge prompts are also not available, limiting reproducibility.

major comments (4)
  1. [§IV-C.1, §III-A.1, §III-A.4] The high-level planning success metric is not an independent oracle. A plan is successful if it satisfies the task graph G and, when it diverges from ground truth, passes an unspecified LLM judge. The same graph G is built by LLM-assisted transition compatibility analysis (§III-A.1), used in the prompt (§III-A.2), used as the GRPO edge reward R_edge (Eq. 4), and used as the verifier V_edge = R_edge (§III-A.4). Since GraphThink is trained and prompted to produce G-conforming plans while the API baselines are not, the reported margins (e.g., 90.04% vs 59.2% on long horizon) may partly measure alignment with GraphThink's own artifact rather than executability or instruction alignment. Please re-score the high-level planner comparison by executing plans in the simulator with the low-level policy, or by using an independent judge not derived from the task graph, and report the resulting numbers.
  2. [§IV-C.1, App. B and G] The long-horizon dataset of 1,396 samples and the LLM judge prompt are central to the generalization claim but are only referenced as appendix material that is not included in the submitted text. Without the dataset construction details, the 17 new task types, and the judge prompt, the long-horizon comparison cannot be reproduced or checked for distribution shift and judge bias. Please include the full construction, the judge prompt, and a release plan, or at least provide a random sample of instances with judge decisions.
  3. [§IV-C.1, Fig. 5] All high-level planning results are reported as single point estimates with no error bars, confidence intervals, or significance tests. Because the metric includes an LLM judge whose outputs are stochastic, the difference between GraphThink (96.22% valid seen) and GPT-5.2+RAG (82.9%) is not shown to be stable. Please report variance across multiple judge calls or seeds, and quantify judge agreement on a subset.
  4. [§III-A.1, App. A] The task graph's correctness is load-bearing for both training and evaluation, but the reliability verification is deferred to Appendix A, which is not included. In particular, if G is incomplete, a valid baseline plan using an unmodeled transition will be marked as failed; if G is too permissive, the LLM judge may accept inexecutable plans. Please include the reliability analysis and, if possible, measure edge precision/recall against a corpus of human- or simulator-validated transitions.
minor comments (5)
  1. [§III-A.3, Eq. (1)] The definition of R_fmt contains a duplicated line for R_fmt = 0.5; the equation should be cleaned up.
  2. [Table III] The subtask 'PutPickObject' appears to be a typo for 'PutObject'.
  3. [Fig. 5 and Fig. 9] The captions use 'Accuracy' interchangeably with 'planning success rate'; please align the terminology.
  4. [Table VII] The column header 'W AH-NL' should be 'WAH-NL' (or the intended benchmark name such as LOTA-Bench) to avoid a typo-like artifact.
  5. [§IV-D] The labels 'Ours(ALF)' and 'Ours(VH)' are used in Table VII but defined only after the table; please define them before first use.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline comparative claim (planner surpasses API LLMs) is scored by a gate that is literally the paper's own training reward (V_edge = R_edge, Eq. 4); ALFRED leaderboard results (Table I) are external and non-circular.

  1. fitted input called prediction [Section III-A.4 (V_edge = R_edge), Eq. (4); gate applied in Section IV-C.1; results in Fig. 5 and Tables IV-VI]
    "To verify that all action transitions satisfy the edge constraints of the task graph, the edge-level legitimacy check is defined analogously to the edge-level reward R_edge (see Eq. (4)): V_edge = R_edge."

    The Section IV-C.1 protocol makes 'graph-constrained verification' a necessary condition for high-level planning success ('If graph verification fails, the plan is marked as failed'). By definition V_edge = R_edge, and R_edge (Eq. 4) is the product over adjacent pairs of the indicator that (s_t, s_{t+1}) ∈ E — exactly what GRPO maximizes (III-A.3), with the same graph G injected into the prompt (III-A.2). GraphThink is trained, prompted and scored with the same artifact, while baselines are scored against a graph they were never trained on; an executable baseline plan using a feasible transition absent from G fails the gate regardless of validity.

  2. renaming known result [Section IV-E, Fig. 9(b)]
    "Meanwhile, Fig. 9(b) shows that its graph pass rate remains at or above 85% across all horizons."

    'Graph pass rate' is the fraction of generated plans whose adjacent transitions all lie in the task graph's edge set E — the very condition checked by V_edge = R_edge (Eq. 4), which is both the GRPO training reward and the inference-time verification gate (III-A.4). Reporting this quantity as evidence of 'stronger robustness and planning fidelity' than baselines renames reward satisfaction as an independent evaluation metric. Since GPT-5.2+RAG and Qwen2.5-7B-Instruct+SFT were never trained to maximize this edge condition, the comparison is forced by construction rather than measured against an external oracle.

full rationale

The manuscript contains one genuinely external, non-circular claim: Table I reports SR/GC on the official ALFRED test set, scored by the benchmark's own task-completion evaluation. That SOTA-on-ALFRED claim is self-contained and credible. The separate abstract claim that the high-level planner 'surpasses leading API-based LLMs on both the validation set and held-out long-horizon tasks' rests on the custom validation protocol of Section IV-C.1, and here the chain closes on itself. Success requires passing 'graph-constrained verification'; edge verification is defined as V_edge = R_edge (III-A.4); R_edge (Eq. 4) is the GRPO training reward GraphThink is optimized to maximize, with the same task graph injected into its prompt (III-A.2). The graph itself is built by LLM-assisted transition compatibility analysis (III-A.1), so the evaluator is the paper's own artifact. A baseline plan that is executable but uses a feasible transition omitted from G is marked failed, while GraphThink was reward-trained never to emit unmodeled transitions; the margin over GPT-5.2+RAG therefore conflates plan quality with self-alignment. The unspecified LLM judge for graph-valid/GT-divergent plans ('it undergoes further validation using an LLM') and the self-constructed, unreleased 1396-sample long-horizon dataset aggravate the comparison but are secondary to the core identity. Fig. 9(b)'s 'graph pass rate' is the same training-reward satisfaction rate renamed as evidence. Self-citation appears once (ref [31], co-authors Cheng and Liu) in the related-work survey of CoT methods and is not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. Verdict: partial circularity (score 6) — the ALFRED leaderboard claim is independent, but the headline comparative planner claim reduces in part, by construction, to scoring against the paper's own reward.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

No new physical entities are introduced; the task graph and scene graph are internal data structures rather than postulated components of the world. The main epistemic cost is carried by hand-chosen thresholds, reward coefficients, and the domain assumption that LLM-constructed graph constraints are correct and complete.

free parameters (6)
  • Reward component weights = 1.0 each (unweighted sum, Eq. 5)
    All four rewards are summed with equal weight; no ablation or fitting of relative contributions is reported, yet this balance shapes the GRPO training.
  • Format reward thresholds = 1.0 for valid XML/JSON; 0.5 for valid XML only; 0.0 otherwise
    Hand-chosen thresholds in Eq. (1) define the training signal for output structure.
  • Instruction reward thresholds = 1.0 exact GT match; 0.5 critical-subtask coverage; 0.0 otherwise
    The two-stage R_inst in Section III-A.3 is a hand-designed schedule; it anchors training to ground-truth ALFRED plans.
  • Viewpoint capture threshold = 45 degrees
    Scene graph relation extraction only runs when the object is within 45 degrees of the heading; this hand-set filter affects which relations enter the scene graph.
  • Verification iteration limit = 3
    The Verify-Feedback-Correct loop runs at most 3 times; this cap affects residual error correction at inference.
  • Number of subtask nodes = 12
    Borrowed from prior ALFRED work; a design choice determining task graph granularity and the planner's action vocabulary.
assumptions (6)
  • domain assumption A set of 12 high-level subtask nodes and 4 object meta-classes is sufficient to cover all ALFRED and long-horizon tasks.
    Section III-A.1: evaluation adopts 12 subtask nodes and meta-classes C = {obj, rec, mov, spec}; if this abstraction omits relevant operations, the task graph and planner cannot represent valid plans.
  • ad hoc to paper The LLM-assisted transition compatibility analysis produces correct preconditions and effects for subtasks, and therefore correct task graph edges.
    Section III-A.1: 'for each subtask v_i, the LLM receives its serialized low-level policy and abstracts it into a structured transition interface'; no independent verification of the LLM's physical reasoning is provided.
  • domain assumption A plan that satisfies task graph node and edge constraints is a good proxy for an executable, instruction-aligned plan.
    Used in Eq. (2)-(4) for rewards and in Section III-A.4 for verification; if the graph is incomplete or too permissive, high graph pass rates are not evidence of good planning.
  • domain assumption The ground-truth ALFRED plan S* is a reliable reference, and the extracted critical subtasks capture user intent.
    Section III-A.3: R_inst compares generated plans to S* and critical subtasks; incorrect annotations would distort training and evaluation.
  • domain assumption Scene graph triples extracted by a VLM and filtered by rules are accurate enough to drive replanning.
    Section III-C.1: candidate triples from VLM are filtered by task relevance and physical-plausibility constraints; persistent perception errors would propagate into replanning decisions.
  • domain assumption The task graph remains valid across environments and novel tasks when extended with the same compatibility analysis.
    Section IV-D: new actions are added via LLM compatibility analysis without trajectory demonstrations; this assumes the analysis transfers to new skills.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GraphThink: Graph-Enhanced LLM Thinking for Long-Horizon Embodied Task Planning." pith.science (2026). https://pith.science/paper/PCVGNG3S

@misc{pith2026260807905,
  author       = {Pith},
  title        = {Pith review of: GraphThink: Graph-Enhanced LLM Thinking for Long-Horizon Embodied Task Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PCVGNG3S}},
  note         = {Machine review of arXiv:2608.07905}
}
read the original abstract

Embodied agents using LLM-based planners often struggle with physical hallucinations, poor generalization to long-horizon tasks, and lack of environmental awareness. We propose GraphThink, a novel framework that integrates a task graph to provide structured knowledge for robust planning and a scene graph to maintain environmental memory for event-driven replanning. Specifically, the task graph guides LLM thinking through contextual prompting and iterative refinement, effectively mitigating planning hallucinations. Furthermore, within the GRPO framework, the task graph offers delicate reward design to train the LLM planner, enhancing long-horizon planning capabilities and improving generalization. Finally, an event-driven replanning module, powered by the scene graph, enables closed-loop environment awareness and error correction. GraphThink achieves state-of-the-art performance on the ALFRED benchmark. In particular, our high-level planner surpasses leading API-based LLMs on both the validation set and held-out long-horizon tasks, underscoring its robust zero-shot and few-shot capabilities. Additional evaluations further demonstrate strong out-of-distribution generalization to novel tasks and environments.

Figures

Figures reproduced from arXiv: 2608.07905 by the authors.

Figure 1
Figure 1. GraphThink consists of three core modules: (a) the high-level planner with task graph generates an initial plan, (b) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Qualitative example illustrating the benefits of the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The two examples illustrate two types of event-driven [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: An example of error-triggered replanning. When the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Success rates for high-level planning on ALFRED validation set and long-horizon tasks. For brevity, we adopt the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Ablation studies on training data volume. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: An example of GraphThink executing a newly composed task involving unseen action primitives in AI2-Thor. The new tasks ‘DirtyObject’ and ‘BreakObject’ are highlighted in blue. Find(Apple) Find(Sink) Put(Apple, Sink) Switchon(Faucet) Switchoff(Faucet) Find(Microwave) Gr…
Figure 8
Figure 8. Figure 8: An example of GraphThink for the task ‘Heat a clean apple and then put it on the dining table’ in VirtualHome. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Performance comparison across increasing task horizons. Subfigure (a) reports planning success rate by accuracy, and [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 27 canonical work pages

  1. [1]

    Vlabench: A large-scale benchmark for language- conditioned robotics manipulation with long-horizon reasoning tasks,

    S. Zhang, Z. Xu, P. Liu, X. Yu, Y . Li, Q. Gao, Z. Fei, Z. Yin, Z. Wu, Y .-G. Jianget al., “Vlabench: A large-scale benchmark for language- conditioned robotics manipulation with long-horizon reasoning tasks,” arXiv preprint arXiv:2412.18194, 2024

    arXiv 2024

  2. [2]

    Online continual learning for interactive instruction following agents,

    B. Kim, M. Seo, and J. Choi, “Online continual learning for interactive instruction following agents,”arXiv preprint arXiv:2403.07548, 2024

    arXiv 2024

  3. [3]

    Realfred: An embodied instruction following benchmark in photo-realistic environ- ments,

    T. Kim, C. Min, B. Kim, J. Kim, W. Jeung, and J. Choi, “Realfred: An embodied instruction following benchmark in photo-realistic environ- ments,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 346–364

    work page 2024

  4. [4]

    Cookbench: A long-horizon embodied planning benchmark for complex cooking scenarios,

    M. Cai, X. Chen, Y . An, J. Zhang, X. Wang, W. Xu, W. Zhang, and T. Liu, “Cookbench: A long-horizon embodied planning benchmark for complex cooking scenarios,”arXiv preprint arXiv:2508.03232, 2025

    arXiv 2025

  5. [5]

    Episodic transformer for vision- and-language navigation,

    A. Pashevich, C. Schmid, and C. Sun, “Episodic transformer for vision- and-language navigation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15 942–15 952

    work page 2021

  6. [6]

    Look wide and interpret twice: Improving performance on interactive instruction-following tasks,

    M. Suganuma, T. Okataniet al., “Look wide and interpret twice: Improving performance on interactive instruction-following tasks,” in 30th International Joint Conference on Artificial Intelligence, IJCAI

    work page

  7. [7]

    Spoc: Imitating shortest paths in simulation enables effective navigation and manipulation in the real world,

    K. Ehsani, T. Gupta, R. Hendrix, J. Salvador, L. Weihs, K.-H. Zeng, K. P. Singh, Y . Kim, W. Han, A. Herrastiet al., “Spoc: Imitating shortest paths in simulation enables effective navigation and manipulation in the real world,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 238–16 250

    work page 2024

  8. [8]

    Film: Following instructions in language with modular methods,

    S. Y . Min, D. S. Chaplot, P. Ravikumar, Y . Bisk, and R. Salakhutdinov, “Film: Following instructions in language with modular methods,”arXiv preprint arXiv:2110.07342, 2021

    arXiv 2021

Show all 51 references
  1. [9]

    Multi-level compositional reason- ing for interactive instruction following,

    S. Bhambri, B. Kim, and J. Choi, “Multi-level compositional reason- ing for interactive instruction following,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 1, 2023, pp. 223–231

    2023

  2. [10]

    Disco: Embodied navigation and interaction via differentiable scene semantics and dual-level control,

    Y . Yang, “Disco: Embodied navigation and interaction via differentiable scene semantics and dual-level control,” inEuropean Conference on Computer Vision, ECCV 2024 (29/09/2024-04/10/2024, Milan), 2024

    2024

  3. [11]

    Multi-modal grounded planning and efficient replanning for learning embodied agents with a few examples,

    T. Kim, B. Kim, and J. Choi, “Multi-modal grounded planning and efficient replanning for learning embodied agents with a few examples,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, number 4, 2025, pp. 4329–4337

    2025

  4. [12]

    Do as i can, not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausmanet al., “Do as i can, not as i say: Grounding language in robotic affordances,”arXiv preprint arXiv:2204.01691, 2022

    2022 arXiv

  5. [13]

    Progprompt: Generating situated robot task plans using large language models,

    I. Singh, V . Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situated robot task plans using large language models,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 11 523–11 530

    2023

  6. [14]

    Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning,

    K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suender- hauf, “Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning,”arXiv preprint arXiv:2307.06135, 2023

    2023 arXiv

  7. [15]

    Retrieval- augmented hierarchical in-context reinforcement learning and hindsight modular reflections for task planning with llms,

    C. Sun, S. Huang, H. Liu, J. Gong, and D. Pompili, “Retrieval- augmented hierarchical in-context reinforcement learning and hindsight modular reflections for task planning with llms,” in2025 IEEE Interna- tional Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 1217–1224

    2025

  8. [16]

    Llm-planner: Few-shot grounded planning for embodied agents with large language models,

    C. H. Song, J. Wu, C. Washington, B. M. Sadler, W.-L. Chao, and Y . Su, “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 2998–3009

    2023

  9. [17]

    Robogpt: an llm-based long-term decision-making embodied agent for instruction following tasks,

    Y . Chen, W. Cui, Y . Chen, M. Tan, X. Zhang, J. Liu, H. Li, D. Zhao, and H. Wang, “Robogpt: an llm-based long-term decision-making embodied agent for instruction following tasks,”IEEE Transactions on Cognitive and Developmental Systems, 2025

    2025

  10. [18]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wuet al., “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,”arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  11. [19]

    Concept- graphs: Open-vocabulary 3d scene graphs for perception and planning,

    Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappaet al., “Concept- graphs: Open-vocabulary 3d scene graphs for perception and planning,” in2024 IEEE International Conference on Robotics and Automation (ICRA). ...

    2024

  12. [20]

    Search3d: Hierarchical open-vocabulary 3d segmentation,

    A. Takmaz, A. Delitzas, R. W. Sumner, F. Engelmann, J. Wald, and F. Tombari, “Search3d: Hierarchical open-vocabulary 3d segmentation,” IEEE Robotics and Automation Letters, 2025

    2025

  13. [21]

    Alfred: A benchmark for interpret- ing grounded instructions for everyday tasks,

    M. Shridhar, J. Thomason, D. Gordon, Y . Bisk, W. Han, R. Mottaghi, L. Zettlemoyer, and D. Fox, “Alfred: A benchmark for interpret- ing grounded instructions for everyday tasks,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10 ...

    2020

  14. [22]

    Ai2-thor: An interactive 3d environment for visual ai,

    E. Kolve, R. Mottaghi, W. Han, E. VanderBilt, L. Weihs, A. Herrasti, M. Deitke, K. Ehsani, D. Gordon, Y . Zhuet al., “Ai2-thor: An interactive 3d environment for visual ai,”arXiv preprint arXiv:1712.05474, 2017

    2017 arXiv

  15. [23]

    Virtualhome: Simulating household activities via programs,

    X. Puig, K. Ra, M. Boben, J. Li, T. Wang, S. Fidler, and A. Tor- ralba, “Virtualhome: Simulating household activities via programs,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8494–8502

    2018

  16. [24]

    Factorizing perception and policy for interactive instruction following,

    K. P. Singh, S. Bhambri, B. Kim, R. Mottaghi, and J. Choi, “Factorizing perception and policy for interactive instruction following,” inProceed- ings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1888–1897

    2021

  17. [25]

    Prompter: Utilizing large language model prompting for a data efficient embodied instruction following,

    Y . Inoue and H. Ohashi, “Prompter: Utilizing large language model prompting for a data efficient embodied instruction following,”arXiv preprint arXiv:2211.03267, 2022

    2022 arXiv

  18. [26]

    Opex: A component- wise analysis of llm-centric agents in embodied instruction following,

    H. Shi, Z. Sun, X. Yuan, M.-A. C ˆot´e, and B. Liu, “Opex: A component- wise analysis of llm-centric agents in embodied instruction following,” arXiv preprint arXiv:2403.03017, 2024

    2024 arXiv

  19. [27]

    Epo: Hierarchical llm agents with environment preference optimization,

    Q. Zhao, H. Fu, C. Sun, and G. Konidaris, “Epo: Hierarchical llm agents with environment preference optimization,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 6401–6415

    2024

  20. [28]

    Inner monologue: Embod- ied reasoning through planning with language models,

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotaret al., “Inner monologue: Embod- ied reasoning through planning with language models,”arXiv preprint arXiv:2207.05608, 2022

    2022 arXiv

  21. [29]

    Pre-emptive action revision by environmental feedback for embodied instruction following agents,

    J. Kim, C. Min, B. Kim, and J. Choi, “Pre-emptive action revision by environmental feedback for embodied instruction following agents,” in 8th Annual Conference on Robot Learning, 2024

    2024

  22. [30]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”Advances in neural information processing systems, vol. 35, pp. 24 824–24 837, 2022

    2022

  23. [31]

    Unsupervised explanation generation via correct instantiations,

    S. Cheng, Z. Wu, J. Chen, Z. Li, Y . Liu, and L. Kong, “Unsupervised explanation generation via correct instantiations,” inProceedings of the AAAI conference on artificial intelligence, vol. 37, number 11, 2023, pp. 12 700–12 708

    2023

  24. [32]

    Safeplan: Leveraging formal logic and chain-of-thought reasoning for enhanced safety in llm-based robotic task planning,

    I. Obi, V . L. Venkatesh, W. Wang, R. Wang, D. Suh, T. I. Amosa, W. Jo, and B.-C. Min, “Safeplan: Leveraging formal logic and chain-of-thought reasoning for enhanced safety in llm-based robotic task planning,”arXiv preprint arXiv:2503.06892, 2025

    2025 arXiv

  25. [33]

    Self-refine: Iter- ative refinement with self-feedback,

    A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y . Yanget al., “Self-refine: Iter- ative refinement with self-feedback,”Advances in Neural Information Processing Systems, vol. 36, pp. 46 534–46 594, 2023

    2023

  26. [34]

    Amor: A recipe for building adaptable modular knowledge agents through process feedback,

    J. Guan, W. Wu, P. Xu, H. Wang, M. Huanget al., “Amor: A recipe for building adaptable modular knowledge agents through process feedback,”Advances in Neural Information Processing Systems, vol. 37, pp. 126 118–126 148, 2024

    2024

  27. [35]

    P-rag: Progressive retrieval augmented generation for planning on embodied everyday task,

    W. Xu, M. Wang, W. Zhou, and H. Li, “P-rag: Progressive retrieval augmented generation for planning on embodied everyday task,” in Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 6969–6978

    2024

  28. [36]

    Instructrag: Leveraging retrieval-augmented generation on instruction graphs for llm-based task planning,

    Z. Wang, S. X. Teo, J. J. Chew, and W. Shi, “Instructrag: Leveraging retrieval-augmented generation on instruction graphs for llm-based task planning,” inProceedings of the 48th International ACM SIGIR Con- ference on Research and Development in Information Retrieval, 2025, pp...

    2025

  29. [37]

    Double-feedback: Enhancing large language models reasoning in robotic tasks by knowl- edge graphs,

    H. Wang, S. Zhang, S. Wang, T. Jiang, and Y . Ge, “Double-feedback: Enhancing large language models reasoning in robotic tasks by knowl- edge graphs,”IEEE Robotics and Automation Letters, 2025

    2025

  30. [38]

    Self-reflective planning with knowledge graphs: Enhancing llm reasoning reliability for question answering,

    J. Zhu, Y . Liu, M. Bao, K. Zhang, Y . Zhang, and Q. Liu, “Self-reflective planning with knowledge graphs: Enhancing llm reasoning reliability for question answering,”arXiv preprint arXiv:2505.19410, 2025

    2025 arXiv

  31. [39]

    You only look at screens: Multimodal chain- of-action agents,

    Z. Zhang and A. Zhang, “You only look at screens: Multimodal chain- of-action agents,”arXiv preprint arXiv:2309.11436, 2023

    2023 arXiv

  32. [40]

    Toolrl: Reward is all tool learning needs,

    C. Qian, E. C. Acikgoz, Q. He, H. Wang, X. Chen, D. Hakkani-T ¨ur, G. Tur, and H. Ji, “Toolrl: Reward is all tool learning needs,”arXiv preprint arXiv:2504.13958, 2025. 13

    2025 arXiv

  33. [41]

    What is the alignment objective of grpo?

    M. V ojnovic and S.-Y . Yun, “What is the alignment objective of grpo?” arXiv preprint arXiv:2502.18548, 2025

    2025 arXiv

  34. [42]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Biet al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,”arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  35. [43]

    Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning,

    T. Xie, Z. Gao, Q. Ren, H. Luo, Y . Hong, B. Dai, J. Zhou, K. Qiu, Z. Wu, and C. Luo, “Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning,”arXiv preprint arXiv:2502.14768, 2025

    2025 arXiv

  36. [44]

    Critic: Large language models can self-correct with tool-interactive critiquing,

    Z. Gou, Z. Shao, Y . Gong, Y . Shen, Y . Yang, N. Duan, and W. Chen, “Critic: Large language models can self-correct with tool-interactive critiquing,”arXiv preprint arXiv:2305.11738, 2023

    2023 arXiv

  37. [45]

    Context-aware planning and environment-aware memory for instruction following embodied agents,

    B. Kim, J. Kim, Y . Kim, C. Min, and J. Choi, “Context-aware planning and environment-aware memory for instruction following embodied agents,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 10 936–10 946

    2023

  38. [46]

    One demo is all it takes: Planning domain derivation with llms from a single demonstration,

    J. Huang, Y . Xiao, Z. Zhang, M. Coates, J. Hao, and Y . Zhang, “One demo is all it takes: Planning domain derivation with llms from a single demonstration,”arXiv preprint arXiv:2505.18382, 2025

    2025

  39. [47]

    A persistent spatial semantic representation for high-level natural language instruction exe- cution,

    V . Blukis, C. Paxton, D. Fox, A. Garg, and Y . Artzi, “A persistent spatial semantic representation for high-level natural language instruction exe- cution,” inConference on Robot Learning. PMLR, 2022, pp. 706–717

    2022

  40. [48]

    Lota-bench: Benchmarking language-oriented task planners for embodied agents,

    J.-W. Choi, Y . Yoon, H. Ong, J. Kim, and M. Jang, “Lota-bench: Benchmarking language-oriented task planners for embodied agents,” inInternational Conference on Learning Representations (ICLR) 2024, 2024, pp. 1–27

    2024

  41. [49]

    Lifelong experience abstraction and planning,

    P. Liu, L. P. Kaelbling, J. B. Tenenbaum, and J. Mao, “Lifelong experience abstraction and planning,” inICML 2025 Workshop on Programmatic Representations for Agent Learning, 2025

    2025

  42. [50]

    Embodied agent interface: Benchmarking llms for embodied decision making,

    M. Li, S. Zhao, Q. Wang, K. Wang, Y . Zhou, S. Srivastava, C. Gok- men, T. Lee, E. L. Li, R. Zhanget al., “Embodied agent interface: Benchmarking llms for embodied decision making,”Advances in Neural Information Processing Systems, vol. 37, pp. 100 428–100 534, 2024

    2024

  43. [2021]

    International Joint Conferences on Artificial Intelligence, 2021, pp. 923–930

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.