Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Language-Grounded Hierarchical Planning and Execution with Multi-Robot 3D Scene Graphs

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A team of robots can translate plain-English commands into formal plans and execute them outdoors using a single shared 3D scene graph.

desk verdict Genuine multi-robot outdoor integration, but the 'reliably ground' claim is not backed by the paper's own grounding ablations. read the letter →

arxiv 2506.07454 v2 pith:UQEQJY64 submitted 2025-06-09 cs.RO cs.AI

Jared Strader , Aaron Ray , Jacob Arkin , Mason B. Peterson , Yun Chang , Nathan Hughes , Christopher Bradley , Yi Xuan Jia
show 6 more authors
Carlos Nieto-Granda Rajat Talak Chuchu Fan Luca Carlone Jonathan P. How Nicholas Roy
This is my paper
classification cs.ROcs.AI
keywords multi-robotsystems3Dscenegraphstaskandmotionplanningnaturallanguagegroundinglargemodelsopen-setobjectmapview-invariantrelocalizationPDDL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a team of robots can build one shared 3D scene graph, relocalize within it from very different viewpoints, and use a large language model to turn plain-English operator commands into formal plans that the robots then carry out in large outdoor spaces. The key move is a single representation that serves both perception and planning: an open-set object-based map tied to a hierarchical scene graph with objects, places, and regions. If the claim holds, a human could task heterogeneous robots with inspection and navigation jobs by typing or speaking a sentence, with no programming. The authors test the claim on two real robots at Camp Buckner and report success rates for grounding, planning, and execution, along with ablations that show where the weakest link sits.

What carries the argument

The central object is the fused 3D scene graph, a hierarchical structure whose nodes are objects (each with a CLIP embedding and a shape descriptor), places (navigable 2D regions computed by a Voronoi-like partition of the mesh), and regions (semantically consistent areas such as road or sidewalk). The object layer is produced by ROMAN, an open-set object map that enables loop-closure detection and view-invariant relocalization; the places layer supports motion planning; and the full graph, serialized to text, defines the PDDL domain. The planning pipeline is PDDLStream, and the language-to-goal step is an LLM prompted in-context with the scene-graph text, robot capabilities, and examples.

What would settle it

Give the system the instruction 'Spot, check out the two boxes near the pole' in a fresh outdoor scene and observe the generated PDDL goal; if the goal is a movement to the boxes rather than an inspection, the grounding step has reproduced the documented failure mode, falsifying the claim of reliable language grounding for that class of commands.

Watch

Extended reading notes

Core claim

The central discovery is that a fused 3D scene graph whose object layer is an open-set object map can simultaneously support view-invariant relocalization and language-grounded task and motion planning. The system builds per-robot scene graphs and object maps, fuses them through object-based loop closures into one shared scene graph, and then treats that graph as a PDDL planning domain. A large language model, prompted with a text serialization of the scene graph, robot capability descriptions, and in-context examples, translates natural-language instructions into per-robot PDDL goals. PDDLStream searches for a feasible task skeleton with continuous motion parameters, and each robot relocalizes against the fused object map before executing its part of the plan. In experiments, two heterogeneous robots executed easy, medium, and hard instructions outdoors, with the authors reporting a failure mode where an instruction to 'check out two nearby boxes' was grounded as movement rather than inspection.

Load-bearing premise

The whole pipeline rests on the assumption that a large language model will translate an operator's natural-language instruction into the correct PDDL goal from a text serialization of the scene graph, an assumption the paper's own grounding results show failing for spatial-relation and region-level commands.

Editorial extensions

If this is right

  • A single 3D scene graph can serve as both the metric-semantic map for localization and the symbolic domain for planning, so the system avoids maintaining separate representations for perception and task planning.
  • Object-based loop closures let multiple robots fuse maps with a far smaller database than visual place recognition, while improving object IoU, precision, and recall in the fused scene graph on the tested datasets.
  • The language-to-PDDL translation is accurate for direct references and unambiguous commands, but spatial-relation and region-level disambiguation remain unreliable, which bounds the kinds of commands the system can safely execute.
  • The system achieves end-to-end execution on a wheeled robot and a quadruped with different sensors and odometry, so the approach transfers across heterogeneous platforms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the LLM grounding step is the bottleneck, replacing it with a hybrid resolver that calls a spatial-relation program or a scene-graph query engine for the ambiguous parts could raise end-to-end reliability without changing the rest of the pipeline.
  • Since relocalization degrades where objects are sparse, we expect the approach to struggle in open fields and long corridor-like stretches; merging object cues with conventional geometric place recognition would be a natural test.
  • The open-set object map is used only to localize and to supply object names; we infer that the same CLIP embeddings could let operators refer to objects never seen during mapping, enabling genuinely open-ended commands.
  • A stale scene graph would silently break tasks that reference changed or moved objects; online scene-graph updating is an implied requirement before the system can operate in non-static environments.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents an integrated multi-robot system that combines 3D scene graph mapping (Hydra/Hydra-Multi), open-set object-based relocalization (ROMAN), LLM-based translation of natural-language instructions into PDDL goals, and hierarchical task and motion planning. The system is evaluated in large-scale outdoor experiments at Camp Buckner with a Clearpath Husky and a Boston Dynamics Spot, covering scene graph fusion, view-invariant relocalization, and end-to-end execution of instructions at three difficulty levels. Ablations on the Kimera-Multi dataset characterize fusion accuracy, and offline language-grounding experiments test seven linguistic categories across several LLMs. The central claim is that the system can 'reliably ground and execute complex instructions' expressed in natural language.

Significance. If the central claims hold, this is a noteworthy systems integration demonstration: it brings together state-of-the-art components (Hydra-Multi, ROMAN, scene-graph-based TAMP, LLM grounding) and evaluates them on real robots in a large-scale outdoor environment. The paper's honest reporting of failures (e.g., the grounding error for 'checkout two nearby boxes', the 64-65% relocalization success) is a strength, as is the inclusion of offline ablations that probe component behavior beyond the small end-to-end trial count. However, the evidence is thin for the stated reliability claim: the end-to-end trials are few (8 single-robot, 12 multi-robot total, with no per-condition trial counts), the offline grounding ablations show weak performance on exactly the categories appearing in the Hard commands, and the relocalization success rate is only about two-thirds. The integration value is clear, but the quantitative support for the central claims needs substantial strengthening.

major comments (4)
  1. [Section 5.1, Table 1 vs. Section 5.2, Table 4] The paper does not reconcile the near-perfect grounding success in the end-to-end experiments with the low offline grounding scores on the same linguistic phenomena. Table 4 reports that the best model, GPT-4.1, achieves only 4/10 on spatial-relation disambiguation and 5/10 on region-level disambiguation, and GPT-4o scores 2/10 and 4/10 on these categories. Yet Table 1 reports 100% grounding success for Multi-Robot trials at all difficulty levels, including Hard commands such as 'Spot, check out the two boxes in the shelter and the closest trash, then head over to the pole', which require spatial-relation and region-level disambiguation. The paper also reports an end-to-end failure in Section 5.1 where 'checkout two nearby boxes' was grounded as a movement predicate rather than an inspection predicate. The authors should explain how the end-to-end Hard commands avoid the failure modes identified in Table 4, specify which LLM and prompt configuration were used in the end-to-end trials, and report the number of Hard trials per condition. Without this reconciliation, the claim that complex instructions are 'reliably ground[ed]' (abstract, Section 5.1) is not supported by the evidence.
  2. [Section 5.1, Table 1] The trial counts are too small to support the reliability claim. Table 1 reports aggregate success rates over 8 single-robot and 12 multi-robot trials, but does not state how many trials were run per difficulty level and per stage. With three difficulty levels and three stages, each percentage likely corresponds to only 1-4 trials, so a single failure changes the rate by 25-50%. No confidence intervals or error bars are provided. The claim that the system 'reliably ground[s] and execute[s] complex instructions' requires either a larger trial count, per-condition trial numbers, or a more modest interpretation of the success rates, especially given that the single-robot Hard execution success is 50%.
  3. [Section 5.2, Table 2] The text states that 'These metrics validate the improved accuracy of ROMAN when integrated with Hydra-Multi as compared to traditional visual place recognition' (Section 5.2). This is not directly supported by the numbers in Table 2: ROMAN has a higher ATE RMSE than SALAD on both the Hybrid setting (10.36 m vs. 9.40 m) and the Outdoor setting (6.61 m vs. 5.67 m). The improvements in object IoU, precision, and recall are modest (e.g., 0.40 vs. 0.36 IoU on Hybrid) and are not statistically characterized. The paper should either report significance or reframe the result as a trade-off: ROMAN yields slightly worse trajectory accuracy but somewhat better scene-graph object consistency, while using drastically less memory. The current wording overstates the conclusion.
  4. [Section 5.1, Position Estimation and Relocalization] The relocalization success rates of 64% (Husky) and 65% (Spot) are reported as evidence for 'real-time, view-invariant relocalization'. While the view-invariance is interesting, a one-third failure rate means that the system frequently cannot localize, and the paper does not state how these failures are handled during task execution (e.g., retries, re-planning, or abort). The claim that this representation 'supports real-time, view-invariant relocalization' is only partially supported. The authors should report the number of relocalization trials, the failure modes (e.g., object sparsity, depth range), and the operational impact of a 64-65% success rate on the end-to-end execution results.
minor comments (4)
  1. [Section 5.2, first paragraph] There is a typo: 'occured' should be 'occurred'. Also, 'theacl jackal2' appears to be a formatting error for 'the acl_jackal2' (or similar robot name).
  2. [Section 4, first paragraph] The language-to-PDDL approach is described as an adaptation of AutoTAMP [13], but the novelty relative to [13] is not made precise. The paper should clarify what is new in the prompt structure, the scene-graph context, or the multi-robot goal assignment, so that the contribution is not conflated with the prior work.
  3. [Table 1] The definition of a 'correct plan' is not specified. It would help to state whether the plan is compared against a hand-authored ground-truth plan, whether any plan satisfying the PDDL goal counts, and how execution correctness is judged (e.g., reaching a goal region, visual verification). A precise rubric would make the success rates more interpretable.
  4. [References] Several references are incomplete, e.g., reference [36] is missing volume, issue, and page numbers, and reference [29] may be a preprint rather than a published venue. Please verify all entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the system integrates independently developed components, and the end-to-end and ablation results are measured against externally defined ground-truth goals rather than being forced by construction.

full rationale

The paper's derivation chain is not circular. Its components—Hydra scene graphs, ROMAN open-set object maps, Hydra-Multi fusion, the scene-graph planner from Ray et al., and the neuro-symbolic translation idea from AutoTAMP—are prior works by overlapping authors, but they are used as modular building blocks and the paper's central contribution is their integration plus new end-to-end experiments. The language-grounding step is evaluated against ground-truth PDDL goals that are not derived from the model outputs; Table 4 reports systematic failures on spatial-relation and region-level disambiguation, which demonstrates that the grounding metric is not satisfied vacuously. The apparent tension between 100% multi-robot Hard grounding in Table 1 and lower offline ablation scores is a consistency and reporting concern, not a circularity: no parameter is fitted to the reported successes, and no prediction is an input by definition. The footnote about lacking ground truth for real-time ablations, and the stated limitation about not exploiting open-set semantics, are honest scope restrictions rather than circular reasoning. Since no equation or construction equates a claimed result with its own input, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted constants or invented entities appear; the central claim rests on the prior correct behavior of Hydra, ROMAN, and the scene graph planner, and on the LLM grounding behaving reliably, which the paper's own ablation does not fully support.

assumptions (4)
  • domain assumption Hydra and ROMAN provide sufficiently accurate scene graphs and open-set object maps for fusion and relocalization.
    Invoked throughout Section 3; the fused scene graph and object map are the substrate for planning and localization, and their accuracy is taken from the cited papers rather than re-validated here.
  • domain assumption The text-serialized scene graph plus robot capability descriptions plus in-context examples is enough information for the LLM to produce correct PDDL goals.
    Used in Section 4 and Fig. 2; Table 4 shows this assumption is fragile for spatial and region-level disambiguation.
  • domain assumption The environment remains static and robots operate only inside the previously fused map.
    Acknowledged in Section 7: the system cannot handle exploration or environmental changes, so all plans assume the fused map stays valid during execution.
  • domain assumption PDDLStream can find correct feasible plans from the translated PDDL goals.
    Relied on in Section 4; planning success rates are reported in Table 1, and failures are attributed to grounding rather than planner correctness, so the planner is assumed sound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language-Grounded Hierarchical Planning and Execution with Multi-Robot 3D Scene Graphs." pith.science (2026). https://pith.science/paper/UQEQJY64

@misc{pith2026250607454,
  author       = {Pith},
  title        = {Pith review of: Language-Grounded Hierarchical Planning and Execution with Multi-Robot 3D Scene Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UQEQJY64}},
  note         = {Machine review of arXiv:2506.07454}
}
read the original abstract

In this paper, we introduce a multi-robot system that integrates mapping, localization, and task and motion planning (TAMP) enabled by 3D scene graphs to execute complex instructions expressed in natural language. Our system builds a shared 3D scene graph incorporating an open-set object-based map, which is leveraged for multi-robot 3D scene graph fusion. This representation supports real-time, view-invariant relocalization (via the object-based map) and planning (via the 3D scene graph), allowing a team of robots to reason about their surroundings and execute complex tasks. Additionally, we introduce a planning approach that translates operator intent into Planning Domain Definition Language (PDDL) goals using a Large Language Model (LLM) by leveraging context from the shared 3D scene graph and robot capabilities. We provide an experimental assessment of the performance of our system on real-world tasks in large-scale, outdoor environments. A supplementary video is available at https://youtu.be/8xbGGOLfLAY.

Figures

Figures reproduced from arXiv: 2506.07454 by the authors.

Figure 1
Figure 1. The concept of operations consists of a mapping and execution stage. (Left) In the mapping stage, the team of robots construct separate 3D scene graphs and open-set object maps, which are fused into a shared representation. (Right) The user provides natural language instructions, and the user intent is translated into a PDDL goal via an LLM. A PDDL-based planner generates a plan, which the robots executes. open-set … view at source ↗
Figure 2
Figure 2. Example of LLM prompt and response including snippets for the descriptions of the translation task, PDDL domain, robot capabilities, in-context examples, and instructions for the robots. commands are translated into PDDL goals using an LLM through in-context learning, and the hierarchical scene graph planner proposed in [5] solves the resulting PDDL problem, generating a sequence of actions that can be executed by a… view at source ↗
Figure 3
Figure 3. (Left) Satellite image of Camp Buckner at West Point, NY, showing the area of operation. The line segments indicate where the robots operate, and the text boxes display their lengths in meters. (Middle) Overhead view of the map constructed from a portion of the blue and green segments. (Right) The platforms used in the experiments. provided by the operator. First, we describe the end-to-end experiments where groundi… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Object matches found by the relocalization system [4] operating on the Husky front and Spot side cameras. Green lines match objects between the two different robot views. The images show areas within the operational region at Camp Buckner. Easy Medium Hard Ground Plan …

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Prompting Robot Teams with Natural Language

    cs.RO 2025-09 conditional novelty 6.0 of 10

    A natural-language team command is distilled into a small recurrent network that encodes the task as an automaton, while a graph-neural-network policy executes it in a decentralized, real-time manner.

Reference graph

Works this paper leans on

37 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    Foundations of spatial perception for robotics: Hierarchical representations and real-time systems,

    N. Hughes, Y. Chang, S. Hu, R. Talak, R. Abdulhai, J. Strader, and L. Carlone, “Foundations of spatial perception for robotics: Hierarchical representations and real-time systems,” Intl. J. of Robotics Research, 2024

    work page 2024

  2. [2]

    Hydra-Multi: Collaborative online construction of 3D scene graphs with multi-robot teams,

    Y. Chang, N. Hughes, A. Ray, and L. Carlone, “Hydra-Multi: Collaborative online construction of 3D scene graphs with multi-robot teams,” in IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems, 2023

    work page 2023

  3. [3]

    Conceptgraphs: Open-vocabulary 3D scene graphs for perception and planning,

    Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappa, C. Gan, C. M. de Melo, J. B. Tenen- baum, A. Torralba, F. Shkurti, and L. Paull, “Conceptgraphs: Open-vocabulary 3D scene graphs for perception and planning,” in IEEE Intl. Conf. on Robotics and Automation, 2024. Language-Grounded TAMP vi...

    work page 2024

  4. [4]

    Roman: Open-set object map alignment for robust view-invariant global localization,

    M. B. Peterson, Y. X. Jia, Y. Tian, A. Thomas, and J. P. How, “Roman: Open-set object map alignment for robust view-invariant global localization,” in Robotics: Science and Systems (RSS), 2025

    work page 2025

  5. [5]

    Task and motion planning in hierarchical 3D scene graphs,

    A. Ray, C. Bradley, L. Carlone, and N. Roy, “Task and motion planning in hierarchical 3D scene graphs,” Proc. of the Intl. Symp. of Robotics Research, 2024

    work page 2024

  6. [6]

    Sayplan: Grounding large language models using 3D scene graphs for scalable robot task planning,

    K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suenderhauf, “Sayplan: Grounding large language models using 3D scene graphs for scalable robot task planning,” Conference on Robot Learning, 2023

    work page 2023

  7. [7]

    Optimal scene graph planning with large language model guidance,

    Z. Dai, A. Asgharivaskasi, T. Duong, S. Lin, M.-E. Tzes, G. Pappas, and N. Atanasov, “Optimal scene graph planning with large language model guidance,” in IEEE Intl. Conf. on Robotics and Automation, 2024

    work page 2024

  8. [8]

    PDDL – the planning domain definition language,

    M. Ghallab, A. Howe, C. Knoblock, I. D. McDermott, A. Ram, M. Veloso, D. Weld, D. W. Sri, A. Barrett, D. Christianson, et al., “PDDL – the planning domain definition language,” Technical Report, Tech. Rep., 1998

    work page 1998

Show all 37 references
  1. [9]

    Indoor and outdoor 3D scene graph generation via language-enabled spatial ontologies,

    J. Strader, N. Hughes, W. Chen, A. Speranzon, and L. Carlone, “Indoor and outdoor 3D scene graph generation via language-enabled spatial ontologies,” IEEE Robotics and Automation Letters, vol. 9, no. 6, pp. 4886–4893, 2024

    2024

  2. [10]

    Least squares quantization in PCM,

    S. Lloyd, “Least squares quantization in PCM,” IEEE Transactions on Infor- mation Theory, vol. 28, no. 2, pp. 129–137, 1982

    1982

  3. [11]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., “Learning transferable visual models from natural language supervision,” in Intl. Conf. on Machine Learning (ICML), 2021, pp. 8748–8763

    2021

  4. [12]

    PDDLStream: Integrat- ing symbolic planners and blackbox samplers via optimistic adaptive planning,

    C. R. Garrett, T. Lozano-P´ erez, and L. P. Kaelbling, “PDDLStream: Integrat- ing symbolic planners and blackbox samplers via optimistic adaptive planning,” in Intl. Conf. on Automated Planning and Scheduling (ICAPS), vol. 30, 2020, pp. 440–448

    2020

  5. [13]

    AutoTAMP: Au- toregressive task and motion planning with LLMs as translators and checkers,

    Y. Chen, J. Arkin, C. Dawson, Y. Zhang, N. Roy, and C. Fan, “AutoTAMP: Au- toregressive task and motion planning with LLMs as translators and checkers,” in IEEE Intl. Conf. on Robotics and Automation, IEEE, 2024, pp. 6695–6702

    2024

  6. [14]

    LOCUS 2.0: Robust and computationally efficient lidar odometry for real-time underground 3D mapping,

    A. Reinke, M. Palieri, B. Morrell, Y. Chang, K. Ebadi, L. Carlone, and A. Agha- mohammadi, “LOCUS 2.0: Robust and computationally efficient lidar odometry for real-time underground 3D mapping,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 9043–9050, 2022

    2022

  7. [15]

    Optimal transport aggregation for visual place recog- nition,

    S. Izquierdo and J. Civera, “Optimal transport aggregation for visual place recog- nition,” in IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  8. [16]

    Resilient and distributed multi-robot visual SLAM: Datasets, experi- ments, and lessons learned,

    Y. Tian, Y. Chang, L. Quang, A. Schang, C. Nieto-Granda, J. P. How, and L. Carlone, “Resilient and distributed multi-robot visual SLAM: Datasets, experi- ments, and lessons learned,” in IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems, 2023

    2023

  9. [17]

    SuperPoint: Self-supervised interest point detection and description,

    D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperPoint: Self-supervised interest point detection and description,” in Workshop on Deep Learning for Visual SLAM at CVPR, 2018, pp. 224–236

    2018

  10. [18]

    LightGlue: Local Feature Match- ing at Light Speed,

    P. Lindenberger, P.-E. Sarlin, and M. Pollefeys, “LightGlue: Local Feature Match- ing at Light Speed,” in Intl. Conf. on Computer Vision (ICCV), 2023

    2023

  11. [19]

    3D scene graph: A structure for unified semantics, 3D space, and camera,

    I. Armeni, Z.-Y. He, J. Gwak, A. R. Zamir, M. Fischer, J. Malik, and S. Savarese, “3D scene graph: A structure for unified semantics, 3D space, and camera,” in Intl. Conf. on Computer Vision (ICCV), 2019, pp. 5664–5673. 12 Strader et al

    2019

  12. [20]

    Kimera: From SLAM to spatial perception with 3D dynamic scene graphs,

    A. Rosinol, A. Violette, M. Abate, N. Hughes, Y. Chang, J. Shi, A. Gupta, and L. Carlone, “Kimera: From SLAM to spatial perception with 3D dynamic scene graphs,” Intl. J. of Robotics Research, vol. 40, no. 12-14, pp. 1510–1546, 2021

    2021

  13. [21]

    S-graphs+: Real-time localization and mapping leveraging hierarchical representations,

    H. Bavle, J. L. Sanchez-Lopez, M. Shaheer, J. Civera, and H. Voos, “S-graphs+: Real-time localization and mapping leveraging hierarchical representations,”IEEE Robotics and Automation Letters, vol. 8, no. 8, pp. 4927–4934, 2023

    2023

  14. [22]

    Clio: Real-time task-driven open-set 3D scene graphs,

    D. Maggio, Y. Chang, N. Hughes, M. Trang, D. Griffith, C. Dougherty, E. Cristo- falo, L. Schmid, and L. Carlone, “Clio: Real-time task-driven open-set 3D scene graphs,” IEEE Robotics and Automation Letters, vol. 9, no. 10, pp. 8921–8928, 2024

    2024

  15. [23]

    Hierarchical open-vocabulary 3D scene graphs for language-grounded robot navigation,

    A. Werby, C. Huang, M. B¨ uchner, A. Valada, and W. Burgard, “Hierarchical open-vocabulary 3D scene graphs for language-grounded robot navigation,” in First Workshop on Vision-Language Models for Nav. and Manip. at ICRA, 2024

    2024

  16. [24]

    Taskography: Evaluating robot task planning over large 3D scene graphs,

    C. Agia, K. M. Jatavallabhula, M. Khodeir, O. Miksik, V. Vineet, M. Mukadam, L. Paull, and F. Shkurti, “Taskography: Evaluating robot task planning over large 3D scene graphs,” in Conference on Robot Learning, 2022, pp. 46–58

    2022

  17. [25]

    Language models as zero- shot planners: Extracting actionable knowledge for embodied agents,

    W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero- shot planners: Extracting actionable knowledge for embodied agents,” 2022

    2022

  18. [26]

    Do as I can, not as I say: Grounding language in robotic affor- dances,

    M. A. et al., “Do as I can, not as I say: Grounding language in robotic affor- dances,” in Conference on Robot Learning, vol. 205, 2023, pp. 287–318

    2023

  19. [27]

    PDDL planning with pretrained large language models,

    T. Silver, V. Hariprasad, R. S. Shuttleworth, N. Kumar, T. Lozano-P´ erez, and L. P. Kaelbling, “PDDL planning with pretrained large language models,” in NeurIPS foundation models for decision making workshop, 2022

    2022

  20. [28]

    Large language models still can’t plan (a benchmark for LLMs on planning and reasoning about change),

    K. Valmeekam, A. Olmo, S. Sreedharan, and S. Kambhampati, “Large language models still can’t plan (a benchmark for LLMs on planning and reasoning about change),” in NeurIPS Foundation Models for Decision Making Workshop, 2022

    2022

  21. [29]

    Translating natural language to planning goals with large-language models,

    Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, and H. Soh, “Translating natural language to planning goals with large-language models,” arXiv preprint:2302.05128, 2023

    2023 arXiv

  22. [30]

    LLM+P: Empowering large language models with optimal planning proficiency,

    B. Liu, Y. Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone, “LLM+P: Empowering large language models with optimal planning proficiency,” arXiv preprint:2304.11477, 2023

    2023 arXiv

  23. [31]

    ORB: An efficient alter- native to SIFT or SURF,

    E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: An efficient alter- native to SIFT or SURF,” in Intl. Conf. on Computer Vision (ICCV), 2011, pp. 2564–2571

    2011

  24. [32]

    Bags of binary words for fast place recogni- tion in image sequences,

    D. G´ alvez-L´ opez and J. D. Tardos, “Bags of binary words for fast place recogni- tion in image sequences,” IEEE Trans. Robotics, vol. 28, no. 5, pp. 1188–1197, 2012

    2012

  25. [33]

    SuperGlue: Learn- ing feature matching with graph neural networks,

    P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperGlue: Learn- ing feature matching with graph neural networks,” in IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 4938–4947

    2020

  26. [34]

    X-view: Graph- based semantic multi-view localization,

    A. Gawel, C. Del Don, R. Siegwart, J. Nieto, and C. Cadena, “X-view: Graph- based semantic multi-view localization,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1687–1694, 2018

    2018

  27. [35]

    SlideSLAM: Sparse, lightweight, decentralized metric-semantic SLAM for multi-robot navigation,

    X. Liu, J. Lei, A. Prabhu, Y. Tao, I. Spasojevic, P. Chaudhari, N. Atanasov, and V. Kumar, “SlideSLAM: Sparse, lightweight, decentralized metric-semantic SLAM for multi-robot navigation,” arXiv preprint:2406.17249, 2024

    2024

  28. [36]

    GOReloc: Graph-based object-level relocal- ization for visual SLAM,

    Y. Wang, C. Jiang, and X. Chen, “GOReloc: Graph-based object-level relocal- ization for visual SLAM,” IEEE Robotics and Automation Letters, 2024

    2024

  29. [37]

    CLIPPER: Robust data association without an initial guess,

    P. C. Lusk and J. P. How, “CLIPPER: Robust data association without an initial guess,” IEEE Robotics and Automation Letters, vol. 9, no. 4, pp. 3092– 3099, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.