Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

GraphPad: Inference-Time 3D Scene Graph Updates for Embodied Question Answering

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GraphPad lets a vision-language model patch its own 3D scene memory at inference time, lifting embodied-question-answering accuracy from 52.3% to 55.3% while using 5 frames instead of 25.

desk verdict A genuinely new idea — letting a VLM edit its 3D scene graph at inference time — but the evaluation is too thin to credit the mechanism: the frame-count claim counts only initial frames, and there is no matched-budget baseline. read the letter →

arxiv 2506.01174 v1 pith:ICCA2U3Y submitted 2025-06-01 cs.AI

Muhammad Qasim Ali , Saeejith Nair , Alexander Wong , Yuchen Cui , Yuhao Chen This is my paper · ORCID
classification cs.AI
keywords embodiedquestionanswering3Dscenegraphsgrapheditingvision-languagemodelsinference-timememoryupdateOpenEQAnavigationlogspatialreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GraphPad is built around a simple proposal: instead of building a scene representation once and treating it as fixed, an embodied agent should be able to edit its own 3D memory while answering a question, through language-level function calls. The paper shows this works on the episodic-memory version of the OpenEQA benchmark, where a single vision-language model improves from 52.3% to 55.3% accuracy after gaining the ability to retrieve frames, insert objects or relations, and write scratchpad notes. This improvement comes with an efficiency gain: GraphPad needs only 5 initial frames rather than the 25 frames of the image-only baseline. If correct, the result means that targeted, query-driven perception can replace exhaustive preprocessing for structured scene understanding.

What carries the argument

The load-bearing object is the Structured Scene Memory (SSM): a directed multigraph $G = (N, E)$ of object tracks with pooled visual and language embeddings, captions, point clouds, and four spatial relation types (on top of, subpart of, contained in, attached to), linked to a graphical scratchpad, a frame memory of evenly spaced keyframes, and a navigation log that indexes each frame's room, field of view, motion, and visible node IDs. What carries the argument is the Modifiability API set -- find objects, analyze objects, and analyze frame -- exposed to the reasoning vision-language model as ordinary language calls. The agentic loop lets the model decide when memory is incomplete, choose a promising frame from the navigation log, run one of the APIs, and fuse the returned JSON patch into the graph and notes, so the representation becomes task-conditioned without retraining.

What would settle it

Replay GraphPad's recorded API calls on an OpenEQA subset twice: once with each insertion replaced by a deliberately wrong object label or relation, and once with insertions taken from ground-truth annotations. If the corrupted runs do not fall below the 25-frame image-only baseline, the +3.0-point gain is not caused by faithful memory edits; if the oracle runs do not beat the model's own edits, the bottleneck is elsewhere in the pipeline.

Watch

Extended reading notes

Core claim

The central claim is that language-driven, inference-time updates to a 3D scene graph produce more informative representations for embodied question answering than either raw images or a static pre-built graph. GraphPad stores the environment as a mutable scene graph with object tracks, four view-invariant spatial relations, a navigation log indexing keyframes, and a scratchpad for task-specific notes; during reasoning the same vision-language model that answers the question can call find objects, analyze objects, or analyze frame to patch missing nodes, relations, and attributes. On OpenEQA's episodic-memory task, this reaches 55.3% accuracy, a +3.0 percentage point gain over an image-only baseline with the same model, while processing 5 initial frames instead of 25. The paper also reports that the navigation log contributes the largest single ablation gain (+9.6 points over frames alone) and that the frame-level API adds 3.6 points over a static scene graph.

Load-bearing premise

The gain rests on the assumption that the same vision-language model that answers questions also produces insertions -- new objects, relations, and notes -- accurate enough that writing them into the scene graph helps more than it hurts; the paper's own limitation section states there is no verification mechanism for detection quality.

Editorial extensions

If this is right

  • Embodied agents can answer questions about a scene more accurately by editing a structured memory at test time than by feeding all captured frames to the vision-language model, at least on OpenEQA.
  • Static 3D scene graphs are not a dead end: their coverage can be repaired on demand once the query is known.
  • A structured navigation log that tells the model what each frame contains may be worth more per token than the frames themselves, since it contributes the largest single ablation gain.
  • Targeted updates help most for attribute recognition, object state, and functional reasoning, while raw object recognition and spatial localization still favor image-only input.
  • The query-driven update policy is cheap in practice: most questions need two or fewer API calls.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension not tested in the paper is to add a verification stage for insertions, since the paper's own limitation section warns that unverified detections can propagate errors into the graph.
  • The navigation log's large ablation contribution suggests that learned frame indexing, rather than the graph itself, might be the scalability bottleneck; one could test whether a text-only frame index alone reproduces GraphPad's gains.
  • The same edit-on-demand pattern should transfer to video question answering or instruction following, where a long recorded observation stream is re-queried by different tasks.
  • Because the update APIs are language-callable and domain-agnostic, they are a plausible interface for a robot to decide what to look at next during active perception, not only after recording.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces GraphPad, an inference-time memory editing system for embodied question answering. GraphPad maintains a structured scene memory consisting of a scene graph, a navigation log, a frame memory, and a scratchpad; during inference, the answering VLM can call three APIs (find objects, analyze objects, analyze frame) to retrieve additional frames, insert objects or relations, and write task-specific notes. The system is evaluated on the episodic-memory variant of OpenEQA using Gemini 2.0 Flash, reporting 55.3% accuracy versus 52.3% for the same VLM with image-only input, while claiming to use five times fewer input frames (5 versus 25). The paper argues that online, language-driven refinement of 3-D memory yields more informative representations without additional training or data collection.

Significance. If the causal claim were established, the idea would be valuable: it offers a general mechanism for adapting structured memories to task requirements at inference time, avoiding retraining and exhaustive preprocessing. The API design is simple and clearly specified, the evaluation uses a public benchmark, and the paper reports the API-call distribution and honestly lists limitations such as detection-error propagation. However, the headline comparison is not compute-matched, the frame-count claim is contradicted by the paper's own accounting of API calls, and the empirical evidence lacks significance testing and uses an unexplained ablation subset. These issues currently prevent the paper from supporting its central claim as stated.

major comments (3)
  1. [Abstract; Section 4.1] The 'five times fewer input frames' claim is not supported by the paper's own numbers. Section 3.2 states that frames requested by the APIs are appended to Frame Memory, Section 4.5 reports an average of 1.9 API calls per question, and Section 5 states that each API call requires a full VLM inference pass. The fair per-question frame count is therefore about 5 + 1.9 = 6.9 (and up to the m=20 search depth), not 5; relative to the 25-frame image-only baseline this is roughly 3.6x fewer, not 5x. More importantly, the image-only baseline receives no additional inference passes, while GraphPad spends up to 20 sequential VLM calls; without an API-call-matched or compute-matched baseline, the +3.0 percentage point gain cannot be attributed to the scene-graph editing mechanism rather than to the extra inference budget. The manuscript also does not clarify whether the Navigation Log indexes all keyframes in the scan or only the frames currently in Frame Memory; in the former case GraphPad has access to the full keyframe set, further invalidating the frame-count comparison.
  2. [Section 3.1; Section 3.3] The scratchpad is self-referential: the VLM writes notes through the find/analyze APIs and is later prompted to cite those notes as 'semantic evidence' in the final answer. Because the same model generates and consumes these annotations, improved accuracy may reflect self-consistency with the model's own generated text rather than new grounded perceptual information. The paper provides no control that isolates the contribution of graph/node insertion from the contribution of the extra self-generated notes (for example, an ablation that performs the same API calls but does not let the answerer see the notes, or one that replaces the notes with text from an independent detector). The limitation section's admission that there is no verification mechanism for object detection quality compounds this attribution problem.
  3. [Sections 4.2-4.4; Tables 2, 3, 5] The empirical support for the component claims is weak. The ablations are computed on an 'OpenEQA184' subset whose selection is not described, and no error bars, confidence intervals, or significance tests are reported for any table. The headline +3.0 percentage point difference on the full set may be within noise; moreover, Table 5 shows that GraphPad is worse than the image-only baseline on object localization (-3.0), spatial understanding (-4.7), and object recognition (-3.8), so the aggregate gain masks category-level degradation. The conclusion that the memory becomes 'more informative' therefore needs a statistical analysis and a category-wise interpretation, not just an aggregate comparison.
minor comments (4)
  1. [Section 4.1] Please specify exactly how the 25-frame image-only baseline is constructed for HM3D (k=5) and ScanNet (k=20), and how the n_img=5 initial frames are selected from the k-th frame set; the current description leaves the relationship between these quantities ambiguous.
  2. [Table 1] The comparisons against 'GPT-4 w/ CG' and 'GPT-4 w/ SVM' use a different base VLM from GraphPad; the paper should state explicitly that these are not controlled comparisons and that differences may reflect the base model as much as the memory representation.
  3. [Section 5] For full transparency, please report the total number of VLM inference calls used for the initial SSM construction (detection, edge discovery, caption consolidation) in addition to the per-question API calls, since the current overhead discussion considers only API-call latency.
  4. [Equation (1)] The definition of Gij as 'the fraction of points in Di within δg of Tj' is ambiguous when Tj is a track with an accumulated point cloud; please specify whether the fraction is with respect to Di or to the union of points in Di and Tj.

Circularity Check

1 steps flagged · score 5.0 of 10

Self-referential scratchpad evidence partially constructs the claimed accuracy gain; the five-times-fewer-frames claim is misleading but not circular.

  1. other [Section 3.1 (Agentic Reasoning Loop) and Section 3.3 (Modifiability APIs)]
    "To ensure that the responses are grounded in all memory components, we prompt the VLM that it must support its final answer with dual evidence—visual evidence from frames in the Frame Memory and semantic evidence from notes in the Scratch-Pad."

    The 'semantic evidence' the final answer must cite is written into the Scratch-Pad by the same VLM during the same inference: each API 'returns ... query-relevant notes' (Sec. 3.3), and the VLM itself chooses the API, the frame, and the natural-language query (Sec. 3.1). The final answer is therefore a function of the model's own prior outputs on the same question, so the +3.0 pp over the image-only baseline can be inflated by self-consistency rather than by independent perceptual grounding. Section 5 concedes there is no verification mechanism for detection quality, so erroneous notes can become the cited 'evidence.' The measured benefit of the editing loop is thus partially constructed by the evaluation protocol.

full rationale

GraphPad is an empirical systems paper evaluated on an external benchmark (OpenEQA), so most of its comparisons are not derivations and do not reduce to their inputs by construction. The principal circular concern is the self-referential scratchpad loop: the same Gemini 2.0 Flash model generates the notes via find/analyze API calls (using queries it chooses from the question) and is then prompted to cite those notes as 'semantic evidence' in the final answer. This creates a consistency loop that can produce accuracy gains even if the notes add no new grounded information; the paper's own limitation about the absence of detection verification makes this gap concrete. However, the navigation-log ablation (+9.6 pp) and search-depth results are independent of the scratchpad loop and give the central memory-organization claim some non-circular support. The 'five times fewer input frames' claim is not circular but is misleading: Sec. 3.2 says API-requested frames are appended without eviction, Sec. 4.5 reports an average of 1.9 API calls per question, and Sec. 5 says each call requires a full VLM inference pass, so the efficiency comparison counts only the initial 5 frames and ignores the additional frames and inference budget. Overall, the core accuracy result is empirical and externally benchmarked, but the attribution of the +3.0 pp to memory editing is partially self-referential; score 5.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The system depends on several hand-set hyperparameters (frame count, search depth, tracking thresholds) and on the assumption that the VLM's detections and notes are trustworthy enough to edit the scene graph. No parameter-free derivation is offered.

free parameters (5)
  • Initial frame memory size n_img = 5 (main result); ablated 2-6
    The number of keyframes initially placed in the Frame Memory is a hand-chosen hyperparameter that affects accuracy (Table 4).
  • Search depth m = 20 (main result); ablated 0-20
    Maximum number of API calls per question, hand-set at 20. Accuracy changes with m (Table 3).
  • Track association thresholds = 0.7 visual, 0.8 language, 0.4 spatial, vote > 2, delta_g = 5cm
    Hand-set thresholds in Eq. (1) determine object tracking and graph quality.
  • Edge discovery interval = every 3 frames
    Frequency of VLM relation prediction is chosen without ablation.
  • DBSCAN and voxel parameters = sklearn defaults, 0.02 m voxel
    Point cloud processing settings are fixed without analysis.
assumptions (3)
  • domain assumption The four relation types (on top of, subpart of, contained in, attached to) are sufficient to answer OpenEQA spatial questions.
    Edge discovery is restricted to these four types (Section 3.2). If questions require other relation types, the graph cannot represent them.
  • domain assumption The VLM's bounding box, caption, and relation outputs are accurate enough that inserting them into the scene graph improves rather than degrades answers.
    The paper states there is no verification of detection quality (Section 5). The whole approach assumes inserted information is reliable.
  • domain assumption The initial sparse set of keyframes contains the visual information needed to answer the question (the agent cannot sense new frames).
    GraphPad follows the episodic-memory EQA protocol (Section 3.1), so all API-requested frames come from the pre-recorded scan; if an object never appears in any stored frame, it cannot be recovered.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GraphPad: Inference-Time 3D Scene Graph Updates for Embodied Question Answering." pith.science (2026). https://pith.science/paper/ICCA2U3Y

@misc{pith2026250601174,
  author       = {Pith},
  title        = {Pith review of: GraphPad: Inference-Time 3D Scene Graph Updates for Embodied Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ICCA2U3Y}},
  note         = {Machine review of arXiv:2506.01174}
}
read the original abstract

Structured scene representations are a core component of embodied agents, helping to consolidate raw sensory streams into readable, modular, and searchable formats. Due to their high computational overhead, many approaches build such representations in advance of the task. However, when the task specifications change, such static approaches become inadequate as they may miss key objects, spatial relations, and details. We introduce GraphPad, a modifiable structured memory that an agent can tailor to the needs of the task through API calls. It comprises a mutable scene graph representing the environment, a navigation log indexing frame-by-frame content, and a scratchpad for task-specific notes. Together, GraphPad serves as a dynamic workspace that remains complete, current, and aligned with the agent's immediate understanding of the scene and its task. On the OpenEQA benchmark, GraphPad attains 55.3%, a +3.0% increase over an image-only baseline using the same vision-language model, while operating with five times fewer input frames. These results show that allowing online, language-driven refinement of 3-D memory yields more informative representations without extra training or data collection.

Figures

Figures reproduced from arXiv: 2506.01174 by the authors.

Figure 1
Figure 1. Overview of GraphPad for embodied question answering. The top row illustrates the Structured Scene Memory (SSM) com [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Distribution of API calls per query on OpenEQA [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A robotic agent operating system with source-grounded graph memory and split-wise self-evolution improves long-horizon embodied task success and memory QA scores over baseline controllers.

Reference graph

Works this paper leans on

30 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    Remembr: Building and reasoning over long- horizon spatio-temporal memory for robot navigation

    Abrar Anwar, John Welsh, Joydeep Biswas, Soha Pouya, and Yan Chang. Remembr: Building and reasoning over long- horizon spatio-temporal memory for robot navigation. arXiv preprint arXiv:2409.13682, 2024. 1, 2

    arXiv 2024

  2. [2]

    Mobility vla: Multimodal instruction navigation with long-context vlms and topological graphs

    Hao-Tien Lewis Chiang, Zhuo Xu, Zipeng Fu, Mithun George Jacob, Tingnan Zhang, Tsang-Wei Ed- ward Lee, Wenhao Yu, Connor Schenck, David Rendleman, Dhruv Shah, Fei Xia, Jasmine Hsu, Jonathan Hoech, Pete Florence, Sean Kirmani, Sumeet Singh, Vikas Sindhwani, Carolina Parada, Chelsea Finn, Peng Xu, Sergey Levine, and Jie Tan. Mobility vla: Multimodal instruc...

    arXiv 2024

  3. [3]

    Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017. 5

    work page 2017

  4. [4]

    Embodied question answer- ing

    Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Embodied question answer- ing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 1

    work page 2018

  5. [5]

    Pla: Language-driven open- vocabulary 3d scene understanding

    Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. Pla: Language-driven open- vocabulary 3d scene understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7010–7019, 2023. 2

    2023

  6. [6]

    Collaborative dynamic 3d scene graphs for automated driving

    Elias Greve, Martin B ¨uchner, Niclas V ¨odisch, Wolfram Burgard, and Abhinav Valada. Collaborative dynamic 3d scene graphs for automated driving. In 2024 IEEE Inter- national Conference on Robotics and Automation (ICRA) , pages 11118–11124, 2024. 2

    work page 2024

  7. [7]

    Conceptgraphs: Open-vocabulary 3d scene graphs for per- ception and planning

    Qiao Gu, Ali Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, et al. Conceptgraphs: Open-vocabulary 3d scene graphs for per- ception and planning. In 2024 IEEE International Confer- ence on Robotics and Automation (ICRA), pages 5021–5028. IEEE, 2024. 2

    work page 2024

  8. [8]

    Language-grounded dy- namic scene graphs for interactive object search with mobile manipulation

    Daniel Honerkamp, Martin B ¨uchner, Fabien Despinoy, Tim Welschehold, and Abhinav Valada. Language-grounded dy- namic scene graphs for interactive object search with mobile manipulation. IEEE Robotics and Automation Letters, 2024. 2

    work page 2024

Show all 30 references
  1. [9]

    Visual language maps for robot navigation

    Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps for robot navigation. In Pro- ceedings of the IEEE International Conference on Robotics and Automation (ICRA), London, UK, 2023. 2

    2023

  2. [10]

    Hy- dra: A real-time spatial perception system for 3d scene graph construction and optimization

    Nathan Hughes, Yun Chang, and Luca Carlone. Hy- dra: A real-time spatial perception system for 3d scene graph construction and optimization. arXiv preprint arXiv:2201.13360, 2022. 1, 2

    2022 arXiv

  3. [11]

    Llm-enhanced scene graph learning for household rearrangement

    Wenhao Li, Zhiyuan Yu, Qijin She, Zhinan Yu, Yuqing Lan, Chenyang Zhu, Ruizhen Hu, and Kai Xu. Llm-enhanced scene graph learning for household rearrangement. In SIG- GRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 1, 3

    2024

  4. [12]

    Dynamem: Online dynamic spatio-semantic mem- ory for open world mobile manipulation

    Peiqi Liu, Zhanqiu Guo, Mohit Warke, Soumith Chintala, Chris Paxton, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. Dynamem: Online dynamic spatio-semantic mem- ory for open world mobile manipulation. arXiv preprint arXiv:2411.04999, 2024. 2

    2024 arXiv

  5. [13]

    Clio: Real-time task-driven open-set 3d scene graphs

    Dominic Maggio, Yun Chang, Nathan Hughes, Matthew Trang, Dan Griffith, Carlyn Dougherty, Eric Cristofalo, Lukas Schmid, and Luca Carlone. Clio: Real-time task-driven open-set 3d scene graphs. arXiv preprint arXiv:2404.13696, 2024. 2

    2024 arXiv

  6. [14]

    Openeqa: Embodied question answering in the era of foun- dation models

    Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mcvay, Oleksandr Maksymets, Sergio Arnaud, et al. Openeqa: Embodied question answering in the era of foun- dation models. In Proceedings of the IEEE/CVF Conference on...

    2024

  7. [15]

    Openscene: 3d scene understanding with open vocabularies

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 815–824, 2023. 2

    2023

  8. [16]

    Introducing Gemini 2.0: our new AI model for the agen- tic era

    Sundar Pichai, Demis Hassabis, and Koray Kavukcuoglu. Introducing Gemini 2.0: our new AI model for the agen- tic era. https : / / blog . google / technology / ai / introducing - gemini - 2/, 2024. Google Blog, https://blog.google/technology/ai/ introducing-gemini-2/. 5

    2024

  9. [17]

    Saynav: Grounding large language models for dynamic planning to navigation in new environments

    Abhinav Rajvanshi, Karan Sikka, Xiao Lin, Bhoram Lee, Han pang Chiu, and Alvaro Velasquez. Saynav: Grounding large language models for dynamic planning to navigation in new environments. In 34th International Conference on Automated Planning and Scheduling, 2024. 2

    2024

  10. [18]

    Habitat-matterport 3d dataset (HM3d): 1000 large- scale 3d environments for embodied AI

    Santhosh Kumar Ramakrishnan, Aaron Gokaslan, Erik Wi- jmans, Oleksandr Maksymets, Alexander Clegg, John M Turner, Eric Undersander, Wojciech Galuba, Andrew West- bury, Angel X Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-matterport 3d dataset (HM3d): 1000 large- s...

    2021

  11. [19]

    Sayplan: Ground- ing large language models using 3d scene graphs for scalable task planning

    Krishan Rana, Jesse Haviland, Sourav Garg, Jad Abou- Chakra, Ian Reid, and Niko Suenderhauf. Sayplan: Ground- ing large language models using 3d scene graphs for scalable task planning. In 7th Annual Conference on Robot Learning,

  12. [20]

    Rosinol, A

    A. Rosinol, A. Violette, M. Abate, N. Hughes, Y . Chang, J. Shi, A. Gupta, and L. Carlone. Kimera: from SLAM to spatial perception with 3D dynamic scene graphs. In arXiv preprint arXiv:2101.06894 , 2021. https: //arxiv. org/pdf/2101.06894.pdf. 2

    2021 arXiv

  13. [21]

    Grapheqa: Using 3d semantic scene graphs for real-time embodied question answering

    Saumya Saxena, Blake Buchanan, Chris Paxton, Bingqing Chen, Narunas Vaskevicius, Luigi Palmieri, Jonathan Fran- cis, and Oliver Kroemer. Grapheqa: Using 3d semantic scene graphs for real-time embodied question answering. arXiv preprint arXiv:2412.14480, 2024. 3

    2024

  14. [22]

    Bumble: Unifying reasoning and acting with vision-language models for building-wide mobile manipula- tion

    Rutav Shah, Albert Yu, Yifeng Zhu, Yuke Zhu, and Roberto Mart´ın-Mart´ın. Bumble: Unifying reasoning and acting with vision-language models for building-wide mobile manipula- tion. arXiv preprint arXiv:2410.06237, 2024. 3

    2024 arXiv

  15. [23]

    Search3d: Hierarchical open-vocabulary 3d segmentation

    Ayca Takmaz, Alexandros Delitzas, Robert W Sumner, Francis Engelmann, Johanna Wald, and Federico Tombari. Search3d: Hierarchical open-vocabulary 3d segmentation. arXiv preprint arXiv:2409.18431, 2024. 2

    2024 arXiv

  16. [24]

    Openin: Open-vocabulary instance-oriented navigation in dynamic domestic environ- ments

    Yujie Tang, Meiling Wang, Yinan Deng, Zibo Zheng, Jingchuan Deng, and Yufeng Yue. Openin: Open-vocabulary instance-oriented navigation in dynamic domestic environ- ments. arXiv preprint arXiv:2501.04279, 2025. 2

    2025 arXiv

  17. [25]

    Karma: Augmenting embodied ai agents with long-and-short term memory systems

    Zixuan Wang, Bo Yu, Junzhe Zhao, Wenhao Sun, Sai Hou, Shuai Liang, Xing Hu, Yinhe Han, and Yiming Gan. Karma: Augmenting embodied ai agents with long-and-short term memory systems. arXiv preprint arXiv:2409.14908 , 2024. 2

    2024 arXiv

  18. [26]

    Hierarchical open- vocabulary 3d scene graphs for language-grounded robot navigation

    Abdelrhman Werby, Chenguang Huang, Martin B ¨uchner, Abhinav Valada, and Wolfram Burgard. Hierarchical open- vocabulary 3d scene graphs for language-grounded robot navigation. In First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024, 2024. 3, 4

    2024

  19. [27]

    C-pack: Packed resources for general chinese embeddings, 2024

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. C-pack: Packed resources for general chinese embeddings, 2024. 4

    2024

  20. [28]

    Embodied-rag: General non-parametric embodied memory for retrieval and generation

    Quanting Xie, So Yeon Min, Tianyi Zhang, Kedi Xu, Aarav Bajaj, Ruslan Salakhutdinov, Matthew Johnson-Roberson, and Yonatan Bisk. Embodied-rag: General non-parametric embodied memory for retrieval and generation. arXiv preprint arXiv:2409.18313, 2024. 2, 3

    2024 arXiv

  21. [29]

    3d-mem: 3d scene memory for embodied exploration and reasoning

    Yuncong Yang, Han Yang, Jiachen Zhou, Peihao Chen, Hongxin Zhang, Yilun Du, and Chuang Gan. 3d-mem: 3d scene memory for embodied exploration and reasoning. arXiv preprint arXiv:2411.17735, 2024. 2, 3, 5

    2024 arXiv

  22. [30]

    Tag map: A text-based map for spatial rea- soning and navigation with large language models

    Mike Zhang, Kaixian Qu, Vaishakh Patil, Cesar Cadena, and Marco Hutter. Tag map: A text-based map for spatial rea- soning and navigation with large language models. arXiv preprint arXiv:2409.15451, 2024. 3

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.