Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 3D voxel belief map sets new best scores for zero-shot object navigation.

desk verdict A genuinely integrated 3D belief-map navigation system with a SOTA claim that outruns its statistical support; worth serious review, but the referees should demand seeds, error bars, and a less grandiose reading of the 'posterior'. read the letter →

arxiv 2506.06487 v1 pith:YGTSIZNN submitted 2025-05-27 cs.RO

Zibo Zhou , Yue Hu , Lingkai Zhang , Zonglin Li , Siheng Chen This is my paper · ORCID
classification cs.RO
keywords zero-shotobjectnavigation3Dvoxelbeliefmaphierarchicalsemanticmappinglargelanguagemodelsvision-languageCLIPfrontier-basedexplorationpathplanning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BeliefMapNav claims that a robot searching for an object it has never seen can navigate more accurately and more efficiently if it maintains a dense 3D voxel map of where the target is likely to be, rather than picking goals greedily from a language model's guesses. The system combines CLIP-based visual features at scene, region, and object scales with landmarks and relevance scores produced by an LLM, then fuses this prior with a visibility map that records where the robot has already looked. A planner orders frontier viewpoints by minimizing the expected travel distance weighted by the belief of detecting the target, and replans at every step. On the HM3D, MP3D, and HSSD benchmarks the paper reports state-of-the-art success rate and success weighted by path length among zero-shot baselines, including a 46.4% SPL improvement over the previous best-SR method on HM3D. If correct, this would make zero-shot navigation practical enough to use in homes, warehouses, and inspection tasks without task-specific training.

What carries the argument

The central object is the 3D voxel-based belief map $B=\{(u,b_u)\}$, a dense voxel grid whose values express prior belief that the target exists at each location, built by projecting multi-scale CLIP image features into a hierarchical semantic voxel map and combining them with LLM-generated landmark relevance scores. The two operations that carry the argument are the fusion $\hat b^{\text{post}}_u = \hat p^v_u \cdot b_u$, which uses a visibility map to discount places already observed, and the frontier observation belief $P_{\text{obs}}(x_f) = \max_\theta \big( \sum_{u \in O^\theta_{\text{map}}(x_f)} \hat b^{\text{post}}_u + |O^\theta_{\text{unk}}(x_f)| w_{\text{unobserved}} \big)$, which converts the posterior into a score per viewing direction. The planning module then minimizes expected search distance over frontier permutations using this score as the detection probability, solved by GPU-accelerated simulated annealing with A* path costs. This machinery links semantic reasoning to geometric search in a single optimization objective.

What would settle it

Take a fixed set of HM3D episodes, record the belief map at each episode, and compare the planner's frontier rankings with the actual locations where the target is found; if high-belief frontiers are not systematically more likely to contain the target than low-belief frontiers, the $P_{\text{obs}}$ scores are not probabilities and the expected-distance objective is not minimizing expected search cost.

Watch

Extended reading notes

Core claim

The paper's central discovery is that fusing LLM commonsense with hierarchical spatial-visual semantics inside a 3D voxel belief map lets a zero-shot navigator localize a target from free-form text more precisely than prior BEV value maps or greedy LLM/VLM goal selection, and that planning over this belief with an expected-distance objective yields both higher success and shorter paths. Concretely, each voxel stores belief $b_u = \sum_{l_t} \sum_i \alpha^{l_t}_i p^{l_t}_{u,i} + p_{u,\text{target}}$, a weighted sum of cosine similarities between CLIP text embeddings of LLM-generated landmarks and stored hierarchical visual features. Multiplying by the visibility confidence $\hat p^v_u$ gives a posterior belief $\hat b^{\text{post}}_u = \hat p^v_u b_u$, and the planner selects the frontier ordering $\pi^* = \arg\min_\pi \sum_{i=1}^n (\sum_{k=1}^i d_{A^*}(f_{\pi_{k-1}}, f_{\pi_k})) P_{\text{obs}}(f_{\pi_i})$ where $P_{\text{obs}}$ aggregates posterior belief over each frontier's field of view. Across HM3D, MP3D, and HSSD, the paper reports the best SR and SPL among the zero-shot methods it compares against, with the largest gains on HSSD.

Load-bearing premise

The whole search plan assumes that unnormalized sums of cosine similarities behave like true probabilities of finding the target, so multiplying them by a visibility score and adding them over a frontier's field of view yields a genuine detection likelihood rather than an arbitrary heuristic.

Editorial extensions

If this is right

  • On HM3D, the reported 61.4% SR and 30.6% SPL surpass all zero-shot baselines listed, with SPL 46.4% higher than InstructNav's.
  • On HSSD, the reported 65.2% SR and 32.1% SPL exceed the best prior zero-shot results by 27.8% in SR and 28.9% in SPL.
  • The ablation results attribute part of the gain to the visibility map: removing it drops SR by 8.48% and SPL by 11.4%, suggesting that avoiding revisited regions is a measurable component of efficiency.
  • Removing the belief-based planner lowers SR by 10.4% and SPL by 7.27%, supporting the claim that sequential expected-distance planning beats greedy highest-belief goal selection.
  • The system runs on a single RTX 4090 with 13GB VRAM and replans at every step, so the approach is compatible with real-time onboard search rather than offline map building.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the belief-map formulation is agnostic to what fills the voxels, so the same expected-distance planner could be applied to other priors—object affordances, learned co-occurrence statistics, or human-annotated heatmaps—without changing the optimization.
  • Because $b_u$ is an unnormalized sum of cosine similarities, a testable extension is to calibrate these scores against empirical detection frequencies; if calibration changes frontier rankings, then the reported gains are partly due to the planner's structure rather than the belief values.
  • The planning objective treats $P_{\text{obs}}$ as a probability of detection; a sharper formulation would model missed detections and observation dependencies across frontiers, which could further improve path selection in cluttered scenes.
  • The same voxel belief map could support mobile manipulation by storing not just where an object is likely to be, but where it can be reached and grasped, turning the search planner into a pick-and-place planner.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. BeliefMapNav proposes a zero-shot object navigation system centered on a 3D voxel-based belief map. The method constructs a hierarchical semantic voxel map from multi-scale CLIP features, combines it with LLM-generated hierarchical landmarks and relevance scores to form a prior belief volume, multiplies this prior by a visibility-based confidence map to obtain a posterior belief, aggregates posterior beliefs over each frontier's field of view, and then plans a frontier visiting sequence by simulated annealing over a distance-weighted objective. Experiments on HM3D, MP3D, and HSSD report success rate and SPL numbers, and ablations break down the contributions of the planner, visibility map, semantic levels, landmarks, and vision-language encoders.

Significance. If the reported results hold, the paper makes a useful contribution to zero-shot object navigation: it introduces a genuinely 3D hierarchical belief representation, explicit frontier FOV aggregation, and a non-greedy path-sequence planner, and it evaluates on three standard benchmarks. The paper also provides ablations for each module and a failure analysis on HM3D, which are strengths. The significance is conditional, however, because the headline state-of-the-art claim rests on point estimates with no uncertainty quantification, and because the planner's interpretation as minimizing expected search cost is not mathematically supported by the unnormalized belief scores.

major comments (3)
  1. [Sec. 4.2, Table 1] The state-of-the-art claim is not statistically supported on MP3D or for HM3D SPL. On MP3D the reported advantage over the next-best zero-shot method is +0.1 SR (37.3 vs. 37.2, OpenFMNav) and +0.1 SPL (17.6 vs. 17.5, VLFM); on HM3D the SPL advantage over VLFM is +0.2 (30.6 vs. 30.4). With roughly 2,000 to 2,200 episodes per benchmark, these differences are within one standard error of a success-rate difference, and the paper provides no confidence intervals, significance tests, or multiple-seed results. Since the abstract and Section 4.2 explicitly claim SOTA on all three datasets, these point estimates alone are load-bearing and insufficient. Please provide uncertainty quantification or reframe the MP3D/HM3D claims as competitive rather than SOTA.
  2. [Sec. 3.4.2 and Sec. 3.5] The probabilistic interpretation of the planner's objective is unsupported as written. The belief score b_u in Sec. 3.3.3 is an unnormalized weighted sum of CLIP cosine similarities; the posterior is formed by multiplication with the visibility confidence p_v, and P_obs is a sum of these unnormalized values (plus a constant term for unobserved voxels) over the FOV. No normalization or calibration step makes these values detection probabilities. Consequently, the objective in Sec. 3.5, called the expected search cost, is not a true expectation: a proper expected cost requires the P_obs(f_i) to be normalized and requires an explicit term for the case where the target is never found. As written, W(pi) is a weighted-distance heuristic. Please either calibrate the belief scores into probabilities and include the not-found term, or explicitly reframe the contribution as a heuristic objective and validate it against alternatives in the ablations.
  3. [Sec. 4.1, Implementation details] The statement that 'local planner parameters vary slightly by dataset' is a potential confound for the zero-shot comparison. Since the local planner is part of the full navigation system, per-dataset tuning of its parameters means the benchmark comparison is not fully fixed-hyperparameter zero-shot. The paper should report the exact local-planner parameter values used for each dataset and show that the results are stable across them, or report results with a single unified setting. Without this, the marginal MP3D advantage could be attributable to dataset-specific tuning rather than to the proposed belief-map mechanism.
minor comments (6)
  1. [Sec. 4.2, text after Table 1] The sentence 'on HM3D, we observe a gain of +5.86% in SR and +0.66% in SPL' mixes relative changes against different baselines (InstructNav for SR and VLFM for SPL) without saying so; the abstract's 46.4% SPL improvement is also a relative change against InstructNav's SPL. Please state clearly which baseline each percentage is computed against and whether the change is absolute or relative.
  2. [Table 1 caption] The caption says 'BeliefMapNav can outperform previous SOTAs on both HM3D and MP3D benchmark,' but the table also includes HSSD results; please update the caption to mention all three benchmarks.
  3. [Sec. 4.3, Tables 1 and 2] The full model in Table 2 reports 62.5 SR / 31.6 SPL on 400 HM3D episodes, while Table 1 reports 61.4 SR / 30.6 SPL on the full HM3D validation set. The paper should explicitly note that the ablation uses a different, smaller subset and explain why the numbers differ; otherwise readers may see an inconsistency.
  4. [References and formatting] Several references are incomplete or malformed: 'ProcTHOR [?]', 'Cows [13]', '[27?]', and the parenthetical '(ZSGN)' in Appendix A.6. Please correct these and ensure all cited works appear in the reference list.
  5. [Abstract and Reproducibility] The abstract says 'The code is available on BeliefMapNav' but gives no URL or repository identifier. Please provide a working link or a stable anonymous repository for the review version.
  6. [Appendix A.7, Figure 8] The failure percentages listed in the text (48.32%, 36.78%, 12.82%) sum to 97.92%, leaving 2.08% unaccounted for; please clarify the exact percentage of the 'other' category or adjust the stated percentages.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the belief map and planner are built from external CLIP/LLM embeddings and evaluated against external benchmarks, not defined in terms of the reported SR/SPL outcomes.

full rationale

The paper's central derivation chain is an assembled navigation system, not a claimed first-principles prediction that reduces to its inputs. The belief map (Sec. 3.3) is constructed from CLIP image/text cosine similarities, LLM-generated landmark relevance scores, and hierarchical voxel features; the visibility map (Sec. 3.4) is derived from detection-confidence heuristics; and the planner (Sec. 3.5) minimizes an expected-distance objective using these belief values. None of these quantities is fitted to the reported success rate or SPL; the SR/SPL numbers in Table 1 are external benchmark measurements. The planner's objective is an algorithmic design choice, not a fitted parameter renamed as a prediction. The unnormalized 'belief' scores are not calibrated probabilities, and the SOTA claims lack error bars and significance tests, but those are correctness and statistical-support concerns, not definitional circularity. There is also no load-bearing self-citation chain: components such as CLIP, SAM, GPT-4, and the VLFM local planner are cited as external tools, and the paper does not invoke an author-derived uniqueness theorem to force its design. Consequently, no step meets the standard of being equivalent to its own input by construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The system's central claim rests on several hand-tuned weights, LLM-generated priors, an unnormalized fusion rule, and per-dataset tuning. No new physical entities are introduced; the belief map and visibility map are algorithmic data structures rather than postulated ontology.

free parameters (6)
  • Hierarchical scorer weights w1..w4 = w1=0.05, w2=0.1, w3=2, w4=0.01
    Hand-tuned in Sec. 4.1. These weights directly control which image scale is selected for each semantic level, so they shape the entire belief map.
  • w_unobserved = 0.01
    Secs. 3.4.2 and 4.1 assign this constant weight to every FOV voxel not yet in the map. It controls the exploration bonus in P_obs and is chosen by hand.
  • Visibility distance penalty alpha and detection range [d_min, d_max] = alpha unspecified; d range generated by GPT-4o via prompt
    Appendix A.4 defines C_d using alpha and an LLM-generated range. Alpha is never reported, and the range comes from a stochastic LLM prompt, so this part of the visibility map is not fully specified.
  • Local planner parameters = vary slightly by dataset
    Sec. 4.1 states local planner parameters vary by dataset, so benchmark comparisons include per-dataset tuning. The exact values are not reported.
  • Simulated annealing schedule = T0, Tf, cooling rate, N not reported
    Appendix A.5 says all parameters are tuned to balance quality and time, but gives no values. The schedule determines which frontier ordering the planner outputs.
  • Voxel resolution and map size = 0.25 m resolution, 45,000 voxels
    Sec. 4.1 fixes the granularity and spatial coverage of the belief map. These are system design choices that affect where belief can be represented.
assumptions (5)
  • domain assumption CLIP cosine similarity between landmark text and visual features is a valid proxy for spatial co-occurrence of target and landmark.
    Sec. 3.3.3 computes p^lt_u,i = max_ls cosine(...) and sums these into b_u without calibration. This assumes embedding similarity is proportional to presence likelihood.
  • domain assumption LLM-generated room, region, and object landmarks with per-level relevance scores provide accurate priors for target location in unseen homes.
    Sec. 3.3.2 and the prompt in Appendix A.3 put the entire prior on GPT-4o's commonsense outputs. No evidence is provided that these priors are calibrated or complete.
  • ad hoc to paper Multiplying prior belief by visibility confidence yields a valid posterior detection belief.
    Sec. 3.4.2 defines Bpost = p_v_u * b_u with no normalization, no likelihood model, and no derivation. This unnormalized product is the core fusion used by the planner and is introduced specifically for this paper.
  • domain assumption The cosine-squared angular and distance confidence model approximates real detector failure modes.
    Appendix A.4 defines C_p = C_d * C_horizontal * C_vertical. The assumed monotonic degradation near image edges and extreme depths is not validated against detector statistics.
  • domain assumption Benchmark evaluation protocols and baseline numbers are directly comparable across papers.
    Tables 1 through 5 compare numbers from different papers without released code, error bars, or a stated common protocol. Per-dataset local planner tuning makes this comparability an assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation." pith.science (2026). https://pith.science/paper/YGTSIZNN

@misc{pith2026250606487,
  author       = {Pith},
  title        = {Pith review of: BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YGTSIZNN}},
  note         = {Machine review of arXiv:2506.06487}
}
read the original abstract

Zero-shot object navigation (ZSON) allows robots to find target objects in unfamiliar environments using natural language instructions, without relying on pre-built maps or task-specific training. Recent general-purpose models, such as large language models (LLMs) and vision-language models (VLMs), equip agents with semantic reasoning abilities to estimate target object locations in a zero-shot manner. However, these models often greedily select the next goal without maintaining a global understanding of the environment and are fundamentally limited in the spatial reasoning necessary for effective navigation. To overcome these limitations, we propose a novel 3D voxel-based belief map that estimates the target's prior presence distribution within a voxelized 3D space. This approach enables agents to integrate semantic priors from LLMs and visual embeddings with hierarchical spatial structure, alongside real-time observations, to build a comprehensive 3D global posterior belief of the target's location. Building on this 3D voxel map, we introduce BeliefMapNav, an efficient navigation system with two key advantages: i) grounding LLM semantic reasoning within the 3D hierarchical semantics voxel space for precise target position estimation, and ii) integrating sequential path planning to enable efficient global navigation decisions. Experiments on HM3D, MP3D, and HSSD benchmarks show that BeliefMapNav achieves state-of-the-art (SOTA) Success Rate (SR) and Success weighted by Path Length (SPL), with a notable 46.4% SPL improvement over the previous best SR method, validating its effectiveness and efficiency.

Figures

Figures reproduced from arXiv: 2506.06487 by the authors.

Figure 1
Figure 1. The search process: BeliefMapNav plans frontier paths by minimizing the expected search [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. BeliefMapNav pipeline: The agent initializes with a 360 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The pipeline begins by passing the target and prompt to the LLM to generate hierarchical [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Visualization of the prior belief map, visibility map, and the posterior belief map, with an [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: (a) is the input depth image. (b) shows the confidence computed from the depth image. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Error analysis of the algorithm for parameter tuning. All the results are based on 10 [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Result demo of the algorithm. The chart on the left shows the relationship between the cost [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: The proportion of different causes of failure in the HM3D dataset. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Visualization of the search process. The color of each point in the image represents the [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Visualization of the search process. The color of each point in the image represents the [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Visualization of the search process. The color of each point in the image represents the [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Visualization of the prior belief map, visibility map, and the posterior belief map, with an [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HIMM: Human-Inspired Long-Term Memory Modeling for Embodied Exploration and Question Answering

    cs.RO 2026-02 conditional novelty 6.0 of 10

    An embodied-agent memory framework that disentangles episodic and semantic memories, retrieves past experiences via visual reasoning, and distills program-style rules achieves new state-of-the-art results on A-EQA and...

Reference graph

Works this paper leans on

58 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    Learning to explore using active neural slam.arXiv preprint arXiv:2004.05155, 2020

    Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhut- dinov. Learning to explore using active neural slam.arXiv preprint arXiv:2004.05155, 2020

    arXiv 2004

  2. [2]

    Object goal navigation using goal-oriented semantic exploration.Advances in Neural Information Processing Systems, 33:4247–4258, 2020

    Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Abhinav Gupta, and Russ R Salakhut- dinov. Object goal navigation using goal-oriented semantic exploration.Advances in Neural Information Processing Systems, 33:4247–4258, 2020

    2020

  3. [3]

    Esc: Exploration with soft commonsense constraints for zero-shot object navigation

    Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor, and Xin Eric Wang. Esc: Exploration with soft commonsense constraints for zero-shot object navigation. In International Conference on Machine Learning, pages 42829–42842. PMLR, 2023

    2023

  4. [4]

    Navgpt: Explicit reasoning in vision-and-language navigation with large language models

    Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: Explicit reasoning in vision-and-language navigation with large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7641–7649, 2024

    2024

  5. [5]

    React: Synergizing reasoning and acting in language models, 2023.URL https://arxiv

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023.URL https://arxiv. org/abs/2210.03629, 2023

    arXiv 2023

  6. [6]

    Ok-robot: What really matters in integrating open-knowledge models for robotics.arXiv preprint arXiv:2401.12202, 2024

    Peiqi Liu, Yaswanth Orru, Jay Vakil, Chris Paxton, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. Ok-robot: What really matters in integrating open-knowledge models for robotics.arXiv preprint arXiv:2401.12202, 2024

    arXiv 2024

  7. [7]

    Vlfm: Vision-language frontier maps for zero-shot semantic navigation

    Naoki Yokoyama, Sehoon Ha, Dhruv Batra, Jiuguang Wang, and Bernadette Bucher. Vlfm: Vision-language frontier maps for zero-shot semantic navigation. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 42–48. IEEE, 2024

    work page 2024

  8. [8]

    Gamap: Zero-shot object goal navigation with multi-scale geometric-affordance guidance.Advances in Neural Information Processing Systems, 37:39386–39408, 2024

    Hao Huang, Yu Hao, Congcong Wen, Anthony Tzes, Yi Fang, et al. Gamap: Zero-shot object goal navigation with multi-scale geometric-affordance guidance.Advances in Neural Information Processing Systems, 37:39386–39408, 2024

    work page 2024

Show all 58 references
  1. [9]

    Instructnav: Zero-shot system for generic instruction navigation in unexplored environment.arXiv preprint arXiv:2406.04882, 2024

    Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, and Hao Dong. Instructnav: Zero-shot system for generic instruction navigation in unexplored environment.arXiv preprint arXiv:2406.04882, 2024

    2024 arXiv

  2. [10]

    End-to-end navigation with vision language models: Transforming spatial reasoning into question-answering.arXiv preprint arXiv:2411.05755, 2024

    Dylan Goetting, Himanshu Gaurav Singh, and Antonio Loquercio. End-to-end navigation with vision language models: Transforming spatial reasoning into question-answering.arXiv preprint arXiv:2411.05755, 2024

    2024 arXiv

  3. [11]

    Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill

    Wenzhe Cai, Siyuan Huang, Guangran Cheng, Yuxing Long, Peng Gao, Changyin Sun, and Hao Dong. Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 5228–523...

    2024

  4. [12]

    Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation.Advances in Neural Information Processing Systems, 37:5285–5307, 2024

    Hang Yin, Xiuwei Xu, Zhenyu Wu, Jie Zhou, and Jiwen Lu. Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation.Advances in Neural Information Processing Systems, 37:5285–5307, 2024. 10

    2024

  5. [13]

    Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation

    Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, and Shuran Song. Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2317...

    2023

  6. [14]

    L3mvn: Leveraging large language models for visual target navigation

    Bangguo Yu, Hamidreza Kasaei, and Ming Cao. L3mvn: Leveraging large language models for visual target navigation. In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3554–3560. IEEE, 2023

    2023

  7. [15]

    Thinking in space: How multimodal large language models see, remember, and recall spaces.arXiv preprint arXiv:2412.14171, 2024

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces.arXiv preprint arXiv:2412.14171, 2024

    2024 arXiv

  8. [16]

    How to not train your dragon: Training-free embodied object goal navigation with semantic frontiers.arXiv preprint arXiv:2305.16925, 2023

    Junting Chen, Guohao Li, Suryansh Kumar, Bernard Ghanem, and Fisher Yu. How to not train your dragon: Training-free embodied object goal navigation with semantic frontiers.arXiv preprint arXiv:2305.16925, 2023

    2023 arXiv

  9. [17]

    Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai.arXiv preprint arXiv:2109.08238, 2021

    Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai.arXiv preprint ...

    2021 arXiv

  10. [18]

    Matterport3d: Learning from rgb-d data in indoor environments.arXiv preprint arXiv:1709.06158, 2017

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments.arXiv preprint arXiv:1709.06158, 2017

    2017 arXiv

  11. [19]

    Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navigation

    Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X Chang, and Manolis Savva. Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navigat...

    2024

  12. [20]

    Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames.arXiv preprint arXiv:1911.00357, 2019

    Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames.arXiv preprint arXiv:1911.00357, 2019

    1911 arXiv

  13. [21]

    Visual representations for semantic target driven navigation

    Arsalan Mousavian, Alexander Toshev, Marek Fišer, Jana Košecká, Ayzaan Wahid, and James Davidson. Visual representations for semantic target driven navigation. In2019 International Conference on Robotics and Automation (ICRA), pages 8846–8852. IEEE, 2019

    2019

  14. [22]

    Visual semantic navigation using scene priors.arXiv preprint arXiv:1810.06543, 2018

    Wei Yang, Xiaolong Wang, Ali Farhadi, Abhinav Gupta, and Roozbeh Mottaghi. Visual semantic navigation using scene priors.arXiv preprint arXiv:1810.06543, 2018

    2018 arXiv

  15. [23]

    Zson: Zero-shot object-goal navigation using multimodal goal embeddings.Advances in Neural Information Processing Systems, 35:32340–32352, 2022

    Arjun Majumdar, Gunjan Aggarwal, Bhavika Devnani, Judy Hoffman, and Dhruv Batra. Zson: Zero-shot object-goal navigation using multimodal goal embeddings.Advances in Neural Information Processing Systems, 35:32340–32352, 2022

    2022

  16. [24]

    Thda: Treasure hunt data augmentation for semantic navigation

    Oleksandr Maksymets, Vincent Cartillier, Aaron Gokaslan, Erik Wijmans, Wojciech Galuba, Stefan Lee, and Dhruv Batra. Thda: Treasure hunt data augmentation for semantic navigation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15374– 15383, 2021

    2021

  17. [25]

    Zero-shot object goal visual navigation

    Qianfan Zhao, Lu Zhang, Bin He, Hong Qiao, and Zhiyong Liu. Zero-shot object goal visual navigation. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 2025–2031. IEEE, 2023

    2025

  18. [26]

    V oronav: V oronoi-based zero-shot object navigation with large language model.arXiv preprint arXiv:2401.02695, 2024

    Pengying Wu, Yao Mu, Bingxian Wu, Yi Hou, Ji Ma, Shanghang Zhang, and Chang Liu. V oronav: V oronoi-based zero-shot object navigation with large language model.arXiv preprint arXiv:2401.02695, 2024

    2024 arXiv

  19. [27]

    Scene graph contrastive learning for embodied navigation

    Kunal Pratap Singh, Jordi Salvador, Luca Weihs, and Aniruddha Kembhavi. Scene graph contrastive learning for embodied navigation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10884–10894, 2023. 11

    2023

  20. [28]

    Bird’s-eye-view scene graph for vision- language navigation

    Rui Liu, Xiaohan Wang, Wenguan Wang, and Yi Yang. Bird’s-eye-view scene graph for vision- language navigation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10968–10980, 2023

    2023

  21. [29]

    Hi- erarchical representations and explicit memory: Learning effective navigation policies on 3d scene graphs using graph neural networks

    Zachary Ravichandran, Lisa Peng, Nathan Hughes, J Daniel Griffith, and Luca Carlone. Hi- erarchical representations and explicit memory: Learning effective navigation policies on 3d scene graphs using graph neural networks. In2022 International Conference on Robotics and Autom...

    2022

  22. [30]

    Human-like guidance by generating navigation using spatial-temporal scene graph

    Hayato Suzuki, Kota Shimomura, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi, Shota Okubo, Nanri Takuya, and Wang Siyuan. Human-like guidance by generating navigation using spatial-temporal scene graph. In2024 IEEE Intelligent Vehicles Symposium (IV), pages 1988–19...

    1988

  23. [31]

    Visual language maps for robot navigation

    Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps for robot navigation. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10608–10615. IEEE, 2023

    2023

  24. [32]

    Noisy few-shot 3d point cloud scene segmentation

    Hao Huang, Shuaihang Yuan, CongCong Wen, Yu Hao, and Yi Fang. Noisy few-shot 3d point cloud scene segmentation. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 11070–11077. IEEE, 2024

    2024

  25. [33]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  26. [34]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    2024 arXiv

  27. [35]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  28. [36]

    Seek: Semantic reasoning for object goal navigation in real world inspection tasks.arXiv preprint arXiv:2405.09822, 2024

    Muhammad Fadhil Ginting, Sung-Kyun Kim, David D Fan, Matteo Palieri, Mykel J Kochender- fer, and Ali-akbar Agha-Mohammadi. Seek: Semantic reasoning for object goal navigation in real world inspection tasks.arXiv preprint arXiv:2405.09822, 2024

    2024 arXiv

  29. [37]

    A formal basis for the heuristic determination of minimum cost paths.IEEE transactions on Systems Science and Cybernetics, 4(2):100–107, 1968

    Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths.IEEE transactions on Systems Science and Cybernetics, 4(2):100–107, 1968

    1968

  30. [38]

    Simulated annealing.Statistical science, 8(1):10–15, 1993

    Dimitris Bertsimas and John Tsitsiklis. Simulated annealing.Statistical science, 8(1):10–15, 1993

    1993

  31. [39]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. InEuropean Conference on Computer Vision, pages 38–55. Springer, 2024

    2024

  32. [40]

    Habitat-web: Learning embodied object-search strategies from human demonstrations at scale

    Ram Ramrakhya, Eric Undersander, Dhruv Batra, and Abhishek Das. Habitat-web: Learning embodied object-search strategies from human demonstrations at scale. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5173–5183, 2022

    2022

  33. [41]

    Offline visual representation learning for embodied navigation

    Karmesh Yadav, Ram Ramrakhya, Arjun Majumdar, Vincent-Pierre Berges, Sachit Kuhar, Dhruv Batra, Alexei Baevski, and Oleksandr Maksymets. Offline visual representation learning for embodied navigation. InWorkshop on Reincarnating Reinforcement Learning at ICLR 2023, 2023

    2023

  34. [42]

    Imagine before go: Self-supervised generative map for object goal navigation

    Sixian Zhang, Xinyao Yu, Xinhang Song, Xiaohan Wang, and Shuqiang Jiang. Imagine before go: Self-supervised generative map for object goal navigation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16414–16425, 2024. 12

    2024

  35. [43]

    Prioritized semantic learning for zero-shot instance navigation

    Xinyu Sun, Lizhao Liu, Hongyan Zhi, Ronghe Qiu, and Junwei Liang. Prioritized semantic learning for zero-shot instance navigation. InEuropean Conference on Computer Vision, pages 161–178. Springer, 2024

    2024

  36. [44]

    Imaginenav: Prompting vision-language models as embodied navigator through scene imagination.arXiv preprint arXiv:2410.09874, 2024

    Xinxin Zhao, Wenzhe Cai, Likun Tang, and Teng Wang. Imaginenav: Prompting vision-language models as embodied navigator through scene imagination.arXiv preprint arXiv:2410.09874, 2024

    2024 arXiv

  37. [45]

    Openfmnav: Towards open-set zero-shot object navigation via vision-language foundation models.arXiv preprint arXiv:2402.10670, 2024

    Yuxuan Kuang, Hai Lin, and Meng Jiang. Openfmnav: Towards open-set zero-shot object navigation via vision-language foundation models.arXiv preprint arXiv:2402.10670, 2024

    2024 arXiv

  38. [46]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation. InInternational conference on machine learning, pages 12888–12900. PMLR, 2022

    2022

  39. [47]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  40. [48]

    Wikido: A new benchmark evaluating cross-modal retrieval for vision-language models.Advances in Neural Information Processing Systems, 37:140812– 140827, 2024

    Pavan Kalyan Tankala, Piyush Pasi, Sahil Dharod, Azeem Motiwala, Preethi Jyothi, Aditi Chaudhary, and Krishna Srinivasan. Wikido: A new benchmark evaluating cross-modal retrieval for vision-language models.Advances in Neural Information Processing Systems, 37:140812– 140827, 2024

    2024

  41. [49]

    Cupy: A numpy-compatible library for nvidia gpu calculations.31st confernce on neural information processing systems, 151(7), 2017

    ROYUD Nishino and Shohei Hido Crissman Loomis. Cupy: A numpy-compatible library for nvidia gpu calculations.31st confernce on neural information processing systems, 151(7), 2017. 13 A Appendix A.1 Multi-scale feature extraction To better capture both global context and local d...

    2017

  42. [50]

    Each string should contain only one piece of information and avoid using "or" constructions

    Output three related strings for each level. Each string should contain only one piece of information and avoid using "or" constructions

  43. [51]

    Keep the phrases simple and common; avoid uncertain words like "maybe"

    Each piece of information should consist of only the most relevant phrases, not complete sentences. Keep the phrases simple and common; avoid uncertain words like "maybe"

  44. [52]

    For the landmarks string: The first dimension represents the information level, and the second dimension contains the three related strings for each level

    Output two two-dimensional lists. For the landmarks string: The first dimension represents the information level, and the second dimension contains the three related strings for each level. For the Probability: The first dimension represents the information level, and the seco...

  45. [53]

    In the generation of parameters dmin and dmax in the visibility map, the complete prompt is as follows: System: You are a helpful robot to find an object in an unknown environment

    Output only the string and Probability, do not include any additional text. In the generation of parameters dmin and dmax in the visibility map, the complete prompt is as follows: System: You are a helpful robot to find an object in an unknown environment. User: When we use YO...

  46. [54]

    Please output the distance in meters

  47. [55]

    A.4 Visibility map As shown in Fig 5, detection confidence depends on the pixel locations of the voxels in the image and the distance to the camera

    Please only output the distance range as a list: [distance_min, distance_max] , without any other text. A.4 Visibility map As shown in Fig 5, detection confidence depends on the pixel locations of the voxels in the image and the distance to the camera. Pixels near the image ce...

  48. [56]

    When the current temperature T is greater than the terminal temperatureT f , the algorithm will continue to run

    Initialization: Set the initial and terminal temperature T0 and Tf , the cooling rate α, and the number of samples to simulate N. When the current temperature T is greater than the terminal temperatureT f , the algorithm will continue to run

  49. [57]

    (1) swap: swap two points in the path

    Iterative Process: While the termination criterion is not met, for each sample, the algorithm will perform the following steps: (a) Generating Neighbor Solution: Generate a neighbor solution π′ from the current solutionπby applying three kinds of operations: swap, shift, or re...

  50. [58]

    The final output is the sample with the lowest cost

    Termination: The algorithm ends when the temperature T is less than the terminal tempera- tureT f . The final output is the sample with the lowest cost. The algorithm is implemented in Python and uses the CuPy [49] library to accelerate the process. All the parameters are tune...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.