Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

ForesightNav: Learning Scene Imagination for Efficient Exploration

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ForesightNav claims that a learned imagination module filling in unseen room geometry and semantics from partial maps lets a robot explore more efficiently, beating frontier-based baselines on object-goal navigation.

desk verdict PointNav speedup and the mask-generation trick are solid, but the ObjectNav SOTA claim rests on a 0.01 SPL margin under oracle perception. read the letter →

arxiv 2504.16062 v3 pith:KLMB35YH submitted 2025-04-22 cs.RO cs.CV

Hardik Shah , Jiaxu Xing , Nico Messikommer , Boyang Sun , Marc Pollefeys , Davide Scaramuzza This is my paper · ORCID
classification cs.ROcs.CV
keywords sceneimaginationobjectgoalnavigationexplorationoccupancypredictionsemanticmappingCLIPGeoSemMapclosed-loopevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ForesightNav tries to show that giving a navigating robot a learned 'imagination' module—a network that fills in the unobserved parts of a bird's-eye map of occupancy and CLIP semantic features—makes exploration of unseen indoor scenes more efficient. Rather than picking the nearest frontier each step, the agent predicts where walls and rooms probably are and where the requested object category likely sits, then plans a long-term goal toward that estimate. If true, this would mean robots can explore new buildings with fewer wasted steps and can answer open-vocabulary queries like 'a place for coffee' without retraining. The paper validates the idea on Structured3D, reporting 100% PointNav completion with about a 1.5x speedup and ObjectNav Success 0.73 / SPL 0.67, ahead of frontier-based state of the art.

What carries the argument

The GeoSem Map is a bird's-eye-view grid holding per-cell CLIP embeddings plus occupancy, updated from RGBD observations; the Imagination Module is a U-Net (or MAE-ViT) trained to regress the full map from the partial one. Supervision combines a cosine-similarity loss on CLIP features over interior cells, a weighted binary cross-entropy on occupancy, and a binary cross-entropy on a separately predicted interior/exterior mask. Goals are extracted by computing cosine similarity between the predicted map's interior cells and the CLIP text embedding of the goal query, then clustering candidates with DBSCAN and fitting a Gaussian Mixture Model to pick a robust navigation goal.

What would settle it

Re-run the ObjectNav evaluation on the same Structured3D scenes but build the input GeoSem Map from the actual observation pipeline described in the supplementary material—equirectangular RGBD, perspective projection, LSeg embeddings, top-down projection—and check whether ForesightNav still beats VLFM-CLIP; if the SPL gap disappears or reverses with realistic perception noise, the imagination advantage is an artifact of perfect maps.

Watch

Extended reading notes

Core claim

The paper claims that a learnable imagination module, which predicts the complete occupancy, interior mask, and CLIP semantic features of an indoor scene from a partially observed GeoSem Map, converts exploration from a reactive frontier chase into goal-directed search. On the Structured3D validation split this yields Success 0.73, SPL 0.67, and Distance to Goal 25.32 for ObjectNav, beating frontier-based baselines VLFM-CLIP, StructNav-Frontiers, Random, and Greedy; for PointNav the imagination agent reaches two waypoints with 100% completion in about 1.5 times fewer steps than a vanilla A* planner.

Load-bearing premise

The results assume the agent's egocentric observations are already converted into a near-perfect occupancy and semantic map, because the evaluation feeds the planner ground-truth GeoSem Maps rather than running the real RGBD-to-LSeg pipeline.

Editorial extensions

If this is right

  • A robot using imagined occupancy can shortcut around unseen walls, cutting PointNav steps by roughly 1.5x.
  • Because goal selection works in CLIP text space, the agent can chase arbitrary language targets without retraining or a fixed category set.
  • Predicting an interior mask prevents hallucinations outside walls from steering the goal, a necessary safeguard for open-vocabulary queries.
  • Modular imagination separates perception from planning, potentially easing transfer to real robots compared to end-to-end reinforcement learning policies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to perturb the input GeoSem Map with noise or incomplete observations in the closed-loop benchmark; the robustness of the SPL gain to such noise would separate the imagination contribution from the map quality.
  • The GMM goal selection could be ablated against a plain argmax of the similarity heatmap to isolate how much of the gain comes from imagination versus the clustering post-processing.
  • If the module generalizes across datasets, the same architecture could be trained on real floorplans and used for delivery or search-and-rescue drones, where a coarse predicted layout could save substantial flight time.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces ForesightNav, a modular exploration strategy for PointNav and ObjectNav. An Imagination Module is trained to predict a full GeoSem Map (occupancy plus CLIP semantic features) from a partially observed BEV map; the predicted map is used by an A* planner and by a cosine-similarity goal selection mechanism. Experiments on the Structured3D validation split show a PointNav speedup under ideal mapping (Table 1) and an ObjectNav SPL of 0.67 versus 0.66 for VLFM-CLIP (Table 2). The evaluation is a closed-loop 2D grid simulation with observations sampled from precomputed groundtruth GeoSem Maps and oracle goal detection.

Significance. The core idea—learning to hallucinate occupancy and language-aligned features for unexplored areas—is timely, and the PointNav result (100% completion, 439 vs 640 mean steps) gives a concrete, controlled demonstration that scene-completion priors can accelerate exploration under ideal perception. The modular design and the public code release are strengths. However, the ObjectNav SOTA claim is not yet supported end-to-end: the benchmark bypasses the perception and map-building stack, and the reported SPL margin over the strongest baseline is 0.01. The contribution is best regarded as an imagination-module study with an idealized observation model, pending end-to-end validation.

major comments (3)
  1. [§4 Evaluation Setup; Supplementary §7] The closed-loop evaluation simulates agent observations directly from precomputed groundtruth GeoSem Maps, and success is declared when the goal coordinate appears in the occupancy map, with the assumption that the agent detects the goal category. This bypasses the RGBD perception, LSeg embedding, depth projection, and map-update pipeline presented in Sections 3.1 and 3.3, as well as any object recognition errors. The measured SPL and completion gains therefore validate the Imagination Module under idealized perception rather than the full ForesightNav system. As a result, the claim in Section 4.2 that 'our proposed approach outperforms existing state-of-the-art approaches' is not established for the perception-action loop. Please provide experiments that execute the actual perception stack from RGBD inputs, or at minimum an analysis of how map noise and false-positive goal detections affect the navigation metrics.
  2. [§4.2, Table 2] The reported ObjectNav SPL advantage over VLFM-CLIP is 0.01 (0.67 vs 0.66), and no error bars, confidence intervals, or significance tests are provided. With a margin this small, the improvement could easily be within run-to-run variation. Please report per-category results, variance across the validation scenes, and a paired significance test over episodes, or soften the claim that ForesightNav outperforms SOTA.
  3. [§4.1 and Abstract] The abstract and Section 1 claim 'accurate occupancy prediction,' yet no direct quantitative evaluation of the predicted occupancy or semantic maps appears in the experiments. Table 1 measures navigation outcomes, which are an indirect and possibly threshold-dependent proxy. Reporting occupancy IoU or F1 and CLIP feature similarity on unobserved cells for the validation split would substantiate the 'imagination' claim and help disentangle map quality from planner behavior.
minor comments (5)
  1. [§3.2, Eq. (12)] The weighted BCE loss is denoted wBCE, but the weighting scheme w is not defined; please specify how class imbalance is accounted for.
  2. [§4.1, Table 1] The ViT-based imagination agents show lower completion rates than the vanilla agent (0.85 and 0.89 vs 0.99); a brief explanation of when imagination harms performance would help readers interpret the architecture comparison.
  3. [§3.3] The map update and imagination triggering condition is described as 'low overlap with the field of view at the last update' but no threshold value is given; please specify it, since the replanning frequency affects the navigation metrics.
  4. [§7.1] The phrase 'we loose the pixel-wise depth association' contains a typo; it should be 'lose'.
  5. [Table 2 caption] The dataset is referred to as 'Structure3D' in the caption but 'Structured3D' elsewhere; please make the naming consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reduction found; the imagination network is a learned function evaluated on a held-out split, and the oracle-perception setup is an evaluation-scope limitation, not a circular step.

full rationale

The paper's derivation chain is not circular. The Imagination Module f_imagine is a learned function (UNet or MAE) trained to map a partially observed GeoSem Map M to a complete GeoSem Map J (Sec. 3.2, Eqs. 8-13); training pairs (M,G) are generated by masking the ground-truth map G, and evaluation is on the Structured3D validation split, which is not used for training. The navigation results are behavioral outcomes: predicted occupancy (PointNav) or predicted CLIP similarity (ObjectNav) selects goals, but agent motion and success are evaluated against the actual environment via the ground-truth-based simulation (Eq. 11) and an explicit oracle goal-detection rule. No equation equates the prediction to the input by construction, and no fitted parameter is renamed as a prediction. The main caveat is that the closed-loop evaluation simulates observations directly from precomputed ground-truth GeoSem Maps and assumes oracle goal detection (Sec. 4 Evaluation Setup; Sec. 7), so perception errors are not measured; this is an evaluation-scope limitation, not a circular reduction. Self-citations are limited to an application example (Ref. [36]) and are not load-bearing. The GeoSem Map representation is adopted from prior external work (Ref. [21]); this is an ansatz, not a circular import. The SOTA claim is therefore supported as an exploration-strategy comparison under idealized perception, and the derivation itself is self-contained with respect to the paper's stated setup.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper introduces a learned map representation (GeoSem Map) and a network module (imagination module), but these are computational constructs, not new physical entities, forces, or conserved quantities. No new entity is postulated.

free parameters (6)
  • loss coefficients = lambda_CLIP=10, lambda_occ=1, lambda_interior=1
    Set by hand in Section 4 Network Details to balance the three losses; they affect the learned imagination module and thus all navigation results.
  • occupancy binarization threshold = 0.72
    Section 4 Network Details; used to convert predicted occupancy probabilities to a binary map for A* planning. No justification or sensitivity analysis is given.
  • height filter bounds = z_min=30%, z_max=70% of scene height
    Section 4 Network Details; determines which 3D points count for occupancy, asserting a drone agent at that height band.
  • occupancy density threshold = 10% of max density
    Section 3.1; classifies a grid cell as occupied if its point density exceeds this. Affects the GeoSem Map used for training and evaluation.
  • simulator sensor parameters = unspecified
    Section 3.2 and 4; FOV, sensor range, and turn angle are said to mimic real sensors but are never given values, yet they determine the partial observations and all navigation simulations.
  • goal extraction clustering parameters = unspecified
    Section 3.3; heatmap threshold, DBSCAN parameters, silhouette stop criterion, and GMM number of components are described qualitatively; exact values are not reported, though they directly determine the selected navigation goal.
assumptions (5)
  • domain assumption Simulated 2D grid-world observations from a groundtruth occupancy map faithfully represent real agent exploration and perception.
    All PointNav and ObjectNav results are produced in this grid simulation (Section 4 Evaluation Setup), so real-world transfer depends on this proxy.
  • domain assumption The agent perceives the world through the precomputed groundtruth GeoSem Map, bypassing RGBD-based perception and map building.
    Observations are simulated from the groundtruth GeoSem Map (Section 4.1, 4.2), so LSeg, depth projection, and odometry errors are excluded from the evaluation.
  • domain assumption Structured3D room polygon and wall annotations define an accurate navigability map.
    Groundtruth occupancy is derived from dataset annotations and pointcloud projection (Section 4, Supplementary Section 7.4); any floor plan errors transfer to training and evaluation.
  • domain assumption The target object is detected as soon as its grid cell becomes observed, i.e., a perfect object detector.
    Success in ObjectNav is defined by P[gx,gy] != 0.5, where P is the occupancy map (Section 4.2). This is an oracle detection assumption not stated as a limitation.
  • domain assumption CLIP cosine similarity between text queries and predicted map cells is a valid spatial prior for object location.
    The navigation goal is selected from the similarity heatmap of the imagined map against the text embedding (Section 3.3); this assumes the learned CLIP map encodes semantically meaningful spatial layout.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ForesightNav: Learning Scene Imagination for Efficient Exploration." pith.science (2026). https://pith.science/paper/KLMB35YH

@misc{pith2026250416062,
  author       = {Pith},
  title        = {Pith review of: ForesightNav: Learning Scene Imagination for Efficient Exploration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KLMB35YH}},
  note         = {Machine review of arXiv:2504.16062}
}
read the original abstract

Understanding how humans leverage prior knowledge to navigate unseen environments while making exploratory decisions is essential for developing autonomous robots with similar abilities. In this work, we propose ForesightNav, a novel exploration strategy inspired by human imagination and reasoning. Our approach equips robotic agents with the capability to predict contextual information, such as occupancy and semantic details, for unexplored regions. These predictions enable the robot to efficiently select meaningful long-term navigation goals, significantly enhancing exploration in unseen environments. We validate our imagination-based approach using the Structured3D dataset, demonstrating accurate occupancy prediction and superior performance in anticipating unseen scene geometry. Our experiments show that the imagination module improves exploration efficiency in unseen environments, achieving a 100% completion rate for PointNav and an SPL of 67% for ObjectNav on the Structured3D Validation split. These contributions demonstrate the power of imagination-driven reasoning for autonomous systems to enhance generalizable and efficient exploration.

Figures

Figures reproduced from arXiv: 2504.16062 by the authors.

Figure 1
Figure 1. ForesightNav proposes Imagination aided exploration in unknown environments. The past observations by the agent are used to reason about the scene and extract geometric and seman￾tic information. The agent uses this information to imagine the structure of the unobserved regions, enhancing its ability to ex￾plore efficiently. areas, predicting unseen regions based on prior knowledge to guide navigation and exploratio… view at source ↗
Figure 2
Figure 2. Overview of ForesightNav. At each timestep, we process the RGBD+Pose observations to update our current GeoSem Map, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Example of inputs, predictions, and the associated [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Visualization of a Structured3D scene: (a) Top-down [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 5
Figure 5. Figure 5: Visualization of query spatial distribution: (a) Raw sim [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: An example of querying an open-vocabulary language [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Paths taken by the vanilla agent and the imagine agent [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: GeoSem Map generation for the Structured3D Dataset (Equirectangular RGBD) [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Imaginative World Modeling with Scene Graphs for Embodied Agent Navigation

    cs.RO 2025-08 conditional novelty 6.0 of 10

    SGImagineNav uses an imagined hierarchical scene graph, filled in by an LLM, that guides a robot to unseen objects and achieves 65.4% and 66.8% success on HM3D and HSSD.

  2. Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigation

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A dual-branch text-imagination system, with one LLM branch for state estimation and one for candidate-direction description, improves R2R navigation success over prior LLM-based VLN methods.

Reference graph

Works this paper leans on

45 extracted references · 26 canonical work pages · cited by 2 Pith papers

  1. [1]

    On evaluation of embodied navigation agents

    Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, et al. On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757, 2018. 7, 8

    arXiv 2018

  2. [2]

    3d scene graph: A structure for unified semantics, 3d space, and cam- era

    Iro Armeni, Zhi-Yang He, JunYoung Gwak, Amir R Zamir, Martin Fischer, Jitendra Malik, and Silvio Savarese. 3d scene graph: A structure for unified semantics, 3d space, and cam- era. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 5664–5673, 2019. 8

    work page 2019

  3. [3]

    Objectnav revisited: On evaluation of embodied agents navigating to objects

    Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Olek- sandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans. Objectnav revisited: On evaluation of embodied agents navigating to objects. arXiv preprint arXiv:2006.13171, 2020. 2, 3

    arXiv 2006

  4. [4]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 3

    2021

  5. [5]

    Seman- tic visual navigation by watching youtube videos

    Matthew Chang, Arjun Gupta, and Saurabh Gupta. Seman- tic visual navigation by watching youtube videos. Advances in Neural Information Processing Systems , 33:4283–4294,

    work page

  6. [6]

    Object goal navi- gation using goal-oriented semantic exploration

    Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Ab- hinav Gupta, and Russ R Salakhutdinov. Object goal navi- gation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems , 33:4247–4258,

    work page

  7. [7]

    How to not train your dragon: Training-free embodied object goal navigation with semantic frontiers

    Junting Chen, Guohao Li, Suryansh Kumar, Bernard Ghanem, and Fisher Yu. How to not train your dragon: Training-free embodied object goal navigation with semantic frontiers. arXiv preprint arXiv:2305.16925, 2023. 2, 8

    arXiv 2023

  8. [8]

    Procthor: Large-scale embodied ai using procedural generation

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural generation. Ad- vances in Neural Information Processing Systems, 35:5982– 5994, 2022. 2

    2022

Show all 45 references
  1. [9]

    Can an embodied agent find your ”cat- shaped mug”? llm-based zero-shot object navigation

    Vishnu Sashank Dorbala, James F Mullen Jr, and Di- nesh Manocha. Can an embodied agent find your ”cat- shaped mug”? llm-based zero-shot object navigation. IEEE Robotics and Automation Letters, 2023. 2

    2023

  2. [10]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  3. [11]

    The cognitive map in humans: spatial navi- gation and beyond

    Russell A Epstein, Eva Zita Patai, Joshua B Julian, and Hugo J Spiers. The cognitive map in humans: spatial navi- gation and beyond. Nature neuroscience, 20(11):1504–1513,

  4. [12]

    Beyond the frontier: Pre- dicting unseen walls from occupancy grids by learning from floor plans

    Ludvig Ericson and Patric Jensfelt. Beyond the frontier: Pre- dicting unseen walls from occupancy grids by learning from floor plans. IEEE Robotics and Automation Letters, 2024. 2, 3

    2024

  5. [13]

    Cows on pasture: Base- lines and benchmarks for language-driven zero-shot object navigation

    Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, and Shuran Song. Cows on pasture: Base- lines and benchmarks for language-driven zero-shot object navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23...

    2023

  6. [14]

    Robohop: Segment-based topological map repre- sentation for open-world visual navigation

    Sourav Garg, Krishan Rana, Mehdi Hosseinzadeh, Lach- lan Mares, Niko S ¨underhauf, Feras Dayoub, and Ian Reid. Robohop: Segment-based topological map repre- sentation for open-world visual navigation. arXiv preprint arXiv:2405.05792, 2024. 2

    2024 arXiv

  7. [15]

    Learning to map for active semantic goal navigation

    Georgios Georgakis, Bernadette Bucher, Karl Schmeck- peper, Siddharth Singh, and Kostas Daniilidis. Learning to map for active semantic goal navigation. International Con- ference on Learning Representations, 2022. 2

    2022

  8. [16]

    Cross-modal map learning for vision and language navigation

    Georgios Georgakis, Karl Schmeckpeper, Karan Wanchoo, Soham Dan, Eleni Miltsakaki, Dan Roth, and Kostas Dani- ilidis. Cross-modal map learning for vision and language navigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 15460...

    2022

  9. [17]

    Navigating to objects in the real world

    Theophile Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, and Devendra Singh Chaplot. Navigating to objects in the real world. Science Robotics, 8(79):eadf6991, 2023. 3, 8

    2023

  10. [18]

    Object goal navigation using data regularized q- learning

    Nandiraju Gireesh, DA Sasi Kiran, Snehasis Banerjee, Mo- han Sridharan, Brojeshwar Bhowmick, and Madhava Kr- ishna. Object goal navigation using data regularized q- learning. In 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) , pages 1092–

    2022

  11. [19]

    Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning

    Qiao Gu, Alihusein Kuwajerwala, Sacha Morin, Kr- ishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, et al. Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. arXiv preprint arXiv:2309.16650,

    arXiv

  12. [20]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 7

    2022

  13. [21]

    Visual language maps for robot navigation

    Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps for robot navigation. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10608–10615. IEEE, 2023. 3, 4

    2023

  14. [22]

    Conceptfusion: Open-set multimodal 3d mapping

    Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd Omama, Tao Chen, Alaa Maalouf, Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, et al. Conceptfusion: Open-set multimodal 3d mapping. arXiv preprint arXiv:2302.07241, 2023. 3

    2023 arXiv

  15. [23]

    Uncertainty-aware occupancy map pre- diction using generative networks for robot navigation

    Kapil Katyal, Katie Popek, Chris Paxton, Phil Burlina, and Gregory D Hager. Uncertainty-aware occupancy map pre- diction using generative networks for robot navigation. In 2019 International Conference on Robotics and Automation (ICRA), pages 5453–5459. IEEE, 2019. 2, 3

    2019

  16. [24]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation. arXiv preprint arXiv:2201.03546, 2022. 2, 3

    2022 arXiv

  17. [25]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 8

    2023

  18. [26]

    Learning-augmented model-based planning for vi- sual exploration

    Yimeng Li, Arnab Debnath, Gregory J Stein, and Jana Koˇseck´a. Learning-augmented model-based planning for vi- sual exploration. In 2023 IEEE/RSJ International Confer- ence on Intelligent Robots and Systems (IROS), pages 5165–

    2023

  19. [27]

    Drone-surveillance for search and rescue in natural disaster

    Balmukund Mishra, Deepak Garg, Pratik Narang, and Vipul Mishra. Drone-surveillance for search and rescue in natural disaster. Computer Communications, 156:1–10, 2020. 1

    2020

  20. [28]

    Ippon: Common sense guided informative path planning for object goal navigation

    Kaixian Qu, Jie Tan, Tingnan Zhang, Fei Xia, Cesar Cadena, and Marco Hutter. Ippon: Common sense guided informative path planning for object goal navigation. 3rd Workshop on Language and Robot Learning: Language as an Interface ,

  21. [29]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  22. [30]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  23. [31]

    Pirlnav: Pretraining with imitation and rl finetuning for objectnav

    Ram Ramrakhya, Dhruv Batra, Erik Wijmans, and Abhishek Das. Pirlnav: Pretraining with imitation and rl finetuning for objectnav. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17896– 17906, 2023. 2

    2023

  24. [32]

    Sayplan: Ground- ing large language models using 3d scene graphs for scalable task planning

    Krishan Rana, Jesse Haviland, Sourav Garg, Jad Abou- Chakra, Ian Reid, and Niko Suenderhauf. Sayplan: Ground- ing large language models using 3d scene graphs for scalable task planning. In 7th Annual Conference on Robot Learning,

  25. [33]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  26. [34]

    Sc- explorer: Incremental 3d scene completion for safe and ef- ficient exploration mapping and planning

    Lukas Schmid, Mansoor Nasir Cheema, Victor Reijgwart, Roland Siegwart, Federico Tombari, and Cesar Cadena. Sc- explorer: Incremental 3d scene completion for safe and ef- ficient exploration mapping and planning. arXiv preprint arXiv:2208.08307, 2022. 2

    2022 arXiv

  27. [35]

    Pop-3d: Open-vocabulary 3d occupancy prediction from im- ages

    Antonin V obecky, Oriane Sim ´eoni, David Hurych, Spyri- don Gidaris, Andrei Bursuc, Patrick P ´erez, and Josef Sivic. Pop-3d: Open-vocabulary 3d occupancy prediction from im- ages. In Advances in Neural Information Processing Sys- tems, pages 50545–50557. Curran Associates, I...

    2023

  28. [36]

    Autonomous power line inspection with drones via perception-aware mpc

    Jiaxu Xing, Giovanni Cioffi, Javier Hidalgo-Carri ´o, and Da- vide Scaramuzza. Autonomous power line inspection with drones via perception-aware mpc. In 2023 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS), pages 1086–1093. IEEE, 2023. 1

    2023

  29. [37]

    A frontier-based approach for autonomous exploration

    Brian Yamauchi. A frontier-based approach for autonomous exploration. In Proceedings 1997 IEEE International Sym- posium on Computational Intelligence in Robotics and Au- tomation CIRA’97. ’Towards New Computational Principles for Robotics and Automation’, pages 146–151. IEEE, 1997. 2

    1997

  30. [38]

    Auxiliary tasks and exploration enable objectnav

    Joel Ye, Dhruv Batra, Abhishek Das, and Erik Wijmans. Auxiliary tasks and exploration enable objectnav. arXiv preprint arXiv:2104.04112, 2021. 2

    2021 arXiv

  31. [39]

    Vlfm: Vision- language frontier maps for zero-shot semantic navigation

    Naoki Harrison Yokoyama, Sehoon Ha, Dhruv Batra, Ji- uguang Wang, and Bernadette Bucher. Vlfm: Vision- language frontier maps for zero-shot semantic navigation. In 2nd Workshop on Language and Robot Learning: Language as Grounding, 2023. 2, 8

    2023

  32. [40]

    Poliformer: Scaling on-policy rl with transformers results in masterful navigators

    Kuo-Hao Zeng, Zichen Zhang, Kiana Ehsani, Rose Hendrix, Jordi Salvador, Alvaro Herrasti, Ross Girshick, Aniruddha Kembhavi, and Luca Weihs. Poliformer: Scaling on-policy rl with transformers results in masterful navigators. Confer- ence on Robot Learning, 2024. 2

    2024

  33. [41]

    Tag map: A text-based map for spatial reason- ing and navigation with large language models

    Mike Zhang, Kaixian Qu, Vaishakh Patil, Cesar Cadena, and Marco Hutter. Tag map: A text-based map for spatial reason- ing and navigation with large language models. Conference on Robot Learning, 2024. 2

    2024

  34. [42]

    Imagine before go: Self-supervised generative map for object goal navigation

    Sixian Zhang, Xinyao Yu, Xinhang Song, Xiaohan Wang, and Shuqiang Jiang. Imagine before go: Self-supervised generative map for object goal navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 16414–16425, 2024. 2, 3

    2024

  35. [43]

    Structured3d: A large photo-realistic dataset for structured 3d modeling

    Jia Zheng, Junfei Zhang, Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou. Structured3d: A large photo-realistic dataset for structured 3d modeling. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part IX 16 , pages 519–5...

    2020

  36. [44]

    Esc: Ex- ploration with soft commonsense constraints for zero-shot object navigation

    Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor, and Xin Eric Wang. Esc: Ex- ploration with soft commonsense constraints for zero-shot object navigation. In International Conference on Machine Learning, pages 42829–42842. PMLR, 2023. 2 ForesightNa...

    2023

  37. [45]

    It con- tains 3,500 scenes in total, split into 3,000 for training and 500 for validation

    GeoSem Maps from Equirectangular RGBD + Pose The Structured3D dataset [43] provides richly annotated 3D indoor scenes with panoramic RGBD observations. It con- tains 3,500 scenes in total, split into 3,000 for training and 500 for validation. Each scene Si is composed of multi...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.