Pith. sign in

REVIEW 5 major objections 5 minor 42 references

SD-OVON: A Semantics-aware Dataset and Benchmark Generation Pipeline for Open-Vocabulary Object Navigation in Dynamic Scenes

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a pipeline combining vision-language models and large language models can turn static 3D scans into many semantically plausible dynamic scenes, and supplies roughly 3,000 and 10,000 object-navigation episodes from…

desk verdict SD-OVON is a real infrastructure contribution — a public pipeline for dynamic-scene OVON benchmarks from real scans — but the semantic-realism claim is unvalidated and the main evaluation is partly self-referential. read the letter →

arxiv 2505.18881 v1 pith:XGBB4BZV submitted 2025-05-24 cs.CV cs.AIcs.RO

Dicong Qiu , Jiadi You , Zeying Gong , Ronghe Qiu , Hui Xiong , Junwei Liang This is my paper · ORCID
classification cs.CVcs.AIcs.RO
keywords SD-OVONopen-vocabularyobjectnavigationdynamicscenesscenevariantgenerationsemanticcommonsenseplacementbenchmarkembodiedAIdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that static scans of real rooms can be repopulated again and again with movable objects placed where daily commonsense puts them, turning each scan into many dynamic scenes for training and testing robots that search for objects by name. It contributes a procedural pipeline for that repopulation, two datasets with about 3,000 and 10,000 navigation episodes, and an evaluation in which its own semantics-aware baseline outperforms existing open-vocabulary navigation methods on the smaller dataset. The point is to replace the static-world assumption of earlier benchmarks with scenes that change between episodes, as real rooms do.

What carries the argument

The load-bearing mechanism is the joint relevance score $P(l_{\text{obj}} \mid l_{\text{rgn}}, l_{\text{rec}}) = P(l_{\text{obj}} \mid l_{\text{rgn}}) P(l_{\text{obj}} \mid l_{\text{rec}})$, where both factors are large-language-model-produced scores for how likely an object category appears in a room region and on a receptacle type; this score controls which objects get placed where and, in the Semantic A* baseline, which receptacles the agent visits first. The pipeline around it uses Gaussian-filtered observation sampling, open-vocabulary detection reprojected into fused 3D instances, an EM plane-detection algorithm to locate receptacle surfaces, uniform sampling on convex polygons, and physics simulation so objects settle stably. Receptacles here mean flat furniture surfaces such as tables, beds, shelves, and desks.

What would settle it

Replace the relevance scores with random scores in both the scene generator and the Semantic A* baseline; if navigation success and scene plausibility do not change, the semantic mechanism is not doing the work. A human-plausibility rating study on the same object-region-receptacle triples would settle whether the scenes actually adhere to daily commonsense.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that static scans used by prior navigation benchmarks can be converted into many dynamic variants: after extracting and fusing open-vocabulary instances into 3D, identifying receptacle planes, and inferring room-region semantics, movable objects are placed according to large-language-model-estimated relevance among object, room, and receptacle. The generated datasets contain 2,897 and 10,629 simulator-ready episodes built from 363 and 1,127 scene variants, drawn from 2,703 scanned scenes and 889 object models. On SD-OVON-3k, the paper's semantics-aware baseline reaches a success rate of 14.17% and an SPL of 6.47%, outperforming the compared zero-shot baselines; the paper interprets this as evidence that dynamic, semantics-consistent scenes form a valid and harder benchmark and that remembering region-receptacle semantics helps an agent find small daily objects.

Load-bearing premise

The generated scenes are semantically realistic only if the large language model's relevance scores for how likely an object belongs in a region and on a receptacle match real-world daily commonsense; if those scores are biased, the scenes are just arbitrary arrangements and the claimed advantage over static benchmarks collapses.

Editorial extensions

If this is right

  • Open-vocabulary object navigation (ObjectNav) training and evaluation can move from static scans to scenes that differ between episodes, since each scan yields many variants with movable objects.
  • Data scale for embodied navigation is no longer bounded by the number of captured environments; the same scan can generate arbitrarily many unique task episodes.
  • The released roughly 3,000- and 10,000-episode datasets give researchers a ready-to-use benchmark for open-vocabulary navigation in dynamic scenes.
  • The reported results imply that remembering region and receptacle semantics from a prior visit improves navigation, since the Semantic A* baseline outperforms the random-receptacle baseline and both outperform the compared baselines.
  • Because the generated scenes contain movable objects, the same scene variants can support open-vocabulary mobile manipulation tasks as well.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not check its placement scores against human ratings; a study asking people whether the generated object-region-receptacle combinations are plausible would directly test the realism claim and would also reveal whether the benchmark rewards semantic understanding or just object detection.
  • If the relevance scores are trustworthy, the generator can create controlled difficulty by placing objects in low-relevance regions, giving a stress test for whether agents genuinely use semantics rather than spatial priors.
  • The dynamic-scene setup implies that an agent's memory across episodes is a learnable asset; end-to-end policies that ingest previous episodes' semantic maps are a natural next step beyond the hand-designed A* baselines.
  • The pipeline still depends on existing object models; integrating generative 3D-asset models, as the paper's discussion notes, would let the same machinery create scenes with entirely new object categories.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents SD-OVON, a procedural pipeline that generates dynamic scene variants for open-vocabulary object navigation (OVON) by placing manipulable 3D objects into static scans of real environments using LLM/VLM-derived semantics. It contributes two datasets, SD-OVON-3k and SD-OVON-10k, two baseline agents (Random A* and Semantic A*), and an evaluation showing that the proposed baselines outperform VLFM and L3MVN on SD-OVON-3k. The abstract's central claim is that the pipeline yields an infinite number of photo-realistic scene variants that adhere to real-world semantics and daily commonsense, thereby overcoming the static-scene limitation of HM3D, ScanNet, and MP3D.

Significance. If validated, SD-OVON would be a valuable resource for OVON research because it targets dynamic scenes with movable objects, builds on real scans and real/synthetic object models, and is released with code and datasets. The paper also includes a useful diagnostic result: replacing detectors with a ground-truth detector dramatically improves performance (Table 4), highlighting the importance of detection quality in OVON. However, the central significance rests on the unverified assumption that LLM-generated object-region-receptacle relevance scores are a faithful model of daily commonsense. Because the same scores are used to generate scenes and to guide Semantic A* during evaluation, the reported advantages could be self-consistency effects rather than evidence of semantic realism or benchmark difficulty. No human validation, real-placement comparison, or random-placement ablation is provided.

major comments (5)
  1. [Section 3.4, Equation (5)] The joint relevance is printed as P(lobj|lrgn,lrec) = P(lobj|lrgn)P(lobj|lrgn). The second factor repeats P(lobj|lrgn); if the intended form is P(lobj|lrgn)P(lobj|lrec), then the equality silently assumes conditional independence of region and receptacle given the object. That assumption is not argued and is questionable for correlated categories (e.g., beds are strongly tied to bedrooms). Because these scores directly determine object placement in scene generation and receptacle prioritization in Semantic A* (Algorithm 4), the equation must be corrected and the independence assumption either justified or replaced with a joint scoring scheme.
  2. [Section 3.4 and Table 3] The load-bearing claim that generated scenes 'adhere to real-world semantics and daily commonsense' is asserted but never validated. There is no human plausibility study, no comparison against real object-placement statistics, and no ablation that places objects randomly or uniformly while keeping all else fixed. Since Semantic A* uses the same LLM relevance scores that generated the scenes, its higher success rate in Table 3 is partly a self-consistency result; it cannot by itself establish either semantic realism of the benchmark or the effectiveness of the semantic prioritization. I recommend adding a human evaluation of object-placement plausibility and a random-placement control in the generated scenes.
  3. [Section 5.1 and Table 3] The comparison between the proposed baselines and the state-of-the-art methods is confounded by an information asymmetry. Random A* and Semantic A* receive a full prior exploration of the environment and build receptacle and navigation-point memories during the first visit, while VLFM and L3MVN are evaluated without such prior knowledge. The reported performance gap (e.g., Semantic A* SR 0.1417 vs VLFM 0.0204) therefore conflates the benefit of prior maps with the benefit of semantic prioritization. Please adapt VLFM and L3MVN to the same prior-knowledge setting, or report versions of Random A* and Semantic A* that also operate in a zero-shot exploration setting.
  4. [Section 3.2, Equation (3)] The definitions of semantic similarity and geometric proximity appear to be swapped: phi_sem is defined as point-cloud overlap and phi_geo as the normalized cosine distance between semantic labels. Taken literally, this would fuse instances mainly by geometric overlap while treating label distance as derived from geometry, which would corrupt the instance-fusion stage on which the rest of the pipeline depends. Please correct the definitions and confirm that the released implementation matches the corrected text.
  5. [Abstract and Section 4] The claims of 'infinite unique photo-realistic scene variants' and enhancement of 'realism' are not directly evaluated. The paper reports the number of scenes and episodes (Table 2) but no visual-realism metric, no human realism rating, and no diversity measure. Please add quantitative evidence (e.g., human ratings, FID or similar perceptual metrics, or a downstream real-to-sim transfer study) or moderate the claims, since the abstract's central promise is currently unsupported.
minor comments (5)
  1. [Section 6] The sentence 'the it does not considered placing objects inside articulated receptacles' is ungrammatical; it should read 'it does not consider placing objects inside articulated receptacles.'
  2. [Section 5.2] 'fianl distance to goal' should be 'final distance to goal', and the same typo appears in Tables 4 and 6.
  3. [Section 2.2 and Appendix A.2] The text states ScanNet contains 1500 static scenes, while Table 8 lists 1613; these numbers should be reconciled.
  4. [Appendix D.2, Algorithm 4] Line 8 writes 'argmax P(g|lrgn,lrec)', but Section 3.4 defines the relevance as P(lobj|lrgn,lrec); please harmonize the notation so that the goal object category is consistently denoted.
  5. [Appendix A.3] The text in Section 4 says '13k ObjectNav task episodes' and '1.5k scene variants', while the sum of the two datasets is 13,526 episodes and 1,490 variants; please reconcile the rounded values for consistency.

Circularity Check

1 steps flagged · score 6.0 of 10

Semantic A*'s benchmark advantage is partly self-fulfilling: it is initialized with the same LLM relevance scores (Eq. 5) used to place goal objects in the SD-OVON episodes on which it is evaluated.

  1. self definitional [Section 3.4 (Eq. 5), Section 5.1 (Semantic A*), Algorithm 4]
    "The joint object-region-receptacle relevance P(lobj|lrgn,lrec) =P(lobj|lrgn)P(lobj|lrgn) (5) is then derived to determine the appearance probabilities of manipulable objects across receptacles and regions during object placement. ... Furthermore, Semantic A* prioritizes the receptacles to visit by decreasing order of relevance scores of the target object to region-receptacle pairs following equation 5."

    The same function P(lobj|lrgn,lrec) drives both scene generation and Semantic A*'s navigation policy. Section 3.4 uses the joint relevance to sample where manipulable object categories are placed on receptacles in each region, so the goal objects in the generated episodes are positioned according to this distribution. Section 5.1 then builds Semantic A* with the same region-receptacle semantics inference and ranks receptacles by argmax P(g|lrgn,lrec), handing the agent the exact placement prior that generated the benchmark, while VLFM and L3MVN must estimate semantics from observations.

full rationale

The main circular element is the closed loop between the LLM relevance model used to synthesize scenes (Section 3.4, Eq. 5) and the Semantic A* baseline evaluated on those scenes (Section 5.1, Algorithm 4). Because Semantic A* is built by the same instance-fusion and region-receptacle semantic inference described in Sections 3.2-3.4 and ranks receptacles by the same joint score, its advantage over VLFM and L3MVN is expected even if the LLM's placement scores are idiosyncratic rather than real-world commonsense. This is a genuine partial circularity. The paper is not wholly circular: the datasets are anchored to real-scanned scenes and curated object models, Random A* and the GT-detector ablations give some independent evidence about task solvability, and VLFM/L3MVN are external baselines. The comparison is also complicated by the fact that Random A* and Semantic A* receive a prior exploration phase and known receptacle locations, unlike VLFM and L3MVN; that is an evaluation-fairness concern rather than a circularity pattern. I did not count the absence of human validation of the semantic scores as circularity, since that is a validity concern rather than a reduction to the paper's own inputs. The apparent typo in Eq. 5 (P(lobj|lrgn)P(lobj|lrgn)) makes the exact joint distribution unrecoverable from the text, but this affects reproducibility more than circularity. Overall score 6: one central evaluation result reduces by construction to the shared generative model, while other components retain independent content.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The central claims rest less on a formal derivation than on the correctness of several perceptual and semantic models (G-SAM, LLM prompts, physics simulation) and on the assumption that self-generated episodes are a valid benchmark. These are reasonable engineering assumptions but are not independently validated beyond the paper's own experiments.

free parameters (5)
  • ksem = 0.4
    Hand-set weight for semantic similarity in instance fusion Equation 3; not derived and not swept.
  • kgeo = 1.6
    Hand-set weight for geometric similarity in instance fusion Equation 3; affects which instances merge across views.
  • phi_min = 0.8
    Threshold for accepting a fused instance match in Equation 3; directly controls instance quality.
  • IoUmin = 0.9
    Threshold for treating fused instances as significantly overlapping in the fusion error correction step; no sensitivity analysis.
  • hspawn = 0.3 m
    Height above the receptacle plane at which objects are spawned before physics simulation; arbitrary but influences placement plausibility.
assumptions (7)
  • domain assumption Gaussian coverage model with threshold 0.5 accurately captures which observations are sufficiently informative for object extraction.
    Used in Section 3.1 to decide where to sample observations; if coverage is mis-specified, instances and receptacles can be missed.
  • domain assumption Grounded SAM, image-tagging models, and GroundingDINO provide sufficiently accurate open-vocabulary masks and labels for instance extraction, fusion, and goal detection.
    Invoked in Sections 3.2 and 5.1; all downstream stages depend on detector quality, acknowledged only indirectly in the GT-detector ablation.
  • domain assumption LLM-generated region proposals and relevance scores P(lobj|lrgn) and P(lobj|lrec) reflect real-world daily commonsense well enough for semantically plausible placement.
    Core semantic realism premise; Section 3.4 and Appendix C; no human evaluation or comparison to alternative commonsense sources.
  • ad hoc to paper The joint relevance can be factorized as a product of object-region and object-receptacle relevance (Equation 5), i.e., conditional independence of region and receptacle given object.
    Equation 5 as printed even repeats the same factor; the factorization is asserted without justification.
  • domain assumption Physics simulation of free-falling objects onto receptacle planes produces physically and visually plausible arrangements.
    Used in Section 3.4 to ensure placement stability; no collision or penetration metrics are reported.
  • domain assumption Performance on episodes generated by the same pipeline is a valid proxy for open-vocabulary object navigation in real dynamic environments.
    Underlies the effectiveness claims in Section 5 and the conclusion; no sim-to-real transfer experiment is run.
  • standard math Uniform sampling in a triangle via barycentric coordinates and linear transformations preserves the uniform distribution.
    Appendix B provides a proof; standard geometric probability result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SD-OVON: A Semantics-aware Dataset and Benchmark Generation Pipeline for Open-Vocabulary Object Navigation in Dynamic Scenes." pith.science (2026). https://pith.science/paper/XGBB4BZV

@misc{pith2026250518881,
  author       = {Pith},
  title        = {Pith review of: SD-OVON: A Semantics-aware Dataset and Benchmark Generation Pipeline for Open-Vocabulary Object Navigation in Dynamic Scenes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XGBB4BZV}},
  note         = {Machine review of arXiv:2505.18881}
}
read the original abstract

We present the Semantics-aware Dataset and Benchmark Generation Pipeline for Open-vocabulary Object Navigation in Dynamic Scenes (SD-OVON). It utilizes pretraining multimodal foundation models to generate infinite unique photo-realistic scene variants that adhere to real-world semantics and daily commonsense for the training and the evaluation of navigation agents, accompanied with a plugin for generating object navigation task episodes compatible to the Habitat simulator. In addition, we offer two pre-generated object navigation task datasets, SD-OVON-3k and SD-OVON-10k, comprising respectively about 3k and 10k episodes of the open-vocabulary object navigation task, derived from the SD-OVON-Scenes dataset with 2.5k photo-realistic scans of real-world environments and the SD-OVON-Objects dataset with 0.9k manually inspected scanned and artist-created manipulatable object models. Unlike prior datasets limited to static environments, SD-OVON covers dynamic scenes and manipulatable objects, facilitating both real-to-sim and sim-to-real robotic applications. This approach enhances the realism of navigation tasks, the training and the evaluation of open-vocabulary object navigation agents in complex settings. To demonstrate the effectiveness of our pipeline and datasets, we propose two baselines and evaluate them along with state-of-the-art baselines on SD-OVON-3k. The datasets, benchmark and source code are publicly available.

Figures

Figures reproduced from arXiv: 2505.18881 by the authors.

Figure 1
Figure 1. Visualization of example scene variants generated by SD-OVON. Manipulatable objects [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An illustrating of the SD-OVON pipeline. It (a) randomly samples RGB-D observations [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Example trajectories of successful navigation with [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The complete statistics of object category appearance frequencies across the 363 scene [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: The complete statistics of the navigation goal object category appearance frequencies across [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: The complete statistics of object category appearance frequencies across the 1127 scene [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: The complete statistics of the navigation goal object category appearance frequencies across [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 20 canonical work pages

  1. [1]

    On evaluation of embodied navigation agents

    Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, et al. On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757, 2018

    arXiv 2018

  2. [2]

    Hm3d-ovon: A dataset and benchmark for open-vocabulary object goal navigation

    Naoki Yokoyama, Ram Ramrakhya, Abhishek Das, Dhruv Batra, and Sehoon Ha. Hm3d-ovon: A dataset and benchmark for open-vocabulary object goal navigation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

    work page 2024

  3. [3]

    Habitat-matterport 3d dataset (HM3d): 1000 large-scale 3d environments for embodied AI

    Santhosh Kumar Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alexander Clegg, John M Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-matterport 3d dataset (HM3d): 1000 large-scale 3d environments for embodied AI. In Thirty-fifth Conference on Neural Informatio...

    arXiv 2021

  4. [4]

    Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017

    2017

  5. [5]

    Procthor: Large-scale embodied ai using procedural generation

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural generation. Advances in Neural Information Processing Systems, 35:5982–5994, 2022

    work page 2022

  6. [6]

    Habitat 2.0: Training home assistants to rearrange their habitat

    Andrew Szot, Alex Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir V ondrus, Sameer Dharur, Franziska Meier, Wojciech Galuba, Angel Chang, Zsolt Kira, Vladlen Koltun, Jitendra Malik, Manolis Savva, and Dhruv Batra. Habitat 2.0: Training home assist...

    2021

  7. [7]

    Towards open vocabulary learning: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(7):5092–5113, 2024

    Jianzong Wu, Xiangtai Li, Shilin Xu, Haobo Yuan, Henghui Ding, Yibo Yang, Xia Li, Jiangning Zhang, Yunhai Tong, Xudong Jiang, et al. Towards open vocabulary learning: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(7):5092–5113, 2024

    work page 2024

  8. [8]

    Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments

    Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3674–3683, 2018

    2018

Show all 42 references
  1. [9]

    Llama: Open and efficient foundation language models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language...

    2023

  2. [10]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  3. [11]

    Grounded sam: Assembling open-world models for diverse visual tasks, 2024

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024

    2024

  4. [12]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference on computer vis...

    2024

  5. [13]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  6. [14]

    Vlfm: Vision- language frontier maps for zero-shot semantic navigation

    Naoki Yokoyama, Sehoon Ha, Dhruv Batra, Jiuguang Wang, and Bernadette Bucher. Vlfm: Vision- language frontier maps for zero-shot semantic navigation. In International Conference on Robotics and Automation (ICRA), 2024

    2024

  7. [15]

    L3mvn: Leveraging large language models for visual target navigation

    Bangguo Yu, Hamidreza Kasaei, and Ming Cao. L3mvn: Leveraging large language models for visual target navigation. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3554–3560. IEEE, 2023. 11

    2023

  8. [16]

    Open-vocabulary mobile ma- nipulation in unseen dynamic environments with 3d semantic maps

    Dicong Qiu, Wenzong Ma, Zhenfu Pan, Hui Xiong, and Junwei Liang. Open-vocabulary mobile ma- nipulation in unseen dynamic environments with 3d semantic maps. arXiv preprint arXiv:2406.18115, 2024

    2024 arXiv

  9. [17]

    Auxiliary tasks and exploration enable objectgoal navigation

    Joel Ye, Dhruv Batra, Abhishek Das, and Erik Wijmans. Auxiliary tasks and exploration enable objectgoal navigation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 16117– 16126, 2021

    2021

  10. [18]

    3d-aware object goal navigation via simultaneous exploration and identification

    Jiazhao Zhang, Liu Dai, Fanpeng Meng, Qingnan Fan, Xuelin Chen, Kai Xu, and He Wang. 3d-aware object goal navigation via simultaneous exploration and identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6672–6682, 2023

    2023

  11. [19]

    Poni: Potential functions for objectgoal navigation with interaction-free learning

    Santhosh Kumar Ramakrishnan, Devendra Singh Chaplot, Ziad Al-Halah, Jitendra Malik, and Kristen Grauman. Poni: Potential functions for objectgoal navigation with interaction-free learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pa...

    2022

  12. [20]

    Object goal navigation using goal-oriented semantic exploration

    Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Abhinav Gupta, and Russ R Salakhutdinov. Object goal navigation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems, 33:4247–4258, 2020

    2020

  13. [21]

    Zson: Zero-shot object-goal navigation using multimodal goal embeddings

    Arjun Majumdar, Gunjan Aggarwal, Bhavika Devnani, Judy Hoffman, and Dhruv Batra. Zson: Zero-shot object-goal navigation using multimodal goal embeddings. Advances in Neural Information Processing Systems, 35:32340–32352, 2022

    2022

  14. [22]

    Habitat-web: Learning embodied object-search strategies from human demonstrations at scale

    Ram Ramrakhya, Eric Undersander, Dhruv Batra, and Abhishek Das. Habitat-web: Learning embodied object-search strategies from human demonstrations at scale. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5173–5183, 2022

    2022

  15. [23]

    Thda: Treasure hunt data augmentation for semantic navigation

    Oleksandr Maksymets, Vincent Cartillier, Aaron Gokaslan, Erik Wijmans, Wojciech Galuba, Stefan Lee, and Dhruv Batra. Thda: Treasure hunt data augmentation for semantic navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15374–15383, 2021

    2021

  16. [24]

    Simple but effective: Clip embeddings for embodied ai

    Apoorv Khandelwal, Luca Weihs, Roozbeh Mottaghi, and Aniruddha Kembhavi. Simple but effective: Clip embeddings for embodied ai. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14829–14838, 2022

    2022

  17. [25]

    Pirlnav: Pretraining with imitation and rl finetuning for objectnav

    Ram Ramrakhya, Dhruv Batra, Erik Wijmans, and Abhishek Das. Pirlnav: Pretraining with imitation and rl finetuning for objectnav. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17896–17906, 2023

    2023

  18. [26]

    Prioritized semantic learning for zero-shot instance navigation

    Xinyu Sun, Lizhao Liu, Hongyan Zhi, Ronghe Qiu, and Junwei Liang. Prioritized semantic learning for zero-shot instance navigation. In European Conference on Computer Vision, pages 161–178. Springer, 2024

    2024

  19. [27]

    Matterport3D: Learning from RGB-D data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3D: Learning from RGB-D data in indoor environments. International Conference on 3D Vision (3DV), 2017

    2017

  20. [28]

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler Gi...

    1906 arXiv

  21. [29]

    Open-set image tagging with multi-grained text supervision

    Xinyu Huang, Yi-Jie Huang, Youcai Zhang, Weiwei Tian, Rui Feng, Yuejie Zhang, Yanchun Xie, Yaqian Li, and Lei Zhang. Open-set image tagging with multi-grained text supervision. arXiv e-prints, pages arXiv–2310, 2023

    2023

  22. [30]

    Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning

    Qiao Gu, Ali Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, et al. Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. In 2024 IEEE International Conferenc...

    2024

  23. [31]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. 12

    2024 arXiv

  24. [32]

    The ycb object and model set: Towards common benchmarks for manipulation research

    Berk Calli, Arjun Singh, Aaron Walsman, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. The ycb object and model set: Towards common benchmarks for manipulation research. In 2015 international conference on advanced robotics (ICAR), pages 510–517. IEEE, 2015

    2015

  25. [33]

    Ai2-thor: An interactive 3d environment for visual ai

    Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474, 2017

    2017 arXiv

  26. [34]

    Abo: Dataset and benchmarks for real-world 3d object understanding

    Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, Matthieu Guillaumin, and Jitendra Malik. Abo: Dataset and benchmarks for real-world 3d object understanding. CVPR, 2022

    2022

  27. [35]

    Chang, and Manolis Savva

    Mukul Khanna*, Yongsen Mao*, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexan- der Clegg, Eric Undersander, Angel X. Chang, and Manolis Savva. Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Na...

    2023

  28. [36]

    Homerobot: Open-vocabulary mobile manipulation, 2024

    Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav, Austin Wang, Mukul Khanna, Theophile Gervet, Tsung-Yen Yang, Vidhi Jain, Alexander William Clegg, John Turner, Zsolt Kira, Manolis Savva, Angel Chang, Devendra Singh Chaplot, Dhruv Batra, Roozbeh Mottaghi, Yonatan Bisk, and ...

    2024

  29. [37]

    Frontier detection and reachability analysis for efficient 2d graph-slam based active exploration

    Zezhou Sun, Banghe Wu, Cheng-Zhong Xu, Sanjay E Sarma, Jian Yang, and Hui Kong. Frontier detection and reachability analysis for efficient 2d graph-slam based active exploration. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2051–205...

    2020

  30. [38]

    Autonomous exploration development environment and the planning algorithms

    Chao Cao, Hongbiao Zhu, Fan Yang, Yukun Xia, Howie Choset, Jean Oh, and Ji Zhang. Autonomous exploration development environment and the planning algorithms. In 2022 International Conference on Robotics and Automation (ICRA), pages 8921–8928. IEEE, 2022

    2022

  31. [39]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  32. [40]

    Grounding dino 1.5: Advance the "edge" of open-set object detection, 2024

    Tianhe Ren, Qing Jiang, Shilong Liu, Zhaoyang Zeng, Wenlong Liu, Han Gao, Hongjie Huang, Zhengyu Ma, Xiaoke Jiang, Yihao Chen, Yuda Xiong, Hao Zhang, Feng Li, Peijun Tang, Kent Yu, and Lei Zhang. Grounding dino 1.5: Advance the "edge" of open-set object detection, 2024

    2024

  33. [41]

    Google scanned objects: A high-quality dataset of 3d scanned household items

    Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned household items. In 2022 International Conference on Robotics and Automation (ICRA), pa...

    2022

  34. [42]

    objects": [object_1, object_2, ...]}, please find and return pieces of furnitures with surface to put daily objects on top of, such as

    Thus, the probability density function for this region is: fR(u,v) = 1 if u + v ≤ 1 and 0 ≤ u,v ≤ 1, 0 otherwise . For (1 − r1,1 − r2) when r1 + r2 > 1: R in this region takes the value (1 − r1,1 − r2). First, consider the random variable 1 − R1. Since 1 − R1 is defined by a l...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.