Pith. sign in

REVIEW 5 major objections 7 minor 3 cited by

InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction

T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper tries to make embodied AI scalable by building InfiniteWorld, a unified Isaac Sim platform for visual-language robot interaction with generative assets and two social benchmarks.

desk verdict A coherent Isaac Sim integration with two genuinely new task designs, but the flagship OWSMM benchmark is invalidated by its own coarse-map artifact and the code/data are not released; worth review only if that changes. read the letter →

arxiv 2412.05789 v1 pith:CZDYA2XH submitted 2024-12-08 cs.RO

Pengzhen Ren , Min Li , Zhen Luo , Xinshuai Song , Ziwei Chen , Weijia Liufu , Yixuan Yang , Hao Zheng
show 10 more authors
Rongtao Xu Zitong Huang Tongsheng Ding Luyang Xie Kaidong Zhang Changfei Fu Yang Liu Liang Lin Feng Zheng Xiaodan Liang
This is my paper · ORCID
classification cs.RO
keywords embodiedAIsimulationvisual-languagerobotinteractionIsaacSimgenerative3DassetsReal2Simreconstructionscenegraphexplorationsocialmobilemanipulationbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Embodied AI research is currently scattered across many simulators with incompatible assets, so progress toward scaling laws is slower than it could be. This paper tries to fix that with InfiniteWorld, a simulator built on Nvidia Isaac Sim that gathers generative 3D asset creation, a depth-regularized Real2Sim reconstruction pipeline, an automated annotation tool, and converted public datasets into one platform. It also adds four benchmarks, including scene graph collaborative exploration and open-world social mobile manipulation with hierarchical and horizontal interaction. The paper reports that its platform makes navigation and manipulation tractable for LLM-based agents using its interfaces, while VLM zero-shot control fails and the social manipulation benchmark currently scores zero because the semantic maps it inherits are too coarse.

What carries the argument

The load-bearing object is the unified Isaac Sim–based asset-and-interaction pipeline, with the scene graph collaborative exploration benchmark feeding directly into the open-world social mobile manipulation benchmark. Three sub-machineries carry the asset claim: language-driven scene generation built on HOLODECK with 236 floor and wall texture replacements and 10K base scenes, a Real2Sim pipeline that adds depth and normal regularization losses to PGSR using Depth Pro estimates, and Annot8-3D, a web-based multistage point-cloud annotation framework with AI assistance and optional human-in-the-loop refinement.

What would settle it

Replace the coarse exploration maps in open-world social mobile manipulation with ground-truth or object-level scene graphs while keeping every other setting fixed. If the success rate stays at zero, the paper's stated explanation is wrong; if it rises, the mapping pipeline is confirmed as the bottleneck.

Watch

Extended reading notes

Core claim

On its own terms, InfiniteWorld is a unified and scalable simulator for visual-language robot interaction built on Nvidia Isaac Sim. The central claim is that one platform can remove the fragmentation slowing embodied-AI research by standardizing asset construction: language-driven scene generation, single-image object reconstruction, controllable articulation generation, a depth-and-normal-regularized Real2Sim pipeline, an automated annotation platform, and conversion of existing scene and object datasets into a common USD format. Alongside the asset machinery, the paper introduces four benchmarks, including scene graph collaborative exploration and open-world social mobile manipulation with two interaction modes: hierarchical, where an administrator agent has more environment knowledge, and horizontal, where equal-status agents exchange knowledge through dialogue. The reported results show LLM-based instruction following succeeding at 90.82% on object loco-navigation and 77.28% on loco-manipulation, near-zero VLM zero-shot performance, and a 0% success rate on social mobile manipulation that the paper attributes to coarse semantic maps inherited from the exploration benchmark.

Load-bearing premise

The results of the social manipulation benchmark assume that the semantic maps and scene graphs produced by the exploration benchmark are accurate enough to ground planning, and the paper itself states that these maps were often too coarse, so the zero scores may say more about mapping than about social interaction.

Editorial extensions

If this is right

  • A single simulator that converts HSSD, HM3D, Replica, ScanNet, 3D-Front, PartNet-Mobility, Objaverse, and ClothesNet assets into one format could let different embodied-AI groups share scenes and objects instead of rebuilding them.
  • The two new tasks put scene-graph construction and social planning on the evaluation agenda, not just navigation and manipulation.
  • The large gap between LLM-based action following and VLM zero-shot performance suggests that current vision-language models cannot directly control the robot without explicit navigation and manipulation interfaces.
  • Because open-world social mobile manipulation reuses scene graph collaborative exploration maps, benchmark results are coupled: exploration quality upper-bounds downstream social manipulation performance.
  • The reported zero rate on social mobile manipulation is not evidence that social interaction fails; it is evidence that the mapping pipeline must be made reliable before the benchmark can discriminate agent skill.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own diagnosis suggests a near-term extension it does not run: benchmark agents against ground-truth or object-level scene graphs in open-world social mobile manipulation, separating map error from planning and dialogue ability.
  • If accurate maps unlock nonzero scores, the hierarchical-versus-horizontal contrast could become a useful probe of how much dialogue helps under asymmetric knowledge, going beyond the paper's current all-zero comparison.
  • The scene generation claim is scalable only if the 236x style substituted scenes remain physically valid and reachable; a scene-level collision or reachability audit could test that validity directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper presents InfiniteWorld, a robotics simulation framework built on NVIDIA Isaac Sim, with four pillars: generative 3D asset construction (HOLODECK-based scene generation, TripoSR-based image-to-object reconstruction, CAGE articulation generation), a "depth-prior-constrained" Real2Sim pipeline built on PGSR, an AI-assisted point-cloud annotation tool (Annot8-3D), and a unified .usd asset interface over existing scene and object datasets. On the task side, the paper defines four benchmarks: object loco-navigation, loco-manipulation, scene-graph collaborative exploration (SGCE), and open-world social mobile manipulation (OWSMM) with hierarchical and horizontal interaction variants. Experiments with a Stretch robot show high success for LLM-based instruction following on the two navigation/manipulation benchmarks when scene semantics and built-in planning interfaces are supplied, near-zero success for VLM zero-shot agents, moderate exploration rates (roughly 25-32% SER) for SGCE, and 0% success in all OWSMM conditions, which the authors attribute in Section 4.5 to coarse semantic maps carried over from Benchmark 3.

Significance. The paper addresses a genuine community need, the fragmentation of simulation assets and interfaces, and packages a large amount of integration work into one platform; if released and validated, the SGCE and OWSMM task designs, particularly the distinction between hierarchical (administrator) and horizontal (peer) knowledge exchange, would be a useful addition to embodied-AI benchmarking. Credit is due for reporting the all-zero OWSMM results honestly rather than tuning the protocol until favorable numbers appear. However, the flagship OWSMM benchmark currently fails for reasons orthogonal to social interaction, no oracle or successful condition demonstrates that OWSMM tasks are solvable, the Real2Sim improvement is supported only qualitatively, and none of the core artifacts (code, scenes, assets, benchmark instances) is released; the significance is therefore conditional on substantial additional validation and on a full release.

major comments (5)
  1. [Section 4.1, Section 4.5, Table 5] All three OWSMM conditions report 0.00 success rate, and the manuscript itself locates the cause in the upstream mapping pipeline: Section 4.5 states that the Benchmark-3 maps were built from semantic information "often too coarse", so task objects were missing or misplaced in the maps. Under these conditions the zero results measure map quality, not social interaction ability, and because none of the three reported conditions actually exercises the administrator Q&A or the horizontal dialogue described in Section 4.1, the claimed evaluation of "communication" is unsupported. The paper also provides no oracle, scripted, or ground-truth-map condition showing that the OWSMM tasks are solvable at all, and it does not state which SGCE method produced the maps used in OWSMM. The benchmark therefore cannot yet support the paper's central claim of a comprehensive evaluation; this requires either a working exploration-to-map-to-manipulation pipeline with a positive control, or a re-scoping of the contribution.
  2. [Section 4.5, Tables 2-3] In every LLM-based row of Tables 2 and 3, SPL equals SR exactly (for example, 90.82 = 90.82 and 77.28 = 77.28), which implies that every successful episode achieved exactly the optimal path ratio of 1.0. Combined with the task-generation setting that combines HSSD scene semantics with built-in occupancy maps, D* Lite path following, and adhesion interfaces, this indicates that the LLM-based condition is executing a scripted, oracle-informed plan rather than performing perception-guided navigation; the comparison against the observation-only VLM zero-shot condition is therefore not a like-for-like evaluation of embodied agents. The paper should state precisely what information the LLM receives (in particular whether object coordinates are ground truth), and it should report episode counts, scene splits, and error bars, none of which are given for any table.
  3. [Section 3.2, Figure 2, Section 6.1] The claimed improvement to Real2Sim, adding Depth-Pro depth estimation and planar normal regularization to PGSR, is supported only by qualitative image comparisons in Figure 2; no quantitative reconstruction metrics (such as Chamfer distance, F-score, or PSNR) and no ablations isolating the two added loss terms are reported, and no loss formulation appears in the main text. Because the paper presents this as an "improved" pipeline, the claim is currently unverifiable; it should be backed by numbers and an ablation, or the wording should be softened to describe an adapted pipeline.
  4. [Section 4.1 Benchmark 3, Table 4] The SGCE results show very weak discrimination: the best method (Co-NavGPT with GPT-4) reaches SER 0.3209 against 0.3030 for Random and 0.2581 for the single semantic map, a gap that is meaningless without variance estimates or a significance test, and the reported MRMSE values of 5.78-7.74 m are far too large to ground the object positions needed by the downstream OWSMM tasks, which the paper itself concedes in Section 4.5. The benchmark protocol should specify the number of scenes, episodes, seeds, number of agents, and communication settings, and it should include a positive control verifying that the generated scene graphs are accurate enough to support subsequent task execution.
  5. [Section 3.1, Section 3.4, Abstract, GitHub link] The central deliverable is not available for scrutiny: the repository link is a placeholder, the claimed 10K/2.36M generated scenes and the unified asset conversions are described only by counts with no release, and the manuscript states that the constructed scenarios "will be published upon acceptance". For a platform and benchmark paper, verification of the central claims requires at minimum a full release of code and assets, or a detailed protocol (task instance lists, scene splits, episode counts, seeds) sufficient for independent reimplementation; as it stands, the reader cannot check that the simulator, the asset interfaces, or the benchmarks exist as described.
minor comments (7)
  1. [Reference [55], Table 6] Reference [55] cites REBOUND, an N-body simulation code, but Table 6 compares an annotation tool named ReBound; the citation is mismatched and should be corrected.
  2. [Section 3.1] The paper says scene count "can be easily expanded 236 times" through 236 floor and wall textures, while the abstract says "200+ different scene style changes"; the relationship between these numbers, the 10K scenes, and the claimed 2.36M total should be stated precisely and consistently.
  3. [Section 4.4, Table 5] The OWSMM metrics MPL and LPL are not defined (what constitutes an action path, and how are minimum and longest action paths computed), and the value MPL = 0.00 for the "VLM Explore+Act Prim" row is unexplained.
  4. [Appendix 6.3, Tables 7-8] Tables 7 and 8 report counts of the source datasets (Objaverse, HSSD, HM3D, etc.) rather than the number of assets actually converted to .usd, validated, and made interactive in Isaac Sim; the size of the implemented unified asset library should be quantified.
  5. [Section 4.1 Benchmark 3, Section 4.5] SGCE is described as a multi-robot collaborative task, but the experiments never specify the number of robots or the communication model (range, bandwidth, latency) used for map sharing, making the collaborative setting hard to reproduce.
  6. [Section 4.5] The qualitative remarks about Qwen's "stability" versus Chat-GLM4's "action accuracy", and about the prompt design explaining GPT-4's SGCE result, are not backed by quantitative evidence; please either quantify these claims or remove them.
  7. [Table 1] Table 1 spells the platform "InfinitedWorld" (missing 'i'); the spelling should be corrected to "InfiniteWorld".

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: InfiniteWorld's benchmarks and asset pipeline are presented as constructions, not derivations; the OWSMM zero-score issue is an internally flagged validation gap, not a circular step.

full rationale

The paper contains no derivation chain in the sense targeted by this pass: it reports a simulation framework, asset construction methods, and benchmark results, none of which are derived from a fitted parameter that is then renamed as a prediction. The four benchmarks are defined by the authors rather than predicted from prior equations, which is expected for a simulator paper. No fitted input is called a prediction: Tables 2–5 report measured success rates of external LLM/VLM baselines under fixed task definitions. The self-citations in the paper (e.g., Surfer [56] in related work, LLPlace [76] as a scene-layout reference) are background comparisons and are not load-bearing for any central claim. The relevant limitation is explicit in Section 4.5: for Open World Social Mobile Manipulation, the paper states that 'most of the maps were built using semantic information, which was often too coarse,' so task object instances were missing or mislocalized. This is an internally flagged benchmark-validity problem—the OWSMM results cannot yet support the claim of comprehensive social-interaction evaluation—but it is a dependency failure between Benchmark 3 outputs and Benchmark 4 inputs, not a circular reduction. Because no step equates an output to its own input by construction, and no load-bearing argument reduces to a self-citation, the circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

No new physical or abstract entities are introduced; the administrator role is a knowledge-level configuration rather than a new entity. The ledger instead captures the domain assumptions about asset validity and map accuracy on which the benchmark claims rest.

free parameters (2)
  • Object navigation success thresholds = 2 meters, 60 degrees
    Task success for locating a target is defined by distance < 2 m and within 60 degrees of horizontal field of view (Section 4.1); these hand-set thresholds determine the reported SR.
  • Maximum exploration steps for SGCE = 200
    Section 4.4 sets the maximum exploration steps to 200, which defines the Semantic Exploration Rate and directly affects the benchmark scores.
assumptions (5)
  • domain assumption Isaac Sim physics and rendering are adequate for embodied AI research
    The whole platform is built on Isaac Sim; no validation against real-world gaps or sim-to-real transfer is provided.
  • domain assumption HOLODECK language-driven scene generation produces valid, interactable 3D scenes
    Section 3.1 builds the 10K scene pipeline on HOLODECK; scene validity and physical interactability are asserted, not measured.
  • domain assumption Depth Pro monocular depth estimates are accurate enough to regularize PGSR reconstruction
    Section 3.2 adds depth and normal regularization using Depth Pro and the PGSR local plane assumption, with only qualitative results shown.
  • domain assumption GPT-4o generated task instructions match the HSSD scene semantics
    Section 4.2 uses GPT-4o with HSSD semantics to generate instructions; no validation of instruction correctness is reported.
  • domain assumption Semantic exploration maps from Benchmark 3 are accurate enough to ground social manipulation
    Section 4.5 attributes the 0% success in OWSMM to coarse semantic maps, implying the benchmark outcome depends on an unvalidated mapping assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction." pith.science (2026). https://pith.science/paper/CZDYA2XH

@misc{pith2026241205789,
  author       = {Pith},
  title        = {Pith review of: InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CZDYA2XH}},
  note         = {Machine review of arXiv:2412.05789}
}
read the original abstract

Realizing scaling laws in embodied AI has become a focus. However, previous work has been scattered across diverse simulation platforms, with assets and models lacking unified interfaces, which has led to inefficiencies in research. To address this, we introduce InfiniteWorld, a unified and scalable simulator for general vision-language robot interaction built on Nvidia Isaac Sim. InfiniteWorld encompasses a comprehensive set of physics asset construction methods and generalized free robot interaction benchmarks. Specifically, we first built a unified and scalable simulation framework for embodied learning that integrates a series of improvements in generation-driven 3D asset construction, Real2Sim, automated annotation framework, and unified 3D asset processing. This framework provides a unified and scalable platform for robot interaction and learning. In addition, to simulate realistic robot interaction, we build four new general benchmarks, including scene graph collaborative exploration and open-world social mobile manipulation. The former is often overlooked as an important task for robots to explore the environment and build scene knowledge, while the latter simulates robot interaction tasks with different levels of knowledge agents based on the former. They can more comprehensively evaluate the embodied agent's capabilities in environmental understanding, task planning and execution, and intelligent interaction. We hope that this work can provide the community with a systematic asset interface, alleviate the dilemma of the lack of high-quality assets, and provide a more comprehensive evaluation of robot interactions.

Figures

Figures reproduced from arXiv: 2412.05789 by the authors.

Figure 1
Figure 1. Language-driven automatic scene generation and editing framework based on HOLODECK [ [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The reconstruction visual comparisons on test-set views among GauStudio, SuGaR, PGSR, and our proposed method from [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The Annot8-3D framework pipeline. embodied AI interaction with equal knowledge acquisition capabilities. • Hierarchical interaction. In hierarchical interaction tasks, it is used to simulate the agent interaction mode with a hierarchical knowledge structure in the environ￾ment. For example, compared to ordinary agents, admin￾istrators (such as salespersons, etc.) clearly have more knowledge about the environment. En… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Overview of the functions of InfiniteWorld simulator. Our simulation platform supports different sensors, robot platforms, and [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: An overview of the proposed benchmark. • VLM Zero-Shot. By inputting the global scene infor￾mation and current observations into a vision-language model (VLM), we use prompt engineering to output the actions that the agent should execute. • Single Semantic Map. We use …
Figure 6
Figure 6. Figure 6: Comparative of reconstruction results with and with [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Visualization of Benchmark 1 and Benchmark 2. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A training-free pipeline generates instance-level, physically interactive 3D tabletop scenes from text or one image, with a differentiable rotation optimizer and top-view spatial alignment for collision-free layouts.

  2. {\lambda}: A Benchmark for Data-Efficiency in Long-Horizon Indoor Mobile Manipulation Robotics

    cs.RO 2024-11 conditional novelty 6.0 of 10

    A new benchmark with 571 human-collected demonstrations shows end-to-end robot learning is very data-inefficient on long-horizon mobile manipulation, while a neuro-symbolic planner reaches 44.4% success zero-shot.

  3. FDBPL: Faster Distillation-Based Prompt Learning for Region-Aware Vision-Language Models Adaptation

    cs.CV 2025-05 conditional novelty 4.0 of 10

    FDBPL caches teacher soft labels offline and adds positive-negative region prompts, achieving 2.2x faster training and modest zero-shot gains over prior distillation-based prompt learning.

Reference graph

Works this paper leans on

87 extracted references · 48 canonical work pages · cited by 3 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    H. A. Arief, M. Arief, G. Zhang, Z. Liu, M. Bhat, U. G. Indahl, H. Tveite, and D. Zhao. Sane: Smart annotation and evaluation tools for point cloud data. IEEE Access, 8: 131848–131858, 2020. 2

    2020

  3. [3]

    A lightweight approach to repairing digitized polygon meshes

    Marco Attene. A lightweight approach to repairing digitized polygon meshes. The visual computer, 26:1393–1406, 2010. 1

    2010

  4. [4]

    Depth pro: Sharp monocular metric depth in less than a second

    Aleksei Bochkovskii, Ama ¨el Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. arXiv preprint arXiv:2410.02073, 2024. 4

    arXiv 2024

  5. [5]

    Object goal navi- gation using goal-oriented semantic exploration

    Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Ab- hinav Gupta, and Russ R Salakhutdinov. Object goal navi- gation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems , 33:4247–4258,

  6. [6]

    Soundspaces: Audio-visual navigation in 3d environments

    Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vi- cenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, and Kristen Grauman. Soundspaces: Audio-visual navigation in 3d environments. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16, pages 17–

    2020

  7. [7]

    Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction

    Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang, Weijian Xie, Shangjin Zhai, Nan Wang, Haomin Liu, Hujun Bao, and Guofeng Zhang. Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction. arXiv preprint arXiv:2406.06521, 2024. 2, 3, 4, 1

    arXiv 2024

  8. [8]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 5, 3

    2017

Show all 87 references
  1. [9]

    Robothor: An open simulation-to-real embodied ai platform

    Matt Deitke, Winson Han, Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mottaghi, Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, et al. Robothor: An open simulation-to-real embodied ai platform. In Proceedings of the IEEE/CVF conference on compu...

    2020

  2. [10]

    Procthor: Large-scale embodied ai using procedural generation

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural generation. Ad- vances in Neural Information Processing Systems, 35:5982...

    2022

  3. [11]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems, 36, 2024. 5, 3

    2024

  4. [12]

    A survey of embodied ai: From simulators to research tasks

    Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2):230–244, 2022. 1

    2022

  5. [13]

    Manipulathor: A framework for visual object ma- nipulation

    Kiana Ehsani, Winson Han, Alvaro Herrasti, Eli VanderBilt, Luca Weihs, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Manipulathor: A framework for visual object ma- nipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages ...

  6. [14]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981. 1

    1981

  7. [15]

    Xtreme1 - the next gen platform for multisensory training data, 2023

    LF AI & Data Foundation. Xtreme1 - the next gen platform for multisensory training data, 2023. Software available from https://github.com/xtreme1-io/xtreme1/. 2

    2023

  8. [16]

    An algorithm for finding best matches in logarithmic expected time

    Jerome H Friedman, Jon Louis Bentley, and Raphael Ari Finkel. An algorithm for finding best matches in logarithmic expected time. ACM Transactions on Mathematical Software (TOMS), 3(3):209–226, 1977. 1

    1977

  9. [17]

    3d-front: 3d furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Bin- qiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 10933–10942,

  10. [18]

    Rfuniverse: A physics-based action-centric in- teractive environment for everyday household tasks

    Haoyuan Fu, Wenqiang Xu, Han Xue, Huinan Yang, Ruolin Ye, Yongxi Huang, Zhendong Xue, Yanfeng Wang, and Cewu Lu. Rfuniverse: A physics-based action-centric in- teractive environment for everyday household tasks. arXiv preprint arXiv:2202.00199, 2, 2022. 3

    2022 arXiv

  11. [19]

    The threedworld transport challenge: A visually guided task- and-motion planning benchmark towards physically realistic embodied ai

    Chuang Gan, Siyuan Zhou, Jeremy Schwartz, Seth Alter, Abhishek Bhandwaldar, Dan Gutfreund, Daniel LK Yamins, James J DiCarlo, Josh McDermott, Antonio Torralba, et al. The threedworld transport challenge: A visually guided task- and-motion planning benchmark towards physically ...

    2022

  12. [20]

    Embodiedcity: A benchmark platform for embodied agent in real-world city environment

    Chen Gao, Baining Zhao, Weichen Zhang, Jinzhu Mao, Jun Zhang, Zhiheng Zheng, Fanhang Man, Jianjie Fang, Zile Zhou, Jinqiang Cui, et al. Embodiedcity: A benchmark platform for embodied agent in real-world city environment. arXiv preprint arXiv:2410.09604, 2024. 1

    arXiv

  13. [21]

    Arnold: A benchmark for language-grounded task learning with con- tinuous states in realistic 3d scenes

    Ran Gong, Jiangyong Huang, Yizhou Zhao, Haoran Geng, Xiaofeng Gao, Qingyang Wu, Wensi Ai, Ziheng Zhou, Demetri Terzopoulos, Song-Chun Zhu, et al. Arnold: A benchmark for language-grounded task learning with con- tinuous states in realistic 3d scenes. In Proceedings of the IEEE...

    2023

  14. [22]

    Maniskill2: A unified bench- mark for generalizable manipulation skills

    Jiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yunchao Yao, et al. Maniskill2: A unified bench- mark for generalizable manipulation skills. arXiv preprint arXiv:2302.04659, 2023. 3

    2023 arXiv

  15. [23]

    Sugar: Surface- aligned gaussian splatting for efficient 3d mesh reconstruc- 10 tion and high-quality mesh rendering

    Antoine Gu ´edon and Vincent Lepetit. Sugar: Surface- aligned gaussian splatting for efficient 3d mesh reconstruc- 10 tion and high-quality mesh rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5354–5363, 2024. 4

    2024

  16. [24]

    Sam2point: Segment any 3d as videos in zero-shot and promptable man- ners

    Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Chengzhuo Tong, Peng Gao, Chunyuan Li, and Pheng-Ann Heng. Sam2point: Segment any 3d as videos in zero-shot and promptable man- ners. arXiv preprint arXiv:2408.16768, 2024. 5

    2024 arXiv

  17. [25]

    Rlbench: The robot learning benchmark & learning environment

    Stephen James, Zicong Ma, David Rovick Arrojo, and An- drew J Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Let- ters, 5(2):3019–3026, 2020. 3

    2020

  18. [26]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  19. [27]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4), 2023. 4

    2023

  20. [28]

    Chang, and Manolis Savva

    Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X. Chang, and Manolis Savva. Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal naviga...

    2024

  21. [29]

    Droid: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ash- win Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yun- liang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:24...

    2024 arXiv

  22. [30]

    Beyond the nav-graph: Vision-and-language navigation in continuous environments

    Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision-and-language navigation in continuous environments. InComputer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part XXVIII 16, pages 104–

    2020

  23. [31]

    igibson 2.0: Object-centric simulation for robot learning of everyday household tasks

    Chengshu Li, Fei Xia, Roberto Mart ´ın-Mart´ın, Michael Lin- gelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, et al. igibson 2.0: Object-centric simulation for robot learning of everyday household tasks. arXiv preprint arXiv:2108.032...

    2021 arXiv

  24. [32]

    Behavior-1k: A benchmark for embodied ai with 1,000 ev- eryday activities and realistic simulation

    Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Mart ´ın-Mart´ın, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior-1k: A benchmark for embodied ai with 1,000 ev- eryday activities and realistic simulation. In Conferenc...

    2023

  25. [33]

    Sustech points: A portable 3d point cloud interactive annotation platform system

    E Li, Shuaijun Wang, Chengyang Li, Dachuan Li, Xiangbin Wu, and Qi Hao. Sustech points: A portable 3d point cloud interactive annotation platform system. In 2020 IEEE In- telligent Vehicles Symposium (IV), pages 1108–1115. IEEE,

    2020

  26. [34]

    Instructscene: Instruction- driven 3d indoor scene synthesis with semantic graph prior

    Chenguo Lin and Yadong Mu. Instructscene: Instruction- driven 3d indoor scene synthesis with semantic graph prior. In International Conference on Learning Representations (ICLR), 2024. 3

    2024

  27. [35]

    Soft- gym: Benchmarking deep reinforcement learning for de- formable object manipulation

    Xingyu Lin, Yufei Wang, Jake Olkin, and David Held. Soft- gym: Benchmarking deep reinforcement learning for de- formable object manipulation. In Conference on Robot Learning, pages 432–448. PMLR, 2021. 3

    2021

  28. [36]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 1

    2024

  29. [37]

    Cage: Controllable articulation generation

    Jiayi Liu, Hou In Ivan Tam, Ali Mahdavi-Amiri, and Manolis Savva. Cage: Controllable articulation generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17880–17889, 2024. 1, 4

    2024

  30. [38]

    Aerialvln: Vision-and-language navigation for uavs

    Shubo Liu, Hongsheng Zhang, Yuankai Qi, Peng Wang, Yan- ning Zhang, and Qi Wu. Aerialvln: Vision-and-language navigation for uavs. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 15384– 15394, 2023. 1

    2023

  31. [39]

    Marching cubes: A high resolution 3d surface construction algorithm

    William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. InSem- inal graphics: pioneering efforts that shaped the field, pages 347–353. 1998. 1

    1998

  32. [40]

    Mimicgen: A data generation system for scalable robot learning using human demonstrations

    Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Di- eter Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. arXiv preprint arXiv:2310.17596, 2023. 3

    arXiv

  33. [41]

    Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manip- ulation tasks

    Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wol- fram Burgard. Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manip- ulation tasks. IEEE Robotics and Automation Letters , 7(3): 7327–7334, 2022. 3

    2022

  34. [42]

    Orbit: A unified simulation framework for interactive robot learning environments.IEEE Robotics and Automation Letters, 8(6):3740–3747, 2023

    Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, et al. Orbit: A unified simulation framework for interactive robot learning environments.IEEE Robotics and Automation Letters, 8(6):3740–3747,...

    2023

  35. [43]

    Partnet: A large- scale benchmark for fine-grained and hierarchical part-level 3d object understanding

    Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. Partnet: A large- scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog...

    2019

  36. [44]

    PyMeshLab, 2021

    Alessandro Muntoni and Paolo Cignoni. PyMeshLab, 2021. 1

    2021

  37. [45]

    Robocasa: Large-scale simula- tion of everyday tasks for generalist robots

    Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Man- dlekar, and Yuke Zhu. Robocasa: Large-scale simula- tion of everyday tasks for generalist robots. arXiv preprint arXiv:2406.02523, 2024. 1, 3

    2024 arXiv

  38. [46]

    Kinectfusion: Real-time dense surface mapping and track- ing

    Richard A Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J Davison, Pushmeet Kohi, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. Kinectfusion: Real-time dense surface mapping and track- ing. In 2011 10th IEEE international symposium on mixed ...

    2011

  39. [47]

    Isaac sim 4.0 - robotics simulation and syn- thetic data generation

    NVIDIA. Isaac sim 4.0 - robotics simulation and syn- thetic data generation. https://developer.nvidia.com/isaac- sim, 2024. 1

    2024

  40. [48]

    Open x-embodiment: Robotic learning datasets and rt-x models

    Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, et al. Open x-embodiment: Robotic learning datasets and rt-x models. arXiv preprint arXiv:2310.08864, 2023. 1

    arXiv

  41. [49]

    https://openai.com/index/hello-gpt-4o/

    Openai. https://openai.com/index/hello-gpt-4o/. 2024. 6, 7, 8

    2024

  42. [50]

    Global Structure-from-Motion Revisited

    Linfei Pan, Daniel Barath, Marc Pollefeys, and Jo- hannes Lutz Sch ¨onberger. Global Structure-from-Motion Revisited. In European Conference on Computer Vision (ECCV), 2024. 1

    2024

  43. [51]

    Virtualhome: Simulating household activities via programs

    Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating household activities via programs. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 8494–8502, 2018. 3

    2018

  44. [52]

    Habitat 3.0: A co-habitat for humans, avatars and robots

    Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dal- laire Cote, Tsung-Yen Yang, Ruslan Partsey, Ruta Desai, Alexander William Clegg, Michal Hlavac, So Yeon Min, et al. Habitat 3.0: A co-habitat for humans, avatars and robots. arXiv preprint arXiv:2310.13724, 2023. 2, 3, 4

    arXiv

  45. [54]

    Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai

    Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Un- dersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv prepri...

    2021 arXiv

  46. [55]

    Rein and S

    H. Rein and S. F. Liu. REBOUND: an open-source multi- purpose N-body code for collisional dynamics. aap, 537:A128, 2012. 2

    2012

  47. [56]

    Surfer: Progressive reasoning with world models for robotic manipulation, 2024

    Pengzhen Ren, Kaidong Zhang, Hetao Zheng, Zixuan Li, Yuhang Wen, Fengda Zhu, Mas Ma, and Xiaodan Liang. Surfer: Progressive reasoning with world models for robotic manipulation, 2024. 3

    2024

  48. [57]

    label- cloud: A lightweight domain-independent labeling tool for 3d object detection in point clouds, 2021

    Christoph Sager, Patrick Zschech, and Niklas K ¨uhl. label- cloud: A lightweight domain-independent labeling tool for 3d object detection in point clouds, 2021. 2

    2021

  49. [58]

    Habitat: A platform for embodied ai research

    Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF international conference on computer vision, ...

    2019

  50. [59]

    A fast marching level set method for monotoni- cally advancing fronts.Proceedings of the National Academy of Sciences, page 1591–1595, 1996

    J A Sethian. A fast marching level set method for monotoni- cally advancing fronts.Proceedings of the National Academy of Sciences, page 1591–1595, 1996. 7

    1996

  51. [60]

    igibson 1.0: A simulation environment for interactive tasks in large realistic scenes

    Bokui Shen, Fei Xia, Chengshu Li, Roberto Mart ´ın-Mart´ın, Linxi Fan, Guanzhi Wang, Claudia P ´erez-D’Arpino, Shya- mal Buch, Sanjana Srivastava, Lyne Tchapmi, et al. igibson 1.0: A simulation environment for interactive tasks in large realistic scenes. In 2021 IEEE/RSJ Inter...

    2021

  52. [61]

    Alfred: A benchmark for interpreting grounded instructions for everyday tasks

    Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern ...

    2020

  53. [62]

    Sadler, Wei-Lun Chao, and Yu Su

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M. Sadler, Wei-Lun Chao, and Yu Su. Llm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings on the IEEE/CVF International Conference on Computer Vision (ICCV), 2023. 7

    2023

  54. [63]

    The replica dataset: A digital replica of indoor spaces

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05797,

    1906 arXiv

  55. [64]

    Habitat 2.0: Training home assistants to rearrange their habitat

    Andrew Szot, Alexander Clegg, Eric Undersander, Erik Wi- jmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Singh Chaplot, Oleksandr Maksymets, et al. Habitat 2.0: Training home assistants to rearrange their habitat. Advances in neural information processin...

    2021

  56. [65]

    Triposr: Fast 3d object reconstruction from a single image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. arXiv preprint arXiv:2403.02151, 2024. 1, 3, 4

    2024 arXiv

  57. [66]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1

    2023 arXiv

  58. [67]

    Hand- methat: Human-robot communication in physical and social environments

    Yanming Wan, Jiayuan Mao, and Josh Tenenbaum. Hand- methat: Human-robot communication in physical and social environments. Advances in Neural Information Processing Systems, 35:12014–12026, 2022. 3

    2022

  59. [68]

    Grutopia: Dream general robots in a city at scale

    Hanqing Wang, Jiahe Chen, Wensi Huang, Qingwei Ben, Tai Wang, Boyu Mi, Tao Huang, Siheng Zhao, Yilun Chen, Sizhe Yang, et al. Grutopia: Dream general robots in a city at scale. arXiv preprint arXiv:2407.10943, 2024. 1, 2, 3, 4

    2024 arXiv

  60. [69]

    Robogen: Towards unleashing infi- nite data for automated robot learning via generative simula- tion

    Yufei Wang, Zhou Xian, Feng Chen, Tsun-Hsuan Wang, Yian Wang, Katerina Fragkiadaki, Zackory Erickson, David Held, and Chuang Gan. Robogen: Towards unleashing infi- nite data for automated robot learning via generative simula- tion. arXiv preprint arXiv:2311.01455, 2023. 3

    2023 arXiv

  61. [70]

    Multion: Benchmarking semantic map memory using multi-object navigation

    Saim Wani, Shivansh Patel, Unnat Jain, Angel Chang, and Manolis Savva. Multion: Benchmarking semantic map memory using multi-object navigation. Advances in Neural Information Processing Systems, 33:9700–9712, 2020. 3

    2020

  62. [71]

    Metaurban: A simulation platform for embodied ai in urban spaces.arXiv preprint arXiv:2407.08725, 2024

    Wayne Wu, Honglin He, Yiran Wang, Chenda Duan, Jack He, Zhizheng Liu, Quanyi Li, and Bolei Zhou. Metaurban: A simulation platform for embodied ai in urban spaces.arXiv preprint arXiv:2407.08725, 2024. 1, 2, 3 12

    2024 arXiv

  63. [72]

    Point transformer v3: Simpler, faster, stronger

    Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xi- hui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler, faster, stronger. In CVPR, 2024. 5

    2024

  64. [73]

    Gibson env: Real-world percep- tion for embodied agents

    Fei Xia, Amir R Zamir, Zhiyang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gibson env: Real-world percep- tion for embodied agents. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 9068–9079, 2018. 3

    2018

  65. [74]

    Yamauchi

    B. Yamauchi. A frontier-based approach for autonomous exploration. In Proceedings 1997 IEEE International Sym- posium on Computational Intelligence in Robotics and Au- tomation CIRA’97. ’Towards New Computational Principles for Robotics and Automation’, pages 146–151, 1997. 7

    1997

  66. [75]

    Physcene: Physically interactable 3d scene synthe- sis for embodied ai

    Yandan Yang, Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. Physcene: Physically interactable 3d scene synthe- sis for embodied ai. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 16262–16272, 2024. 1, 3

    2024

  67. [76]

    Llplace: The 3d in- door scene layout generation and editing via large language model

    Yixuan Yang, Junru Lu, Zixiang Zhao, Zhen Luo, James JQ Yu, Victor Sanchez, and Feng Zheng. Llplace: The 3d in- door scene layout generation and editing via large language model. arXiv preprint arXiv:2406.03866, 2024. 3

    2024 arXiv

  68. [77]

    Holodeck: Language guided gen- eration of 3d embodied ai environments

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Al- varo Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided gen- eration of 3d embodied ai environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and ...

    2024

  69. [78]

    Gaustudio: A modular frame- work for 3d gaussian splatting and beyond

    Chongjie Ye, Yinyu Nie, Jiahao Chang, Yuantao Chen, Yi- hao Zhi, and Xiaoguang Han. Gaustudio: A modular frame- work for 3d gaussian splatting and beyond. arXiv preprint arXiv:2403.19632, 2024. 4

    2024 arXiv

  70. [79]

    Homerobot: Open-vocabulary mobile manipulation

    Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav, Austin Wang, Mukul Khanna, Theophile Gervet, Tsung-Yen Yang, Vidhi Jain, Alexander William Clegg, John Turner, et al. Homerobot: Open-vocabulary mobile manipulation. arXiv preprint arXiv:2306.11565, 2023. 4

    2023 arXiv

  71. [80]

    Co-navgpt: Multi-robot cooperative visual semantic navigation using large language models

    Bangguo Yu, Hamidreza Kasaei, and Ming Cao. Co-navgpt: Multi-robot cooperative visual semantic navigation using large language models. arXiv preprint arXiv:2310.07937 ,

    arXiv

  72. [81]

    Transporter networks: Rearranging the visual world for robotic manipu- lation

    Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, et al. Transporter networks: Rearranging the visual world for robotic manipu- lation. In Conference on Robot Learning , pages 7...

    2021

  73. [82]

    Clothesnet: An information-rich 3d garment model repository with simulated clothes environ- ment

    Bingyang Zhou, Haoyu Zhou, Tianhai Liang, Qiaojun Yu, Siheng Zhao, Yuwei Zeng, Jun Lv, Siyuan Luo, Qiancai Wang, Xinyuan Yu, et al. Clothesnet: An information-rich 3d garment model repository with simulated clothes environ- ment. In Proceedings of the IEEE/CVF International Co...

    2023

  74. [83]

    3d bat: A semi-automatic, web-based 3d annotation toolbox for full-surround, multi-modal data streams

    Walter Zimmer, Akshay Rangesh, and Mohan Trivedi. 3d bat: A semi-automatic, web-based 3d annotation toolbox for full-surround, multi-modal data streams. In 2019 IEEE In- telligent Vehicles Symposium (IV), pages 1816–1821. IEEE,

    2019

  75. [85]

    Simulation Details 6.1. Depth-Prior-Constrained Real2Sim Pipeline Specifically, Our 3D scene reconstruction pipeline includes the entire process from photographic data to accurate and visually coherent models. Its main steps are as follows: • SfM. The process begins with colma...

  76. [86]

    • Denoising

    algorithm to detect and align the dominant plane and rotate the whole scene for z-axis alignment. • Denoising. Using a connectivity-cluster approach, we ef- fectively filter noise, setting a threshold to remove extra- neous points from high spatial areas. This step reduces mod...

    2019

  77. [87]

    Comparison of 3D annotation tools

    [2] POINT[33] Cloud[57] [55] [15] (Ours) Year 2019 2020 2020 2021 2023 2023 2024 2D/3D cam.+LiDAR fusion ✓ - ✓ ✓ ✓ ✓ ✓ AI-assisted labeling ✓ ✓ ✓ - ✓ ✓ ✓ Label custom attributes - - ✓ - ✓ ✓ ✓ HD Maps - - ✓ - - ✓ ✓ Web-based ✓ - ✓ - - ✓ ✓ 3D navigation ✓ ✓ ✓ - ✓ ✓ ✓ 3D transfor...

    2019

  78. [88]

    free”, “obstacle

    Experimental Details 7.1. Task Setting We provide simulation assistance to help users complete various customized tasks on InfiniteWorld Simulation. • Occupy Map. For each scene, we generate an occupy map, a two-dimensional grid map used for embodied agent navigation. The occu...

  79. [2019]

    In Section 7, we present more experimental details and results

    2 13 InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction Supplementary Material In the supplementary material, we present more details about the simulator in Section 6. In Section 7, we present more experimental details and results

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.