Pith. sign in

REVIEW 3 major objections 5 minor 49 references

FROSS: Faster-than-Real-Time Online 3D Semantic Scene Graph Generation from RGB-D Images

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new pipeline builds online 3D semantic scene graphs from RGB-D video at 7 ms per frame, over 20 times faster than prior real-time systems, by lifting 2D scene graphs into 3D Gaussian objects.

desk verdict FROSS is a genuine advance for online 3D SSG generation, but the 7 ms speed claim excludes camera pose estimation and the 'superior performance' claim overstates the mean predicate recall results. read the letter →

arxiv 2507.19993 v2 pith:EJYFT7ZQ submitted 2025-07-26 cs.CV

Hao-Yu Hou , Chun-Yi Lee , Motoharu Sonogashira , Yasutomo Kawanishi This is my paper · ORCID
classification cs.CV
keywords 3DsemanticscenegraphonlinegenerationRGB-DperceptionGaussianobjectrepresentationreal-timecomputervisionindoorunderstandingReplicaSSGdataset2D-to-3Dlifting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that generating 3D semantic scene graphs from live RGB-D video does not require precise 3D reconstruction: it is enough to lift 2D scene graphs directly into 3D space, modeling each object as a Gaussian distribution. The authors report that this shortcut runs at 7 ms per frame end-to-end, faster than real time on a single CPU core and one GPU, while achieving the highest relationship, object, and predicate recall among evaluated methods on the 3DSSG benchmark. They also introduce ReplicaSSG, a Replica-based dataset with inter-object relationship annotations, to evaluate the approach on higher-quality imagery. If the claim holds, online scene-graph perception becomes practical for robots, augmented reality, and other latency-constrained systems.

What carries the argument

The load-bearing construction is the 3D Gaussian object representation. Each object is a Gaussian whose mean comes from back-projecting its bounding-box center and whose covariance comes from the back-projected bounding-box covariance $\Sigma_i^{\mathrm{2D}} = \frac{1}{12}\mathrm{diag}(W_i^2, H_i^2)$, with the depth variance filled by $((\Sigma_i^{\mathrm{3D}''})_{1,1} + (\Sigma_i^{\mathrm{3D}''})_{2,2})/2$ in Eq. 6. The same object seen from multiple views is identified by computing the Hellinger distance between Gaussians of the same semantic class, and merging is a weighted mixture reduction with weights equal to detection frequency. These Gaussians substitute for point clouds in both object correspondence and relationship aggregation, which is what makes faster-than-real-time operation possible.

What would settle it

On a scene with ground-truth instance masks, replace the Eq. 6 depth variance with empirically measured depth extents and recompute the Hellinger-distance merging decisions; if the merged object set changes substantially, or if threshold-based merges disagree with ground-truth object identity on many object pairs, the central mechanism is not supported.

Watch

Extended reading notes

Core claim

FROSS constructs a global 3D semantic scene graph incrementally by converting each detected 2D object into a Gaussian over its bounding box, back-projecting that Gaussian into 3D through the pseudo-inverse of the projective Jacobian, and filling the missing depth variance with the average of the two spatial variances (Eqs. 4–7). Objects of the same predicted class are merged when the Hellinger distance between their Gaussians falls below a threshold, with detection counts acting as weights in a Gaussian mixture reduction (Eqs. 10–11). Relationships extracted in 2D are carried along, redirected to the merged nodes, and settled by majority vote across viewpoints. Because no point cloud is ever built, the pipeline avoids the reconstruction cost that dominates earlier online systems, and the paper reports 27.9% relationship recall, 62.4% object recall, and 33.0% predicate recall on 3DSSG at 7 ms latency.

Load-bearing premise

The whole merging algorithm rests on the heuristic in Eq. 6: the depth variance of each back-projected Gaussian is assumed to equal the average of its two image-plane variances, a choice the paper states only as a hypothesis.

Editorial extensions

If this is right

  • 3D semantic scene graph generation can be decoupled from SLAM and point-cloud reconstruction while still beating reconstruction-based methods on both accuracy and speed.
  • The 7 ms end-to-end latency, including detection and relationship extraction, leaves room for robots and AR systems to run scene-graph perception alongside other tasks on limited hardware.
  • The ablations show that 2D scene graph quality gates the final 3D SSG quality, so stronger 2D SG models should translate directly into stronger 3D SSGs without any 3D-specific retraining.
  • FROSS tolerates estimated camera trajectories from ORB-SLAM3 with negligible recall change, indicating the approach works with imperfect odometry in practical deployments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the pipeline never builds a dense map, it could extend naturally to dynamic scenes where reconstruction-based methods struggle, although the paper does not evaluate this setting.
  • The Eq. 6 depth-variance heuristic is a natural target for learned or per-pixel depth uncertainty; replacing it could tighten merge decisions and improve object recall without changing the framework.
  • The ReplicaSSG dataset, with Visual Genome-based categories, may support zero-shot transfer evaluation for future 2D scene graph models, not just FROSS.
  • FROSS's relationship recall roughly doubles when ground-truth 2D SGs are used (67.6 vs 22.3 on ReplicaSSG), suggesting that the measured 3D SSG numbers are more a verdict on the 2D detector than on the 3D lifting idea.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FROSS, a pipeline for online 3D semantic scene graph generation from RGB-D sequences. It first extracts 2D scene graphs with RT-DETR and EGTR, then lifts detected objects to 3D as Gaussian distributions using depth and camera pose, and incrementally merges these distributions into a global graph via a Hellinger-distance threshold. The authors evaluate on the 3DSSG benchmark and on a new ReplicaSSG dataset with relationship annotations, reporting recall numbers and a 7 ms latency claim. They also ablate the effect of ground-truth 2D SGs and estimated camera trajectories.

Significance. If the central claim of faster-than-real-time online SSG generation is substantiated end-to-end, the paper makes a useful contribution: it avoids explicit point-cloud processing, introduces a lightweight Gaussian merging mechanism, and releases a new annotated dataset (ReplicaSSG). The public implementation and dataset are strengths. However, the speed claim as stated is not yet demonstrated for a complete online system, and the 'superior performance' claim is only partially supported by the reported metrics.

major comments (3)
  1. [Section 4.2, Table 1, Table 2] The headline '7 ms end-to-end' latency is not an end-to-end system latency. Table 1's caption explicitly states 'without environmental mapping,' and Section 4.2 says the main experiments use ground-truth camera trajectories. The runtime breakdown in Table 2 covers only object detection (2.31 ms), relation extraction (4.51 ms), and merging (0.12 ms), summing to about 6.9 ms. In an actual online deployment, camera poses must be obtained from SLAM or hardware; the ORB-SLAM3 ablation in Table 4 reports only recall, not latency. The paper should either qualify the speed claim as component-level throughput (i.e., excluding pose estimation) or provide a measured end-to-end latency that includes the pose-estimation module. As written, 'faster-than-real-time online 3D SSG generation' is not supported for the full system.
  2. [Section 4.3, Table 1] The claim that 'FROSS achieves the highest performance among all baseline methods' is contradicted by the mRecall Predicate column: FROSS scores 18.0, while SGFN scores 24.0 and Wu scores 26.6. Per-class predicate recall for 'attached to' is 29.4 for FROSS versus 58.4 for SGFN. The performance advantage holds for object-related metrics, but the paper overstates the overall result. The authors should restrict the 'superior performance' claim to the metrics where FROSS actually leads, or discuss the trade-off explicitly.
  3. [Section 3.3, Eq. (6)] The depth-variance assignment is a load-bearing modeling choice. The paper states 'We hypothesize that this variance approximates the average variance of the other dimensions,' but provides no derivation, validation, or sensitivity analysis. Since the Hellinger distance (Eq. 8) and the merging decisions (Section 3.4) depend directly on the covariance matrices, an inaccurate depth variance could substantially change the resulting graph structure. The paper should include an ablation varying the depth-variance parameter, or a justification from the projection geometry, to demonstrate that the merging algorithm is not overly sensitive to this assumption.
minor comments (5)
  1. [Section 4.1.3] The paper states that FROSS enforces a one-to-one correspondence between predicted and ground-truth objects, while other baselines may allow multiple predictions per ground-truth object. It should be clarified whether the reported baseline numbers are recomputed under the same one-to-one protocol; otherwise the comparison may be inconsistent.
  2. [Supplementary Table 8] The per-class ReplicaSSG object recall table is difficult to parse because the mean value (28.8) appears inline after the second row of class entries, without clear column alignment. Please reformat the table.
  3. [Abstract] There is a missing space in 'Faster-than-Real-TimeOnline' in the abstract; it should read 'Faster-than-Real-Time Online'.
  4. [Section 2.2] The description of SceneGraphFusion [34] as a method that 'can achieve real-time performance' while also being criticized for substantial latency is confusing; clarify what is meant by real-time in that context.
  5. [Section 3.4] The weighted Gaussian merging formula in Eq. (11) is standard, but the weights are defined as 'detection frequencies'; please specify how these frequencies are accumulated during incremental merging, especially after multiple merges.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FROSS's 3D lifting and merging pipeline is derived from external mathematical citations and validated on external benchmarks, with the depth-covariance heuristic explicitly labeled as a hypothesis.

full rationale

The derivation chain is self-contained and does not reduce to its inputs. The 3D covariance back-projection (Eqs. 4-7) follows the external affine-approximation Jacobian of Zwicker et al. [46], and the one modeling choice that is not derived—the depth variance in Eq. (6)—is explicitly introduced as 'We hypothesize that this variance approximates the average variance of the other dimensions,' so it is a stated assumption rather than a fitted parameter disguised as a prediction. The merging algorithm uses the standard Hellinger distance (Eqs. 8-9) and Gaussian-mixture reduction (Eqs. 10-11); the only threshold, delta_d = 0.85, is chosen by grid search on the validation split (Section 4.6.3, Table 5), not on the test set, so no recall number is forced by construction. The 2D SG components (RT-DETR and EGTR) are trained on external data (3DSSG and Visual Genome) and the final recall metrics are computed against ground-truth object and relationship annotations, so the reported predictions are not defined in terms of the outputs. The latency claim is qualified in the paper ('end-to-end latency without environmental mapping' and ground-truth trajectories in the main experiments); that is a measurement-scope limitation, not circular reasoning. No load-bearing self-citation chain or imported uniqueness theorem appears; citations to [22] and [46] are external mathematical sources. The analysis therefore finds no circular step.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The main contributions rest on two heuristics: the depth-variance assignment in Eq. 6 and the assumption that 2D relationship semantics lift to 3D unchanged. Both are stated but not derived. One free parameter, the Hellinger threshold, is tuned on validation and controls the entire merging behavior.

free parameters (3)
  • Hellinger distance threshold delta_d = 0.85
    Tuned via grid search on the validation split (Table 5) to favour relationship recall; controls which objects are merged.
  • Object confidence threshold = 0.7
    Filters detected objects before lifting to 3D; set in Section 4.2.
  • Top-10 relationships per frame = 10
    Prunes each 2D scene graph to the ten highest-confidence relationship triplets (Section 4.2).
assumptions (4)
  • ad hoc to paper The depth variance of the back-projected 3D Gaussian approximates the average of the two spatial variances (Eq. 6).
    Stated as 'We hypothesize' in Section 3.3; it fills the missing depth covariance and directly shapes the Gaussians used for merging.
  • domain assumption Pseudo-inverse of the projection Jacobian gives a valid 3D covariance when inverted from 2D (Eq. 4).
    The Jacobian in [46] is defined for 3D-to-2D projection; reversing it is an approximation not proven for this use case.
  • domain assumption Relationships predicted per 2D frame remain valid when edges are lifted to 3D and merged by majority vote.
    The 3D SSG edges carry no geometric verification; the method trusts the 2D SG model's relationship outputs across viewpoints.
  • domain assumption Main evaluation uses ground truth camera trajectories; estimated trajectories from ORB-SLAM3 are tested only on ReplicaSSG.
    The headline latency and 3DSSG results assume perfect pose, which is not available in open-world deployment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FROSS: Faster-than-Real-Time Online 3D Semantic Scene Graph Generation from RGB-D Images." pith.science (2026). https://pith.science/paper/EJYFT7ZQ

@misc{pith2026250719993,
  author       = {Pith},
  title        = {Pith review of: FROSS: Faster-than-Real-Time Online 3D Semantic Scene Graph Generation from RGB-D Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EJYFT7ZQ}},
  note         = {Machine review of arXiv:2507.19993}
}
read the original abstract

The ability to abstract complex 3D environments into simplified and structured representations is crucial across various domains. 3D semantic scene graphs (SSGs) achieve this by representing objects as nodes and their interrelationships as edges, facilitating high-level scene understanding. Existing methods for 3D SSG generation, however, face significant challenges, including high computational demands and non-incremental processing that hinder their suitability for real-time open-world applications. To address this issue, we propose FROSS (Faster-than-Real-Time Online 3D Semantic Scene Graph Generation), an innovative approach for online and faster-than-real-time 3D SSG generation that leverages the direct lifting of 2D scene graphs to 3D space and represents objects as 3D Gaussian distributions. This framework eliminates the dependency on precise and computationally-intensive point cloud processing. Furthermore, we extend the Replica dataset with inter-object relationship annotations, creating the ReplicaSSG dataset for comprehensive evaluation of FROSS. The experimental results from evaluations on ReplicaSSG and 3DSSG datasets show that FROSS can achieve superior performance while operating significantly faster than prior 3D SSG generation methods. Our implementation and dataset are publicly available at https://github.com/Howardkhh/FROSS.

Figures

Figures reproduced from arXiv: 2507.19993 by the authors.

Figure 1
Figure 1. We introduce FROSS, an online real-time 3D semantic [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of the FROSS framework: (a) The process initiates with object detection via RT-DETR [ [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the object merging process using the proposed algorithm described in Section [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison between FROSS and Wu [ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results of FROSS on four scenes in the ReplicaSSG dataset. Please note that only representative objects are visualized, [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: The occurrence frequency of each object category in the [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 8
Figure 8. Figure 8: The number of objects present in each scene within the [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 45 canonical work pages

  1. [1]

    3D scene graph: A structure for unified semantics, 3D space, and camera

    Iro Armeni, Zhi-Yang He, JunYoung Gwak, Amir R Zamir, Martin Fischer, Jitendra Malik, and Silvio Savarese. 3D scene graph: A structure for unified semantics, 3D space, and camera. InInt. Conf. Comput. Vis., pages 5664–5673,

    work page

  2. [2]

    ORB-SLAM3: An accurate open-source library for visual, visual–inertial, and multimap slam.IEEE Trans

    Carlos Campos, Richard Elvira, Juan J G ´omez Rodr´ıguez, Jos´e MM Montiel, and Juan D Tard ´os. ORB-SLAM3: An accurate open-source library for visual, visual–inertial, and multimap slam.IEEE Trans. Robotics, 37(6):1874–1890,

    work page

  3. [3]

    I-Design: Personal- ized llm interior designer.arXiv:2404.02838, 2024

    Ata C ¸ elen, Guo Han, Konrad Schindler, Luc Van Gool, Iro Armeni, Anton Obukhov, and Xi Wang. I-Design: Personal- ized llm interior designer.arXiv:2404.02838, 2024. 2

    arXiv 2024

  4. [4]

    A look at gaussian mixture reduction algorithms

    David F Crouse, Peter Willett, Krishna Pattipati, and Lennart Svensson. A look at gaussian mixture reduction algorithms. InInt. Conf. Inform. Fusion, pages 1–8. IEEE, 2011. 5

    work page 2011

  5. [5]

    LayoutGPT: Compositional visual planning and generation with large language models.Adv

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Ar- jun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. LayoutGPT: Compositional visual planning and generation with large language models.Adv. Neural Inform. Process. Syst., 36, 2024. 2

    work page 2024

  6. [6]

    PDDL2.1: An extension to PDDL for expressing temporal planning domains.J

    Maria Fox and Derek Long. PDDL2.1: An extension to PDDL for expressing temporal planning domains.J. Arti- ficial Intell. Research, 20:61–124, 2003. 2

    work page 2003

  7. [7]

    GraphDreamer: Compositional 3D scene synthesis from scene graphs

    Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Sch ¨olkopf. GraphDreamer: Compositional 3D scene synthesis from scene graphs. InIEEE Conf. Comput. Vis. Pattern Recog., 2024. 2

    work page 2024

  8. [8]

    Image captioning with scene-graph based semantic concepts

    Lizhao Gao, Bo Wang, and Wenmin Wang. Image captioning with scene-graph based semantic concepts. InInt. Conf. on Mach. Learn. Comput., pages 225–229, 2018. 2

    work page 2018

Show all 49 references
  1. [9]

    Visual graphs from motion (VGfM): Scene understanding with object ge- ometry reasoning

    Paul Gay, James Stuart, and Alessio Del Bue. Visual graphs from motion (VGfM): Scene understanding with object ge- ometry reasoning. InACCV, pages 330–346. Springer, 2019. 1, 2, 3, 4, 6, 7

    2019

  2. [10]

    FF: The fast-forward planning system.AI magazine, 22(3):57–57, 2001

    J ¨org Hoffmann. FF: The fast-forward planning system.AI magazine, 22(3):57–57, 2001. 2

    2001

  3. [11]

    Hydra: A real-time spatial perception system for 3D scene graph con- struction and optimization

    Nathan Hughes, Yun Chang, and Luca Carlone. Hydra: A real-time spatial perception system for 3D scene graph con- struction and optimization. InRobotics: Sci. Syst., 2022. 3

    2022

  4. [12]

    EGTR: Extracting graph from trans- former for scene graph generation

    Jinbae Im, JeongYeon Nam, Nokyung Park, Hyungmin Lee, and Seunghyun Park. EGTR: Extracting graph from trans- former for scene graph generation. InIEEE Conf. Comput. Vis. Pattern Recog., pages 24229–24238, 2024. 2, 3, 4, 6, 8, 1

    2024

  5. [13]

    Image retrieval using scene graphs

    Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei. Image retrieval using scene graphs. InIEEE Conf. Comput. Vis. Pattern Recog., pages 3668–3678, 2015. 2, 6

    2015

  6. [14]

    Image genera- tion from scene graphs

    Justin Johnson, Agrim Gupta, and Li Fei-Fei. Image genera- tion from scene graphs. InIEEE Conf. Comput. Vis. Pattern Recog., pages 1219–1228, 2018. 2

    2018

  7. [15]

    Dense relational captioning: Triple-stream networks for relationship-based captioning

    Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh, and In So Kweon. Dense relational captioning: Triple-stream networks for relationship-based captioning. InIEEE Conf. Comput. Vis. Pattern Recog., pages 6271–6280, 2019. 2

    2019

  8. [16]

    3-D scene graph: A sparse and semantic rep- resentation of physical environments for intelligent agents

    Ue-Hwan Kim, Jin-Man Park, Taek-Jin Song, and Jong- Hwan Kim. 3-D scene graph: A sparse and semantic rep- resentation of physical environments for intelligent agents. IEEE Trans. Cybernetics, 50(12):4921–4933, 2019. 2, 3, 6, 7, 8

    2019

  9. [17]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Semi-supervised clas- sification with graph convolutional networks. InInt. Conf. Learn. Represent., 2017. 3

    2017

  10. [18]

    Visual Genome: Connecting language and vision using crowdsourced dense image annotations.Int

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A Shamma, et al. Visual Genome: Connecting language and vision using crowdsourced dense image annotations.Int. J. Comput. Vis., 123:32–73, ...

    2017

  11. [19]

    Relation- aware graph attention network for visual question answering

    Linjie Li, Zhe Gan, Yu Cheng, and Jingjing Liu. Relation- aware graph attention network for visual question answering. InInt. Conf. Comput. Vis., pages 10313–10322, 2019. 2

    2019

  12. [20]

    Explore con- textual information for 3D scene graph generation.IEEE Trans

    Yuanyuan Liu, Chengjiang Long, Zhaoxuan Zhang, Bokai Liu, Qiang Zhang, Baocai Yin, and Xin Yang. Explore con- textual information for 3D scene graph generation.IEEE Trans. Vis. Comput. Graph., 29(12):5556–5568, 2022. 3, 6

    2022

  13. [21]

    RT-DETRv2: Improved base- line with bag-of-freebies for real-time detection transformer

    Wenyu Lv, Yian Zhao, Qinyao Chang, Kui Huang, Guanzhong Wang, and Yi Liu. RT-DETRv2: Improved base- line with bag-of-freebies for real-time detection transformer. arXiv:2407.17140, 2024. 3, 6, 8

    2024 arXiv

  14. [22]

    Kirsten, Luis Felipe Zeni, and Claudio R

    Jeffri Murrugarra-Llerena, Lucas N. Kirsten, Luis Felipe Zeni, and Claudio R. Jung. Probabilistic intersection-over- union for training and evaluation of oriented object detectors. IEEE Trans. Image Process., 33:671–681, 2024. 4, 5

    2024

  15. [23]

    Indoor segmentation and support inference from RGBD images

    Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. Indoor segmentation and support inference from RGBD images. InEur. Conf. Comput. Vis., 2012. 6

    2012

  16. [24]

    PointNet: Deep learning on point sets for 3D classifica- tion and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning on point sets for 3D classifica- tion and segmentation. InIEEE Conf. Comput. Vis. Pattern Recog., pages 652–660, 2017. 3

    2017

  17. [25]

    Scene graph refinement network for vi- 9 sual question answering.IEEE Trans

    Tianwen Qian, Jingjing Chen, Shaoxiang Chen, Bo Wu, and Yu-Gang Jiang. Scene graph refinement network for vi- 9 sual question answering.IEEE Trans. Multimedia, 25:3950– 3961, 2022. 2

    2022

  18. [26]

    Kimera: From SLAM to spatial perception with 3D dynamic scene graphs.Int

    Antoni Rosinol, Andrew Violette, Marcus Abate, Nathan Hughes, Yun Chang, Jingnan Shi, Arjun Gupta, and Luca Carlone. Kimera: From SLAM to spatial perception with 3D dynamic scene graphs.Int. J. Robotics Research, 40(12- 14):1510–1546, 2021. 3

    2021

  19. [27]

    ViNG: Learning Open-World Navigation with Visual Goals

    Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, and Sergey Levine. ViNG: Learning Open-World Navigation with Visual Goals. InIEEE Int. Conf. Robotics Auto., 2021. 1

    2021

  20. [28]

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur- Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler G...

    1906 arXiv

  21. [29]

    Knowledge-based embodied question answer- ing.IEEE Trans

    Sinan Tan, Mengmeng Ge, Di Guo, Huaping Liu, and Fuchun Sun. Knowledge-based embodied question answer- ing.IEEE Trans. Pattern Anal. Mach. Intell., 45(10):11948– 11960, 2023. 2

    2023

  22. [30]

    RIO: 3D object instance re- localization in changing indoor environments

    Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, and Matthias Niessner. RIO: 3D object instance re- localization in changing indoor environments. InInt. Conf. Comput. Vis., 2019. 3, 5

    2019

  23. [31]

    Learning 3D semantic scene graphs from 3D in- door reconstructions

    Johanna Wald, Helisa Dhamo, Nassir Navab, and Federico Tombari. Learning 3D semantic scene graphs from 3D in- door reconstructions. InIEEE Conf. Comput. Vis. Pattern Recog., pages 3961–3970, 2020. 1, 2, 3, 5, 6, 7, 8

    2020

  24. [32]

    Learn- ing 3D semantic scene graphs with instance embeddings.Int

    Johanna Wald, Nassir Navab, and Federico Tombari. Learn- ing 3D semantic scene graphs with instance embeddings.Int. J. Comput. Vis., 130(3):630–651, 2022

    2022

  25. [33]

    VL-SAT: Visual-linguistic semantics assisted training for 3D semantic scene graph prediction in point cloud

    Ziqin Wang, Bowen Cheng, Lichen Zhao, Dong Xu, Yang Tang, and Lu Sheng. VL-SAT: Visual-linguistic semantics assisted training for 3D semantic scene graph prediction in point cloud. InIEEE Conf. Comput. Vis. Pattern Recog., pages 21560–21569, 2023. 2, 3

    2023

  26. [34]

    SceneGraphFusion: Incre- mental 3D scene graph prediction from rgb-d sequences

    Shun-Cheng Wu, Johanna Wald, Keisuke Tateno, Nassir Navab, and Federico Tombari. SceneGraphFusion: Incre- mental 3D scene graph prediction from rgb-d sequences. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7515–7525,

  27. [35]

    Incremental 3D semantic scene graph pre- diction from RGB sequences

    Shun-Cheng Wu, Keisuke Tateno, Nassir Navab, and Fed- erico Tombari. Incremental 3D semantic scene graph pre- diction from RGB sequences. InIEEE Conf. Comput. Vis. Pattern Recog., pages 5064–5074, 2023. 2, 3, 6, 7, 8, 1

    2023

  28. [36]

    Scene graph generation by iterative message passing

    Danfei Xu, Yuke Zhu, Christopher B Choy, and Li Fei-Fei. Scene graph generation by iterative message passing. In IEEE Conf. Comput. Vis. Pattern Recog., pages 5410–5419,

  29. [37]

    Auto-encoding scene graphs for image captioning

    Xu Yang, Kaihua Tang, Hanwang Zhang, and Jianfei Cai. Auto-encoding scene graphs for image captioning. InIEEE Conf. Comput. Vis. Pattern Recog., pages 10685–10694,

  30. [38]

    Neural Motifs: Scene graph parsing with global con- text

    Rowan Zellers, Mark Yatskar, Sam Thomson, and Yejin Choi. Neural Motifs: Scene graph parsing with global con- text. InIEEE Conf. Comput. Vis. Pattern Recog., pages 5831–5840, 2018. 2, 6, 1, 3

    2018

  31. [39]

    Commonscenes: Generating commonsense 3d indoor scenes with scene graph diffusion

    Guangyao Zhai, Evin Pınar ¨Ornek, Shun-Cheng Wu, Yan Di, Federico Tombari, Nassir Navab, and Benjamin Busam. Commonscenes: Generating commonsense 3d indoor scenes with scene graph diffusion. InAdv. Neural Inform. Process. Syst., 2023. 2

    2023

  32. [40]

    Echoscene: Indoor scene generation via information echo over scene graph diffusion

    Guangyao Zhai, Evin Pınar ¨Ornek, Dave Zhenyu Chen, Ruo- tong Liao, Yan Di, Nassir Navab, Federico Tombari, and Benjamin Busam. Echoscene: Indoor scene generation via information echo over scene graph diffusion. InEur. Conf. Comput. Vis., pages 167–184. Springer, 2024. 2

    2024

  33. [41]

    Exploiting edge-oriented reasoning for 3D point-based scene graph analysis

    Chaoyi Zhang, Jianhui Yu, Yang Song, and Weidong Cai. Exploiting edge-oriented reasoning for 3D point-based scene graph analysis. InIEEE Conf. Comput. Vis. Pattern Recog., pages 9705–9715, 2021. 2, 3, 6

    2021

  34. [42]

    EgoSG: Learning 3D scene graphs from egocentric RGB-D sequences

    Chaoyi Zhang, Xitong Yang, Ji Hou, Kris Kitani, Weidong Cai, and Fu-Jen Chu. EgoSG: Learning 3D scene graphs from egocentric RGB-D sequences. InIEEE Conf. Comput. Vis. Pattern Recog., pages 2535–2545, 2024

    2024

  35. [43]

    Knowledge- inspired 3D scene graph prediction in point cloud.Adv

    Shoulong Zhang, Aimin Hao, Hong Qin, et al. Knowledge- inspired 3D scene graph prediction in point cloud.Adv. Neu- ral Inform. Process. Syst., 34:18620–18632, 2021. 3, 6

    2021

  36. [44]

    DETRs beat YOLOs on real-time object detection

    Yian Zhao, Wenyu Lv, Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingqing Dang, Yi Liu, and Jie Chen. DETRs beat YOLOs on real-time object detection. InIEEE Conf. Comput. Vis. Pattern Recog., pages 16965–16974,

  37. [45]

    EditRoom: LLM-parameterized graph diffusion for composable 3D room layout editing

    Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Lin- jie Li, Zhengyuan Yang, Kevin Lin, Jianfeng Wang, Lijuan Wang, and Xin Eric Wang. EditRoom: LLM-parameterized graph diffusion for composable 3D room layout editing. arXiv:2410.12836, 2024. 2

    arXiv

  38. [46]

    EW A splatting.IEEE Trans

    Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. EW A splatting.IEEE Trans. Vis. Comput. Graph., 8(3):223–238, 2002. 4 10 FROSS: Faster-than-Real-Time Online 3D Semantic Scene Graph Generation from RGB-D Images Supplementary Material

    2002

  39. [47]

    The only difference is the exclusion of the ‘none’ relationship category, as FROSS does not predict it

    Detailed Evaluation Metric The evaluation procedure in this paper follows closely with Wu [35] to ensure a fair comparison. The only difference is the exclusion of the ‘none’ relationship category, as FROSS does not predict it. Wu [35] also provided results evaluated under thi...

  40. [48]

    near” and “on

    Additional Experimental Results 7.1. Object and Predicate Performance per Class The per-class performance comparison of FROSS and other baselines is presented in Tables 6 and 7. In addition, FROSS’s per-class object and predicate performance on the proposed ReplicaSSG dataset ...

  41. [49]

    More specifically, Figure 6 and 7 illustrate the occurrence frequency of objects and relation- ships across all categories in the dataset

    Statistics of the ReplicaSSG Dataset The statistics of the proposed ReplicaSSG Dataset are pre- sented in Figures 6-9. More specifically, Figure 6 and 7 illustrate the occurrence frequency of objects and relation- ships across all categories in the dataset. In addition, Fig- u...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.