Pith. sign in

REVIEW 3 major objections 5 minor 52 references

CoPa-SG: Dense Scene Graphs with Parametric and Proto-Relations

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CoPa-SG generates complete, precise scene graphs with 86M relations.

desk verdict A genuinely novel dataset with parametric and proto-relations, but the 'highly precise' claim needs a tighter predicate definition and human validation. read the letter →

arxiv 2506.21357 v1 pith:HXK4VUMU submitted 2025-06-26 cs.CV

Julian Lorenz , Mrunmai Phatak , Robin Schön , Katja Ludwig , Nico Hörmann , Annemarie Friedrich , Rainer Lienhart This is my paper
classification cs.CV
keywords scenegraphsparametricrelationsproto-relationssyntheticdatasetpanopticsegmentationspatialproceduralgenerationgraph
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to remove the data bottleneck in scene graph research by replacing human annotation with a deterministic pipeline that extracts every valid relation from a known 3D scene. It introduces CoPa-SG, a synthetic indoor dataset with more than 86 million relation annotations across 36,000 rendered views, and claims 100% coverage of subject-object pairs. To make relations unambiguous, it defines parametric relations that store an angle or distance alongside the predicate, and proto-relations that encode the volume a new object would need to occupy to satisfy a relation. If the pipeline is right, scene graph models can be trained and evaluated on exhaustive ground truth instead of sparse, salient-relation annotations, and downstream agents can answer placement and planning queries.

What carries the argument

The carrying mechanism is the deterministic relation extraction pipeline: scenes are voxelized at 1 cm resolution, distances between object surfaces are read from a K-d tree over voxels, and directional relations are decided by casting rays between an object's back surface and a subject's front surface along a test direction, keeping the smallest angular deviation. The two new representations—parametric relations (a predicate plus an angle or distance parameter) and proto-relations (voxel volumes storing the angle that a hypothetical object placement would produce)—are both outputs of this same machinery, and they are what allow the dataset to be both precise and queryable.

What would settle it

Sample a few hundred subject-object pairs from CoPa-SG, show the rendered views to several human annotators, and ask them to label one directional predicate such as 'right of' or 'behind'; if agreement between annotators and the dataset labels is low for camera-independent relations, the deterministic rules do not track human spatial semantics and the precision claim fails.

Watch

Extended reading notes

Core claim

The central claim is that a complete and precise scene graph can be generated automatically from any 3D scene by a fixed set of geometric rules, without subjective human interpretation. Each relation is a 6-tuple of subject, object, predicate, a real-valued parameter, camera perspective, and test direction; distance predicates store the shortest scene-space distance computed on a 1 cm voxel grid, while directional predicates store the minimum angle by which a ray deviates from a test direction after ray-cast front/back surface tests. Proto-relations extend this by recording, for every voxel, the smallest angle among rays crossing it, yielding a volume that predicts how a newly placed object would relate to an anchor. The resulting CoPa-SG contains more than 72,000 relations per scene on average and more than 86 million in total, making it, the authors argue, exhaustive rather than salient-biased.

Load-bearing premise

The load-bearing premise is that the fixed geometric rules for deciding when one object is 'behind,' 'right of,' 'next to,' or 'touching' another match what people would mean by those words, even though no human check of the resulting labels is reported.

Editorial extensions

If this is right

  • Scene graph benchmarks can move from top-k recall to exhaustive metrics: the paper shows that on CoPa-SG a perfect model would reach only 0.077 mR@50, so mAP and ng-mR@k are the suitable evaluation protocols.
  • Relation prediction becomes a two-output task of existence plus continuous parameter; the adapted DSFormer reaches roughly 15 degrees mean angle error and 1.15 m distance error on the test set.
  • Proto-relations give agents an explicit answer to 'where would this object satisfy a relation?', and because the volumes support constructive solid geometry operations, they compose into planning queries such as placing a lamp left of a shelf without obstructing a door.
  • Aggregating predictions over multiple views improves camera-independent relation AP by up to 6.2 points and saturates at about 15 views per scene.
  • Dense, queryable graphs can be reasoned over with a small language model that turns natural-language prompts into graph queries, enabling on-device reasoning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper does not develop is running the same extraction rules on real scanned 3D scenes with clean meshes; the decisive question is whether noisy geometry still supports stable front/back surface ray casts.
  • The angle and distance parameters suggest a continuous, compositional predicate semantics in which relations become functions of a test direction rather than discrete labels, which could change how spatial predicates are learned and transferred.
  • Proto-relations could be repurposed as feasibility or cost volumes for embodied agents, using the stored angles to score candidate placements; the paper demonstrates a reasoning query but does not test this planning use.
  • Because the discretized evaluation thresholds (10 degrees and 20 degrees, 1 m and 1.2 m) define positives and negatives, reported numbers may be threshold-sensitive; a continuous evaluation curve across thresholds would test how stable the benchmark conclusions are.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces CoPa-SG, a synthetic indoor scene graph dataset generated from Infinigen 3D scenes, together with a deterministic rule-based pipeline that extracts exhaustive spatial relations (more than 86M annotations) between all object instances. The paper also introduces two new relation representations: parametric relations, which attach a scalar angle or distance to a predicate, and proto-relations, which encode, as a voxel volume, the set of locations where a new object would satisfy a given relation to an anchor object. The authors adapt the DSFormer scene graph model to predict parametric relations, benchmark MotifNet, VCTree, and DSFormer on a thresholded variant of the dataset, and demonstrate an LLM/Cypher query framework and proto-relation CSG reasoning as downstream applications.

Significance. If the ground-truth quality claim holds, CoPa-SG would be a significant community resource: it is orders of magnitude larger than existing scene graph datasets, covers all subject-object pairs per scene rather than only salient relations, provides depth, normals, and multiple views, and ships a reproducible deterministic extraction pipeline as well as inference code and weights. The parametric and proto-relation notions are genuinely new and could be useful for planning and embodied-agent applications. However, the central claim of the paper is that the dataset is 'highly precise ground truth'; that claim is not yet convincingly validated, for the concrete reasons detailed below. The stress-test concern about the existential ray test does land, and the extraction rule for the 'on' predicate is absent, so the current manuscript overstates the reliability and completeness of the released annotations.

major comments (3)
  1. [Sec. 3.1 (directional relations)] The extraction procedure accepts any single ray from the back surface of obj that hits the front surface of sbj, with angular deviations up to 90 degrees allowed, and stores only the minimum deviation as the parameter alpha. Consequently, the raw 86M annotations include positive relations such as 'behind (85 degrees)' or 'right of (80 degrees)', where the binary predicate label is semantically ambiguous and the stored parameter records a deviation only after the relation has been binarized. The thresholded variant in Sec. 5.2 removes only the 10-20 degree band at evaluation time; it does not clean the raw resource. The abstract and Sec. 1 claim 'highly precise ground truth' for the released dataset, but that claim is not supported without either thresholding the raw positives to a validated alpha range or providing a quantitative validation of the raw labels (e.g., human agreement, manual inspection, or a distributional analysis of alpha and its effect on model rankings). I request one of these as a prerequisite for the precision claim.
  2. [Sec. 3.1 (on predicate)] The predicate 'on' appears in the predicate set in Sec. 3.1, in the distribution of Fig. 6, and in the experimental tables (Tables 2 and 3), but the extraction rules described in Sec. 3.1 cover only distance-based relations and directional relations. The set P \ (A union D) is said to contain predicates without parameters, with 'on' as the example, yet no geometric rule is given for when 'on' holds. Without this definition, the completeness claim for the dataset is not checkable and the benchmark cannot be reproduced. Please specify the extraction rule for 'on' (e.g., surface contact under a gravity direction, support relations, or a tolerance threshold) and state whether it is camera-dependent.
  3. [Sec. 5.2 (thresholded variant)] The thresholded variant uses angle <= 10 degrees positive, angle > 20 degrees negative, distance <= 1 m positive, and distance > 1.2 m negative. These thresholds are introduced without justification or sensitivity analysis, and they interact with the existential ray test: a ray at 9.5 degrees is a positive relation while a ray at 20.5 degrees is negative, so the benchmark notion of 'relation' is a step function of an arbitrarily chosen cutoff. Since the paper proposes CoPa-SG as a reference benchmark for training and evaluation, the threshold choice should be derived from data or validated, and the sensitivity of the reported model rankings to these thresholds should be reported.
minor comments (5)
  1. [Sec. 1, contribution 4] The text reads 'more than 86M million relation annotations'; this should be '86M' or '86 million'.
  2. [Sec. 4, Eq. (1)-(2)] The sentence 'we apply the sigmoid function to the respective output before calculating LD' appears to refer to the angle loss LA, not LD; the notation should be corrected.
  3. [Sec. 4, Eq. (1)-(2)] The losses are normalized by the sum of positive flags; if a batch contains no positive instance for some predicate, division by zero occurs. Please specify the numerical safeguard used during training.
  4. [Table 3] The distance parameter error is reported as 1.15 m for all architecture variants; this constant value should be explained (e.g., whether distance parameters were trained, whether the reported value is dominated by a single predicate, or whether it is a baseline value).
  5. [Sec. 3.3 / Table 1] The 'Coverage' definition in Table 1, 'how many of all possible subject-object pairs are covered by at least one relation annotation', should be made precise regarding which subject-object pairs count (e.g., all pairs in the 3D scene, or only pairs visible in at least one view); as written, 100% coverage for CoPa-SG is tautological.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: dataset labels are produced by deterministic geometry rules and model benchmarking is a standard train/test loop.

full rationale

The paper's contributions are a synthetic dataset and two new relation representations, not a derivation of predictions from fitted inputs. The extraction pipeline in Sec. 3.1 converts voxelized 3D geometry into (predicate, parameter) labels using deterministic ray-casting rules; no parameter is fitted to a subset of labels and then re-predicted. The parametric relation's alpha is computed directly from the ray geometry, and the model experiments in Secs. 5.2-5.3 train on one split and evaluate on a disjoint split of the same procedurally generated distribution, which is a standard benchmark loop rather than a circular derivation. The only self-citations ([19], [20]) refer to prior datasets or models used as baselines or as the architecture being adapted; they do not justify the correctness of the labels or the value of any extracted parameter. Proto-relations (Sec. 3.2) are introduced by definition: the volume is constructed as the set of voxels passed by rays satisfying the same angular test, so the stated property that any new object intersecting the volume would fulfill the relation holds analytically by construction, not as an empirical prediction. Consequently, no load-bearing step reduces to its own input. Concerns about 90-degree-deviation labels or the unspecified rule for the 'on' predicate are validity or documentation issues, not circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 2 invented entities

The CoPa-SG ground truth depends on a set of hand-chosen thresholds and semantic assumptions. The evaluation thresholds in Sec 5.2 (10/20 degrees, 1/1.2 m) are free parameters. The rule-based predicate definitions and object-orientation mapping are domain assumptions not validated externally. The dataset's claimed precision rests on these choices.

free parameters (6)
  • Angle threshold for positive relation = 10 degrees
    Used in Sec 5.2 to convert parametric relations into positive labels for evaluation.
  • Angle threshold for negative relation = 20 degrees
    In Sec 5.2, relations with angle >20 degrees count as negative.
  • Distance threshold for next to positive = 1 m
    In Sec 5.2, next to relations with distance <=1 m count as positive.
  • Distance threshold for next to negative = 1.2 m
    In Sec 5.2, distances >1.2 m count as negative.
  • Voxel resolution = 1 cm
    Chosen for distance estimation and proto-relation volumes; limits distance precision in Sec 3.1.
  • Ray resolution = 0.05 cm at 6 m
    Ray cast density in Sec 3.1; affects angular precision of directional relations.
assumptions (4)
  • domain assumption Spatial predicate definitions in Sec 3.1 match human semantics
    The paper defines 'in front of', 'behind', etc. with ray-cast front/back surfaces, but never validates against human annotations.
  • ad hoc to paper Typical object orientations for camera-independent relations
    Sec 3.1 and supplementary Sec 8 assign a fixed 'front' direction to categories like shelf and TV to define camera-independent predicates.
  • domain assumption Synthetic Infinigen scenes are representative for benchmark and downstream use
    In Sec 1 and Sec 3.3, the dataset is built from Infinigen; transfer to real images is not evaluated.
  • domain assumption Voxelization at 1 cm preserves relevant geometry
    Sec 3.1 uses a 1 cm voxel grid for distances; thin objects or sub-centimeter gaps may be mis-measured.
invented entities (2)
  • Parametric relation (predicate plus scalar parameter) independent evidence
    purpose: Represents relations with a continuous angle or distance instead of a bare label.
    The parameter values are defined by the geometric pipeline and could be verified by direct measurement in the 3D scene, though no external validation is reported.
  • Proto-relation volume independent evidence
    purpose: Encodes the region where a new object would satisfy a given relation to an anchor object.
    The volume is a concrete region in space; placing an object inside it is a checkable prediction, so it is falsifiable by construction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoPa-SG: Dense Scene Graphs with Parametric and Proto-Relations." pith.science (2026). https://pith.science/paper/HXK4VUMU

@misc{pith2026250621357,
  author       = {Pith},
  title        = {Pith review of: CoPa-SG: Dense Scene Graphs with Parametric and Proto-Relations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HXK4VUMU}},
  note         = {Machine review of arXiv:2506.21357}
}
read the original abstract

2D scene graphs provide a structural and explainable framework for scene understanding. However, current work still struggles with the lack of accurate scene graph data. To overcome this data bottleneck, we present CoPa-SG, a synthetic scene graph dataset with highly precise ground truth and exhaustive relation annotations between all objects. Moreover, we introduce parametric and proto-relations, two new fundamental concepts for scene graphs. The former provides a much more fine-grained representation than its traditional counterpart by enriching relations with additional parameters such as angles or distances. The latter encodes hypothetical relations in a scene graph and describes how relations would form if new objects are placed in the scene. Using CoPa-SG, we compare the performance of various scene graph generation models. We demonstrate how our new relation types can be integrated in downstream applications to enhance planning and reasoning capabilities.

Figures

Figures reproduced from arXiv: 2506.21357 by the authors.

Figure 1
Figure 1. COPA-SG contains exhaustive scene graph annotations. In contrast, traditional scene graphs focus mostly on salient rela￾tions. This makes our dataset more suitable for training and eval￾uating scene graph generation models that will be employed in downstream applications. Note that the magnified graph displays only a subset of the available relations for better readability. plete and miss many valid relations. The p… view at source ↗
Figure 3
Figure 3. The ray sweep technique to determine back surfaces and [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Schematic of the proto-relation extraction process. Sim [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Example proto-relation (right of) for the TV board. The area highlighted in red shows the associated volume for the proto￾relation. Any new object placed inside the volume would fulfil a right of relation with regard to the TV board with an angle devia￾tion of 0 degree…
Figure 7
Figure 7. Figure 7: Qualitative prediction results on C [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Example interaction with Qwen2.5 [40] on our dataset. The LLM returns a Cypher query which is executed to return the final results. Using the provided functions and Python's arithmetic operators, the solution is as follows: ```python left(obj_f3ab68) & next_to(obj_f3ab…
Figure 9
Figure 9. Figure 9: Example interaction with Phi4 [1] for proto-relation queries. The language model returns a subset of Python that is interpreted as set of constructive solid geometry operations. available functions, each associated with a proto-relation category (e.g. in front of, touc…
Figure 10
Figure 10. Figure 10: Test directions for camera-independent relations that [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Predicate class distribution in the overall dataset. All [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 43 canonical work pages

  1. [1]

    Hewett, Mojan Javaheripi, Piero Kauffmann, James R

    Marah Abdin, Jyoti Aneja, Harkirat Behl, S ´ebastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Rus- sell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli...

    work page 2024

  2. [2]

    Scenescript: Reconstructing scenes with an autoregressive structured language model, 2024

    Armen Avetisyan, Christopher Xie, Henry Howard-Jenkins, Tsun-Yi Yang, Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Campbell Orme, Jakob Engel, Edward Miller, Richard Newcombe, and Vasileios Balntas. Scenescript: Reconstructing scenes with an autoregressive structured language model, 2024. 3

    work page 2024

  3. [3]

    Matterport3d: Learning from rgb-d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158, 2017. 3

    arXiv 2017

  4. [4]

    Knowledge-embedded routing network for scene graph gen- eration

    Tianshui Chen, Weihao Yu, Riquan Chen, and Liang Lin. Knowledge-embedded routing network for scene graph gen- eration. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6156–6164, 2019. 6

    work page 2019

  5. [5]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 3

    2017

  6. [6]

    Procthor: Large-scale embodied ai using procedural genera- tion, 2022

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Eric Kolve, Ali Farhadi, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural genera- tion, 2022. 3

    work page 2022

  7. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representa- tions, 2021. 7

    work page 2021

  8. [8]

    Viewer-centered object represen- tation in the human visual system revealed by viewpoint af- tereffects

    Fang Fang and Sheng He. Viewer-centered object represen- tation in the human visual system revealed by viewpoint af- tereffects. Neuron, 45(5):793–800, 2005. 4

    work page 2005

Show all 52 references
  1. [9]

    Cypher: An evolving query language for property graphs

    Nadime Francis, Alastair Green, Paolo Guagliardo, Leonid Libkin, Tobias Lindaaker, Victor Marsault, Stefan Plantikow, Mats Rydberg, Petra Selmer, and Andr ´es Taylor. Cypher: An evolving query language for property graphs. In SIG- MOD’18 Proceedings of the 2018 International C...

    2018

  2. [10]

    Fast r-cnn

    Ross Girshick. Fast r-cnn. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 1440–1448,

    2015

  3. [11]

    Tenenbaum, Antonio Torralba, Florian Shkurti, and Liam Paull

    Qiao Gu, Alihusein Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Cor- ban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, Chuang Gan, Celso Miguel de Melo, Joshua B. Tenenbaum, Antonio Torralba, Florian Shkurti, and Liam Paull. Conce...

    2023

  4. [12]

    Scenenet: An annotated model generator for indoor scene understanding

    Ankur Handa, Viorica P ˘atr˘aucean, Simon Stent, and Roberto Cipolla. Scenenet: An annotated model generator for indoor scene understanding. In 2016 IEEE International Confer- ence on Robotics and Automation (ICRA), pages 5737–5743,

    2016

  5. [13]

    Scene graph reasoning for visual ques- tion answering, 2020

    Marcel Hildebrandt, Hang Li, Rajat Koner, V olker Tresp, and Stephan G¨unnemann. Scene graph reasoning for visual ques- tion answering, 2020. 1

    2020

  6. [14]

    Explainable and ex- plicit visual reasoning over scene graphs

    Juanzi Li Jiaxin Shi, Hanwang Zhang. Explainable and ex- plicit visual reasoning over scene graphs. In CVPR, 2019. 1

    2019

  7. [15]

    Sapiens: Foundation for human vision mod- els, 2024

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els, 2024. 1

    2024

  8. [16]

    Shamma, Michael S

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. ...

    2017

  9. [17]

    Panoptic scene graph generation with semantics-prototype learning, 2024

    Li Li, Wei Ji, Yiming Wu, Mengze Li, You Qin, Lina Wei, and Roger Zimmermann. Panoptic scene graph generation with semantics-prototype learning, 2024. 1

    2024

  10. [18]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceeding...

    2014

  11. [19]

    Haystack: A panoptic scene graph dataset to eval- uate rare predicate classes

    Julian Lorenz, Florian Barthel, Daniel Kienzle, and Rainer Lienhart. Haystack: A panoptic scene graph dataset to eval- uate rare predicate classes. In 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023. 3, 6

    2023

  12. [20]

    A fair ranking and new model for panoptic scene graph generation, 2024

    Julian Lorenz, Alexander Pest, Daniel Kienzle, Katja Lud- wig, and Rainer Lienhart. A fair ranking and new model for panoptic scene graph generation, 2024. 3, 5, 6

    2024

  13. [21]

    Visual relationship detection with language priors, 2016

    Cewu Lu, Ranjay Krishna, Michael Bernstein, and Li Fei- Fei. Visual relationship detection with language priors, 2016. 6 9

    2016

  14. [22]

    Visual relationship detection with language priors

    Cewu Lu, Ranjay Krishna, Michael Bernstein, and Li Fei- Fei. Visual relationship detection with language priors. In Computer Vision – ECCV 2016 , pages 852–869, Cham,

    2016

  15. [23]

    Aria synthetic environments dataset

    Meta. Aria synthetic environments dataset. https:// www.projectaria.com/datasets/ase, 2024. 3

    2024

  16. [24]

    Openvdb: an open-source data structure and toolkit for high-resolution volumes

    Ken Museth, Jeff Lait, John Johanson, Jeff Budsberg, Ron Henderson, Mihai Alden, Peter Cucka, David Hill, and An- drew Pearce. Openvdb: an open-source data structure and toolkit for high-resolution volumes. In ACM SIGGRAPH 2013 Courses, New York, NY , USA, 2013. Association fo...

    2013

  17. [25]

    Neo4j graph database, 2025

    Neo4j, Inc. Neo4j graph database, 2025. Accessed: 2025- 03-05. 8

    2025

  18. [26]

    Pixels to graphs by asso- ciative embedding

    Alejandro Newell and Jia Deng. Pixels to graphs by asso- ciative embedding. In Proceedings of the 31st International Conference on Neural Information Processing Systems, page 2168–2177, Red Hook, NY , USA, 2017. Curran Associates Inc. 6

    2017

  19. [27]

    Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Rus- sell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Ni...

    2023

  20. [28]

    Infinite photore- alistic worlds using procedural generation

    Alexander Raistrick, Lahav Lipson, Zeyu Ma, Lingjie Mei, Mingzhe Wang, Yiming Zuo, Karhan Kayan, Hongyu Wen, Beining Han, Yihan Wang, Alejandro Newell, Hei Law, Ankit Goyal, Kaiyu Yang, and Jia Deng. Infinite photore- alistic worlds using procedural generation. In Proceedings ...

    2023

  21. [29]

    Infinigen indoors: Photorealistic in- door scenes using procedural generation

    Alexander Raistrick, Lingjie Mei, Karhan Kayan, David Yan, Yiming Zuo, Beining Han, Hongyu Wen, Meenal Parakh, Stamatis Alexandropoulos, Lahav Lipson, Zeyu Ma, and Jia Deng. Infinigen indoors: Photorealistic in- door scenes using procedural generation. In Proceedings of the IE...

    2024

  22. [30]

    Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai

    Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Un- dersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv prepri...

    2021 arXiv

  23. [31]

    Hwang, and Martha Palmer

    Nathan Schneider, Vivek Srikumar, Jena D. Hwang, and Martha Palmer. A hierarchy with, of, and for preposition supersenses. In Proceedings of the 9th Linguistic Annotation Workshop, pages 112–123, Denver, Colorado, USA, 2015. Association for Computational Linguistics. 1

    2015

  24. [32]

    Hwang, Vivek Srikumar, Jakob Prange, Austin Blodgett, Sarah R

    Nathan Schneider, Jena D. Hwang, Vivek Srikumar, Jakob Prange, Austin Blodgett, Sarah R. Moeller, Aviram Stern, Adi Bitan, and Omri Abend. Comprehensive supersense dis- ambiguation of English prepositions and possessives. InPro- ceedings of the 56th Annual Meeting of the Assoc...

    2018

  25. [33]

    Scene graph contrastive learning for embodied navigation

    Kunal Pratap Singh, Jordi Salvador, Luca Weihs, and Aniruddha Kembhavi. Scene graph contrastive learning for embodied navigation. In 2023 IEEE/CVF International Con- ference on Computer Vision (ICCV) , pages 10850–10860,

    2023

  26. [34]

    Learning to compose dynamic tree structures for visual contexts

    Kaihua Tang, Hanwang Zhang, Baoyuan Wu, Wenhan Luo, and Wei Liu. Learning to compose dynamic tree structures for visual contexts. In Conference on Computer Vision and Pattern Recognition, 2019. 3, 6, 7

    2019

  27. [35]

    The semantics of English prepositions: Spatial scenes, embodied meaning, and cogni- tion

    Andrea Tyler and Vyvyan Evans. The semantics of English prepositions: Spatial scenes, embodied meaning, and cogni- tion. Cambridge University Press, 2003. 1

    2003

  28. [36]

    Unity, 2025

    Unity Technologies. Unity, 2025. 3

    2025

  29. [37]

    Rio: 3d object instance re- localization in changing indoor environments

    Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, and Matthias Nießner. Rio: 3d object instance re- localization in changing indoor environments. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 7658–7667, 2019. 3

    2019

  30. [38]

    Learning 3d semantic scene graphs from 3d in- door reconstructions

    Johanna Wald, Helisa Dhamo, Nassir Navab, and Federico Tombari. Learning 3d semantic scene graphs from 3d in- door reconstructions. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3961–3970, 2020. 3

    2020

  31. [39]

    The all-seeing project: Towards panop- tic visual recognition and understanding of the open world

    Weiyun Wang, Min Shi, Qingyun Li, Wenhai Wang, Zhen- hang Huang, Linjie Xing, Zhe Chen, Hao Li, Xizhou Zhu, Zhiguo Cao, et al. The all-seeing project: Towards panop- tic visual recognition and understanding of the open world. arXiv preprint arXiv:2308.01907, 2023. 1, 2

    2023 arXiv

  32. [40]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  33. [41]

    Panoptic scene graph gen- eration

    Jingkang Yang, Yi Zhe Ang, Zujin Guo, Kaiyang Zhou, Wayne Zhang, and Ziwei Liu. Panoptic scene graph gen- eration. In ECCV, 2022. 1, 2, 7

    2022

  34. [42]

    Depth any- thing v2, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2, 2024. 1

    2024

  35. [43]

    Neural motifs: Scene graph parsing with global con- text, 2018

    Rowan Zellers, Mark Yatskar, Sam Thomson, and Yejin Choi. Neural motifs: Scene graph parsing with global con- text, 2018. 3, 6, 7

    2018

  36. [44]

    Fine-grained scene graph generation with data transfer, 2022

    Ao Zhang, Yuan Yao, Qianyu Chen, Wei Ji, Zhiyuan Liu, Maosong Sun, and Tat-Seng Chua. Fine-grained scene graph generation with data transfer, 2022. 1

    2022

  37. [45]

    Vla-3d: A dataset for 3d semantic scene understanding and navigation.arXiv preprint arXiv:2411.03540, 2024

    Haochen Zhang, Nader Zantout, Pujith Kachana, Zongyuan Wu, Ji Zhang, and Wenshan Wang. Vla-3d: A dataset for 3d semantic scene understanding and navigation.arXiv preprint arXiv:2411.03540, 2024. 3 10

    2024 arXiv

  38. [46]

    Multiview scene graph

    Juexiao Zhang, Gao Zhu, Sihang Li, Xinhao Liu, Haorui Song, Xinran Tang, and Chen Feng. Multiview scene graph. In The Thirty-eighth Annual Conference on Neural Informa- tion Processing Systems, 2024. 8

    2024

  39. [47]

    Open3D: A modern library for 3D data processing

    Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3D: A modern library for 3D data processing. arXiv:1801.09847,

    arXiv

  40. [48]

    HiLo: Ex- ploiting high low frequency relations for unbiased panoptic scene graph generation

    Zijian Zhou, Miaojing Shi, and Holger Caesar. HiLo: Ex- ploiting high low frequency relations for unbiased panoptic scene graph generation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pages 21637–21648, 2023. 1 11 COPA-SG: Dense Scene ...

    2023

  41. [50]

    Directional Relations Camera-independent relations are defined based on the pose of the associated object. Fig. 10 shows how the various directions are defined with respect to the object’s pose. Figure 10. Test directions for camera-independent relations that depend on the object pose

  42. [51]

    Directional

    Infinigen Object Mapping Since Infinigen [29] does not include a set of class labels for the various objects, we create a mapping from object instance name to class label. You can see the mapping in Tabs. 5 and 6. The “Directional” column specifies whether the respective objec...

  43. [52]

    Predicate Distribution in COPA-SG We define a fixed split of C OPA-SG into training, valida- tion, and test. Fig. 11 shows the distribution of the con- tained predicate classes among the different data splits. left right behind front above below left right behind front touchin...

  44. [2016]

    Springer International Publishing. 1, 6

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.