{"id":"9c7cc055-7930-4bd4-97fe-1b31a72764f4","arxiv_id":"2506.19639","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"HOIverse is a synthetic indoor scene graph dataset with dense human-object interaction and parametric relation annotations, benchmarked with scene graph generation models.","lead":"HOIverse is a new synthetic dataset of indoor scenes with dense, procedurally generated annotations linking humans to objects and objects to each other, including parametric spatial relations and human keypoints. The paper positions it as a resource for scene understanding in human-robot collaboration and benchmarks three scene graph models on it.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Procedural annotations may not match real interactions: no contact/gaze validation, so the claim of error-free ground truth is unsubstantiated.","rationale":"The reader's weakest assumption is precisely the load-bearing concern: the procedural pipeline assumes that geometric placement and ray casting produce semantically meaningful interactions. This is central because the paper's headline claim is 'accurate and unambiguous ground truth' and 'error-free' data, but the procedure is unvalidated. If a person labeled 'sitting on' is not actually in contact with the chair, all downstream benchmarks on HOIverse are compromised. The paper offers no human validation, no contact checks, and no distribution analysis of the parametric relation parameters (e.g., the ray-deviation angles for 'looking at'), which would reveal whether relations are semantically plausible. This is a correctness risk, not merely a reproducibility issue, and it supports a conditional verdict: the authors should release code/data or provide validation studies demonstrating that the procedural relations align with genuine human-object interactions. The reader already reached CONDITIONAL, and this concern reinforces that verdict without moving it to REJECT, because the issue is empirically testable and addressable in revision. I agree with the reader's assessment, and I would keep the verdict unchanged while emphasizing the need for the proposed validation.","tokens_in":9432,"tokens_out":4565,"duration_ms":49820,"concrete_test":"Download or request a subset of HOIverse scenes with code, and for every annotated 'sitting on', 'touching' and 'holding' relation compute the minimal Euclidean distance between the SMPL-X human mesh and the target object mesh from the stored scene geometry (or rerun the pipeline in Infinigen). Count how many positive relations have distance > 2 cm with no overlap. If the failure rate exceeds 5% of the sampled instances, the pose-to-interaction placement assumption is refuted and the ground truth is not error-free.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central value proposition is that HOIverse provides 'accurate and unambiguous' ground truth relations. But in the Human Populator (Sec. III-B), humans are placed by sampling predefined poses based only on free floor space and object presence; the paper does not verify that the body actually contacts or is supported by the target object. In the Relation Annotation (Sec. III-D), 'sitting on', 'touching', 'holding' etc. are assigned from the pose label rather than from a geometric contact test, and 'looking at'/'pointing at' are computed by ray casts with an allowed deviation up to 90 degrees. If a pose is misaligned (e.g., a 'sitting' human is a few cm in front of the chair, or the gaze ray hits a wall instead of the intended object), the ground truth will be silently wrong. The paper states (Sec. III-B) that it strives to generate 'reliable, correct (i.e. error-free) and precise ground truth data,' yet no human evaluation, contact check, or parameter distribution analysis is provided. Because the dataset's raison d'être is annotation accuracy, this unvalidated assumption is the most load-bearing weakness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"HOIverse is a synthetic dataset for scene graph generation with human-object interactions, built by extending the CoPa-SG pipeline to Infinigen indoor scenes. Humans are placed deterministically using a set of predefined SMPL-X poses, and relations between all human-object and object-object pairs are computed procedurally as parametric relations, including three new relations (looking at, pointing at, body facing). The dataset provides RGB, depth, segmentation, human keypoints, first-person viewpoints, and negative ground truth, with roughly 16.2k images, 40k interaction annotations, and 22M scene graph relations. The paper also reports predicate-classification results for MotifNet, VCTree, and DSFormer on this dataset.","tokens_in":9680,"tokens_out":3535,"duration_ms":38057,"significance":"If the annotation pipeline is trustworthy, HOIverse fills a real gap: it combines scene graph generation with dense human-object interaction annotations in indoor scenes, offers procedural ground truth that avoids manual annotation inconsistency, and includes modalities (keypoints, egocentric views, negative relations) that are useful for human-robot collaboration and scene understanding. The deterministic pipeline, public dataset availability, and explicit parametric relation design are strengths. However, the central value proposition, accurate and unambiguous ground truth, is not yet demonstrated, and the evaluation protocol is too under-specified to support the benchmark conclusions. The significance is therefore conditional on validation of the annotation semantics and on making the relation definitions reproducible.","major_comments":[{"comment":"The core claim that HOIverse provides 'accurate' and 'error-free' ground truth is not supported by any validation of the procedural annotation. The human populator places predefined poses based on free floor space and object presence, and the relation annotator assigns interactions such as 'sitting on' and 'holding' from the pose label rather than from contact/support or grasp checks. A 'sitting' human placed a few centimeters in front of a chair, or a 'looking at' ray that terminates on a wall within the 90-degree cone, would be silently annotated as correct. I recommend adding a geometric contact verification (e.g., body-mesh/object intersection or support-plane test), a human evaluation of a random sample, and an error analysis of the populator (e.g., percentage of placements with unintended penetrations or missed contacts).","section":"III-B and III-D"},{"comment":"The annotation definitions are not self-contained. The paper states that it follows CoPa-SG [8] for parametric relations, but [8] is under review and the supplementary PDF is not available to the reader. Load-bearing details, such as the exact ray-casting procedure, the parameter ranges for each relation, and the positive/negative thresholds (angle <= 10 degrees / >= 20 degrees, distance <= 1 m / >= 1.2 m in Section IV-B), are either inherited from an inaccessible source or introduced only in the evaluation section. Please provide the complete relation definitions, thresholds, and code in a public repository so the annotations can be reproduced and assessed.","section":"III-A, III-D, and IV-B"},{"comment":"The evaluation protocol is under-specified and the benchmark numbers are not interpretable without additional detail. 'ng-mR' is introduced as 'No Graph Constraint Mean Recall@k' and is said to be computed over 1000 predicate relations, but no formula, no k value, and no averaging procedure are given; the mAP computation is likewise not described. The table reports no variance or number of runs, so the difference between MotifNet (0.504) and VCTree (0.511) mAP cannot be assessed. Please specify the metrics exactly and report mean +/- standard deviation over at least three seeds.","section":"IV-B and Table II"}],"minor_comments":[{"comment":"The text mentions that models achieve '0.020 and 0.032 mR@50 and mR@100 respectively,' but these values are not shown in Table II; either include them or remove the sentence.","section":"IV-B"},{"comment":"Figures 3, 4, 5, and 6 are not referenced in the body text; please add explicit references.","section":"Figures"},{"comment":"There is a typo in the looking-at description: 'everything above is not regarded aslooking at' should be 'as looking at'.","section":"III-D"},{"comment":"Please define precisely what counts as a 'relation' and an 'interaction annotation' when reporting 40k interaction annotations and 22M scene graph annotations, since these numbers depend on whether camera-dependent duplicates and negative relations are included.","section":"IV-A"},{"comment":"The dataset website is given, but the paper does not specify a license, versioning scheme, or download procedure; please add these details.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's reliance on CoPa-SG [8], an under-review paper by the same group, is a reviewability concern. If the CoPa-SG definitions are not made available to the reviewers, the editor should ask the authors for the supplementary material or a public code release containing the full relation definitions and thresholds. The benchmark section also needs a precise metric definition and variance reporting before the numbers can be considered evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about HOIverse. First, it is a real addition to the synthetic-scene-graph space: a dense, parametric dataset with 22M relations, egocentric viewpoints per human, and human keypoints, built by extending the authors' under-review CoPa-SG pipeline with humans and three new human-centric relations (looking at, pointing at, body facing). That fills a genuine gap, and the deterministic pipeline is a sensible choice for a ground-truth resource. Second, the paper's core value proposition — \"accurate and unambiguous\" ground truth — is not yet supported. The stress-test note is on target: the human populator places predefined poses using floor-space and object-presence checks, but never verifies contact. A \"sitting\" human can be a few centimeters off the chair, and the relation is still recorded as sitting on. Similarly, looking at and pointing at use ray casts with an allowed deviation up to 90 degrees; without a human evaluation or even a sampling of failure cases, the claim of error-free annotations is an overclaim. This is not a minor issue, because the dataset's reason to exist is annotation quality. That said, the stress-test concern does not sink the paper; it is an addressable validation gap, not an internal contradiction. What the paper does well: the reported statistics are concrete (525 scenes, 16.2k images, 40k interaction annotations), the figures make the annotation types clear, and the benchmark — while basic — gives a first read on model performance. The authors also honestly note that mR@k is not suitable for their dense multi-label setting, which is a fair methodological point. The soft spots beyond validation: the benchmark lacks variance, the metric definitions (e.g., ng-mR over 1000 relations) are underspecified, and the only comparison is among three models with no cross-dataset evaluation. The reliance on CoPa-SG, an under-review paper by the same group, makes independent assessment awkward, though not wrong. And while the website link is given, the paper ships no code or data artifacts, so I could not verify the 22M count or the pipeline in practice. Who is this for: researchers building or evaluating scene-graph and HOI models, especially for embodied AI or human-robot collaboration. It deserves a serious referee: the dataset is novel, the extension is meaningful, and the flaws are fixable. My recommendation: send it to peer review, but the revision must include a validation study — at least a few hundred human judgments on the procedural relations, plus a contact/gaze sanity check and a clearer account of the ray-cast and pose-alignment failure modes.","headline":"HOIverse is a genuinely new dense synthetic HOI-scene-graph dataset, but the claim of error-free ground truth is not yet backed by validation, and the paper's load-bearing weakness is exactly the one the stress-test flags.","tokens_in":779,"tokens_out":953,"would_cite":false,"duration_ms":26141,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A synthetic dataset called HOIverse claims to be the first scene graph dataset with accurate, procedurally generated ground truth for human-object interactions in indoor scenes.","keywords":["scene graphs","human-object interaction","synthetic dataset","parametric relations","indoor scene understanding","human-robot collaboration","egocentric vision","ground truth annotation"],"falsifier":"Sample a random set of HOIverse frames labeled with contact or gaze relations and check ground truth directly: does the human mesh intersect or touch the chair for sitting on, and does the head ray's hit object for looking at match a human gaze judgment? If a substantial fraction of sampled labeled relations fail such checks, the dataset's claim of accurate ground truth would be contradicted.","tokens_in":9264,"feed_emoji":"🤖","tokens_out":8914,"duration_ms":86929,"temperature":0.7,"pith_summary":"HOIverse is a synthetic dataset built to supply complete scene graph annotations for indoor scenes that contain people. The authors claim it is the first synthetic scene graph dataset specifically for human-object interactions, providing dense and consistent ground truth computed by a procedural pipeline instead of manual labeling: every human-object and object-object pair in a scene is assigned parametric relations with distance or angle parameters, plus human-specific relations such as looking at, pointing at, and body facing. Across 525 procedurally generated scenes and roughly 16.2k rendered images, the dataset contains about 22 million relation annotations, with an average of 1.35k relations per camera view, and it includes RGB, segmentation, depth, and human keypoints. A benchmark of three current scene graph generation models on the dataset shows that dense multi-label evaluation requires metrics beyond standard Recall@k.","feed_headline":"Synthetic scenes map 22 million human-object relations","feed_subtitle":"HOIverse supplies dense, consistent scene graph labels for indoor scenes and benchmarks scene graph models on them.","key_machinery":"The mechanism that carries the argument is the procedural relation-annotation pipeline. Spatial relations among objects and between humans and objects are computed from 3D geometry using volume-based overlap checks, while the human-specific relations looking at, pointing at, and body facing are extracted by casting rays from a person's head, index-finger, and spine joints and recording the ray with the smallest angular deviation from the viewing or pointing direction as a parameter. A deterministic rule set, rather than human annotators, decides every (subject, relation, object) triplet, which is exactly what underwrites the dataset's claim to complete and consistent annotations.","core_discovery":"The paper's claim is that complete and consistent scene graph ground truth for human-object interaction can be produced synthetically, avoiding the sparse, salient-only, and sometimes inconsistent annotations of manually labeled datasets. HOIverse populates each generated indoor scene with two to six humans in fourteen predefined interaction poses and then computes relations for every human-object and object-object pair procedurally. Relations are parametric: a predicate such as next to or looking at carries a measured distance or angle, so its meaning is fixed by geometry rather than by an annotator's judgment. The authors further claim this makes HOIverse the first synthetic scene graph dataset that describes human-object interactions with accurate ground truths for indoor 3D scenes, and they show existing scene graph generation models can be trained on it, with the best mean average precision achieved by a two-stage model that decouples detection from predicate classification.","pith_inferences":["Editorial extension: because the annotation rules are procedural, swapping a human pose or an object in a generated scene would produce paired counterfactual ground truths; models that rely on label correlations rather than genuine interaction evidence could be exposed by testing whether their predictions flip accordingly.","Editorial extension: the ray-based definition of looking at does not model gaze as humans actually direct it, so a behavioral check against human judgments of gaze would clarify what the relation means and whether downstream robot planning should trust it.","Editorial extension: the dataset currently centers on single-human-to-object relations; adding relations between two people, such as handing an object or pointing for another person, would be a natural next step for human-robot collaboration scenarios."],"forward_implications":["Scene graph generation models trained on exhaustively annotated data cannot be judged by standard Recall@k, so the paper reports no-graph-constraint mean recall over 1000 predicates and mean average precision, with the best mAP achieved by a two-stage model that separates detection from predicate classification.","Because relations carry parameters, an application can read an exact distance or angle from the annotation instead of retraining a classifier for each new definition of a spatial predicate.","The egocentric camera attached to each human's head yields first-person views aligned with the person's pose, enabling evaluation of scene graph models from a first-person perspective.","The procedural pipeline can produce effectively unlimited additional annotated frames by generating new scenes and sampling new human placements, without additional human labeling effort."],"supporting_citations":[{"why":"Supplies the parametric relation extraction approach and procedural annotation pipeline that the dataset extends to humans.","marker":"[8]"},{"why":"Generates the procedural indoor scenes and renders the RGB, depth, and segmentation ground truth.","marker":"[24]"},{"why":"Provides the set of human poses fitted to body meshes that the pipeline places in scenes.","marker":"[30]"},{"why":"Supplies the articulated 3D body model with joint and finger keypoints used for human meshes and keypoint ground truth.","marker":"[31]"},{"why":"Implements the ray casting used to extract looking at, pointing at, and body facing relations.","marker":"[33]"},{"why":"Provides human textures that make rendered people look realistic.","marker":"[32]"}],"fun_headline_variants":["Synthetic scenes link humans to objects with precise relations","HOIverse: synthetic indoor scenes with dense human-object labels","Procedural scene graphs capture human-object interactions accurately","Benchmark dataset for scene graph models on human interactions","Synthetic data for scene understanding with humans in 3D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The dataset's usefulness rests on the assumption that geometric heuristics, such as ray casts from joints and volume-based relation checks, produce interactions that match what a human would call sitting on, looking at, touching, and so on, yet the pipeline never verifies that a placed pose actually contacts or engages its intended object.","fun_headline_variants_meta":{"raw":{"variants":["Synthetic scenes link humans to objects with precise relations","HOIverse: synthetic indoor scenes with dense human-object labels","Procedural scene graphs capture human-object interactions accurately","Benchmark dataset for scene graph models on human interactions","Synthetic data for scene understanding with humans in 3D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000983,"raw_usage":{"total_tokens":4143,"prompt_tokens":890,"completion_tokens":3253,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":3173}},"tokens_in":506,"tokens_out":3253,"duration_ms":24225,"temperature":1.0,"reasoning_tokens":3173,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:28:53.758184+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Sample a random set of HOIverse frames labeled with contact or gaze relations and check ground truth directly: does the human mesh intersect or touch the chair for sitting on, and does the head ray's hit object for looking at match a human gaze judgment? If a substantial fraction of sampled labeled relations fail such checks, the dataset's claim of accurate ground truth would be contradicted.","supporting_citations":[{"cited_title":"Copa-sg: Dense scene graphs with parametric and proto-relations,","cited_arxiv_id":null,"evidence_quote":"Supplies the parametric relation extraction approach and procedural annotation pipeline that the dataset extends to humans."},{"cited_title":"Infini- gen indoors: Photorealistic indoor scenes using procedural generation,","cited_arxiv_id":null,"evidence_quote":"Generates the procedural indoor scenes and renders the RGB, depth, and segmentation ground truth."},{"cited_title":"AGORA: Avatars in geography optimized for regression analysis,","cited_arxiv_id":null,"evidence_quote":"Provides the set of human poses fitted to body meshes that the pipeline places in scenes."},{"cited_title":"Expressive body capture: 3D hands, face, and body from a single image,","cited_arxiv_id":null,"evidence_quote":"Supplies the articulated 3D body model with joint and finger keypoints used for human meshes and keypoint ground truth."}],"review_version":2}