{"id":"02d5e6f9-8a5d-4651-8254-6de43fe6315b","arxiv_id":"2504.13072","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"HiScene generates compositional 3D scenes by treating a room as an object under isometric view, then decomposing and regenerating each instance with video-diffusion amodal completion.","lead":"A new system generates 3D scenes from text by first drawing an isometric view of the whole room, then splitting it into objects and regenerating each one. It aims to give users complete, editable objects instead of one inseparable scene.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Scene initialization by object-centric TRELLIS on room-level isometric images is the unvalidated keystone; downstream parsing and refinement cannot recover objects that S0 fuses or omits.","rationale":"The reader's weakest_assumption identifies the same stage, and I agree with that assessment. The idea is plausible and the amodal completion component has independent quantitative support (Table 2), but the scene-level keystone is unproven. A conditional acceptance should make the proposed benchmark a release requirement, since the central claim of compositional, complete, and aligned objects cannot be assessed without it. I also note that the spatial-alignment stage lacks any quantitative pose or scale metric, but that concern is downstream of the initialization and segmentation failure mode; if the benchmark passes, alignment metrics should be added as a further condition.","tokens_in":20104,"tokens_out":8590,"duration_ms":85271,"concrete_test":"Build a benchmark of 50 room-level isometric images with ground-truth 3D, for example by rendering 3D-FRONT or Replica scenes orthographically. Run only the first stage: TRELLIS image-to-3D on those images, and evaluate S0 geometry with Chamfer distance and F-score at a fixed point threshold against the ground-truth scene. Then run the full parsing pipeline (EntitySeg + OmniSeg3D-GS) and report instance-level mIoU and per-object recall against known object masks. If either S0 fidelity or instance recall is below a prespecified threshold (for example, recall below 0.8), the central claim fails because downstream completion cannot recover lost object identities.","verdict_should_be":"UNCHANGED","load_bearing_attack":"HiScene's central promise—complete, separable, spatially aligned objects—rests on Scene Initialization (Sec. 3.2): a single isometric room image is fed to TRELLIS, a native 3D model designed for single assets (Sec. 3.1). The 'scene-as-object' framing does not close the distribution gap: TRELLIS's structured-latent training is object-centric, and a room contains floor/wall geometry, large empty regions, and inter-object contact that object datasets rarely exhibit. No scene-level validation of S0 is reported; Table 1 metrics (CLIP/ImageReward/Aesthetic/user scores) are computed on 2D renderings and do not measure whether the initial 3DGS is coherent or whether EntitySeg/OmniSeg3D-GS parsing (Sec. 3.2) yields clean instances. Downstream amodal completion and shape-prior injection can repair occluded or low-quality regions, but they cannot resurrect objects that S0 fuses, truncates, or omits, nor can they fix a segmentation that merges instances. Thus the decomposition-and-regeneration cascade inherits every failure of the first stage, and the paper supplies no evidence beyond qualitative figures that this stage succeeds at room scale.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HiScene, a text-to-3D-scene generation system that treats a scene as a hierarchy of objects under isometric views. The pipeline first generates an isometric scene image from text, lifts it to a 3D Gaussian Splatting scene via the object-centric native 3D generator TRELLIS, and then parses the scene into instances using EntitySeg and OmniSeg3D-GS. Each occluded instance is completed with a video-diffusion-based amodal completion model trained on a new 1.32M-pair dataset that includes 468K shadow-aware synthetic pairs, and the completed instance is regenerated with a shape-prior injection from sparse-view LRM to preserve spatial alignment. The method is evaluated on 12 scenes with CLIP/ImageReward/Aesthetic scores and a 20-user study, and the amodal completion model is evaluated on COCO-A and BSDS-A. The paper also reports ablations on the video model, shadow data, and spatial alignment, and gives a runtime of about 12 minutes per scene.","tokens_in":20438,"tokens_out":4405,"duration_ms":40025,"significance":"If the central claim holds—that users can obtain complete, separable, spatially aligned 3D objects inside an editable scene—HiScene would be a practically valuable bridge between 2D text-to-image generation and 3D scene editing. The paper contributes a large amodal-completion dataset with shadow effects, a SVD-based video-diffusion completion model, and a shape-prior injection mechanism that enables regeneration with preserved alignment. The qualitative results are visually appealing, and the runtime advantage over SDS-based baselines is compelling. However, the current evaluation does not directly measure the paper's core promise of compositional completeness in 3D: scene-level scores are computed on 2D renderings of 12 scenes, and the amodal module is validated on 2D datasets. The keystone assumption that TRELLIS generalizes from single objects to full-room isometric views is asserted without scene-level validation. These are significant gaps for a journal submission, though they appear addressable with additional experiments.","major_comments":[{"comment":"The first stage feeds a full-room isometric image into TRELLIS, a model trained on object-centric assets, and the manuscript supplies no scene-level validation of the resulting 3DGS scene S0. Since the subsequent EntitySeg/OmniSeg3D-GS parsing operates on S0, any instance that is fused, truncated, or omitted at this stage cannot be recovered by the later amodal completion and shape-prior stages; the decomposition-and-regeneration cascade inherits every failure of initialization. The paper should report, on a modest set of scenes, instance-level parsing quality—for example, the number of detected instances versus the expected count, per-instance 3D overlap against manually annotated Gaussians, or at least a failure-case analysis—to demonstrate that S0 is parseable into cleanly separated objects.","section":"Section 3.2, Scene Initialization"},{"comment":"The amodal completion model is validated on 2D datasets (COCO-A, BSDS-A) with mIoU on amodal masks, but the paper's central claim concerns 3D instance completeness: each object in the generated scene should be an intact, manipulatable 3D entity. Table 1 only reports 2D-rendered aesthetic/CLIP/ImageReward scores, and the user study asks about matching degree and overall quality, not about object completeness. The authors should directly measure per-instance 3D completeness—for instance, by comparing the regenerated object's geometry with the initially visible region, by measuring the fraction of instances whose regenerated 3DGS covers the amodal mask footprint across multiple views, or by asking users to judge whether each object remains intact under rotation.","section":"Section 4.2, Table 2"},{"comment":"The comparison to GALA3D and DreamScene is under-specified: the paper states that both baselines require predefined 3D layouts, but it does not describe how these layouts were provided (e.g., from an LLM, from the same text, or from a human). If the baselines were run without their expected layout input, the comparison is biased; if layouts were supplied, the protocol should be stated. Additionally, all results derive from only 12 scenes, and no per-scene breakdown or statistical significance is reported. Given the near-ceiling user scores (2.90 out of 3.00), the advantage over baselines should be supported with confidence intervals or per-item analysis.","section":"Section 4.1, Table 1"},{"comment":"The spatial alignment claim rests on a purely qualitative ablation (Figure 9). Spatial alignment is a quantitative geometric property; the paper should report a metric such as Chamfer distance, pose error, or scale error between the regenerated instance and its original placement in S0, averaged over scenes. The hand-selected timestep range t in [0.2, 0.4] is a design choice, and its sensitivity should be examined with at least a small sweep to show the full method is not fragile to this choice.","section":"Section 3.4 and Figure 9"}],"minor_comments":[{"comment":"The sentence containing 'thn illustration of the use' should read 'an illustration of the use'; please fix this typo.","section":"Supplementary B.3"},{"comment":"References [24] and [25] cite the same SAM paper (Kirillov et al., 2023) and should be merged into a single reference to avoid duplication and confusion.","section":"References [24] and [25]"},{"comment":"The notation in Equation (1) does not define the ranges of L, C, and N; please add explicit definitions for these quantities in the text.","section":"Equation (1)"},{"comment":"The table header 'Datasets & Method COCO-A BSDS-A I2I I2V I2I I2V' is difficult to parse; the caption should clarify that I2I and I2V refer to image-model and video-model variants, respectively, and that COCO-A and BSDS-A are the two evaluation datasets.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"This is a well-written systems paper with a clear pipeline and appealing results, but the evaluation does not yet support the headline claim of complete, separable 3D objects. The authors appear to have the data and infrastructure to add the missing measurements, so I would welcome a revised version. I would also ask the editor to consider whether 12 test scenes is sufficient for a journal submission in this venue, particularly when the central claim is a compositional-completeness guarantee rather than only aesthetic quality."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, the central idea is genuinely novel: instead of generating a layout and placing objects, they generate an isometric view image and treat the whole room as a single 'object' for a native 3D generator (TRELLIS), then decompose and regenerate each piece. This is a smart way to borrow the layout priors of 2D generation while staying in 3D. Second, the video-diffusion amodal completion is a real sub-contribution: training SVD to reveal occluded objects via temporal transitions, with a new synthetic shadow-aware dataset, and beating Pix2gestalt on COCO-A/BSDS-A. That part is solid and reproducible as far as a preprint can be.\n\nThe paper does a lot right. The pipeline is careful, the figures are informative, and the runtime advantage (12 minutes vs 1–2 hours for SDS-based methods) is meaningful for practitioners. The shape prior injection via LRM to preserve alignment in TRELLIS refinement is a sensible engineering fix, and the ablation showing that spatial alignment matters is convincing qualitatively.\n\nNow the soft spots. The biggest one is exactly what the stress-test note says: the keystone stage, Scene Initialization, runs object-centric TRELLIS on a room-level isometric image, and there is no quantitative check that the resulting 3D Gaussian soup is a coherent room. The downstream parsing and refinement can only repair what survives that first stage; if TRELLIS fuses two objects or truncates the floor, no amount of amodal completion brings it back. The paper provides no scene-level decomposition metric—no per-instance completeness, no alignment error, no segmentation IoU. Table 1 is about aesthetics and text alignment, Table 2 is about 2D amodal masks, and the user study has 12 scenes with no error bars. Those metrics do not measure the paper's own promise: complete, separable, spatially aligned 3D objects. The qualitative results are good enough to suggest the idea works, but 'good enough on a few examples' is not a soundness proof.\n\nThere are also smaller gaps: the occlusion threshold and timestep are hand-chosen (legitimate design choices, not predictions), the dataset and code are withheld until acceptance, and there are no statistical tests on the user study. None of these are fatal on their own, but together they make the paper hard to verify from the preprint alone.\n\nWho should read it: people working on text-to-3D scene generation and on amodal completion. The first group gets a new top-down paradigm; the second gets a strong baseline with useful data. I would send it to peer review—the core idea is novel enough to deserve referee time—but I would expect the reviewers to push hard for quantitative decomposition metrics and a scene-level validation of the initialization. If those hold up, this is a solid paper; if they don't, the method is a collection of neat tricks without the claimed benefit.","headline":"Clever isometric-view pipeline with a strong amodal completion module, but the headline claim of separable complete objects is never directly measured and the scene-initialization keystone is unvalidated.","tokens_in":20902,"tokens_out":3453,"would_cite":true,"duration_ms":31583,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HiScene's central claim is that a full 3D scene can be created from a text prompt by generating the whole room as one isometric object, then decomposing it and regenerating every item as an independent, aligned 3D instance.","keywords":["hierarchical 3D scene generation","isometric view","3D Gaussian Splatting","amodal completion","video diffusion","shape prior injection","text-to-3D scene","scene parsing"],"falsifier":"Take a set of isometric room images with known ground-truth object instances and run TRELLIS scene initialization followed by OmniSeg3D-GS with EntitySeg masks; if overlapping objects such as chairs or table and pillows merge into single Gaussian groups and cannot be separated into the known instances, the decomposition-and-regeneration cascade fails at its first step.","tokens_in":19938,"feed_emoji":"🛋️","tokens_out":4791,"duration_ms":46008,"temperature":0.7,"pith_summary":"The paper tries to establish a new route to text-to-3D scene generation: instead of dictating layouts with large language models or lifting a single image into an inseparable mesh, it generates the entire room as one object, then rewrites each item as a separate, manipulatable 3D entity. The payoff, if correct, is that users get natural scene arrangements and intact objects they can edit, replace, or simulate, all from a prompt without any predefined layout. The method depends on isometric views for their low occlusion and consistent scale, on analysis-by-synthesis parsing of an initial 3D Gaussian scene, and on video-diffusion amodal completion followed by spatial shape prior injection to keep regenerated objects coherent with their original positions.","feed_headline":"Text prompts become editable 3D scenes, objects intact","feed_subtitle":"A room is treated as one object, then each chair, table, and lamp is regenerated as an independent 3D asset you can grab.","key_machinery":"The load-bearing representation is the isometric view taken as a scene-as-object projection: it makes a room behave like a single object to an object-centric 3D generator. Around that, the machinery is a three-stage pipeline: (1) TRELLIS native 3D generation creates an initial 3D Gaussian Splatting scene from the isometric image; (2) OmniSeg3D-GS contrastive segmentation, fed by EntitySeg instance masks, separates objects, and object-centric circular renders are screened by a vision-language model for occlusion; (3) a fine-tuned Stable Video Diffusion model performs amodal completion as a video transition, and the completed views go through sparse-view LRM whose voxelized geometry is encoded by TRELLIS's structure encoder and injected as a noisy latent initialization at intermediate timestep $t\\in[0.2,0.4]$, anchoring regeneration to the original placement.","core_discovery":"The paper's central discovery is that a full 3D scene can be generated compositionally by treating a room as a hierarchical object under an isometric view. The room is first generated as a whole by a native object-level 3D generation model, then parsed into instances through 3D semantic segmentation, each instance is completed with a video-diffusion model that treats occlusion removal as a temporal dissolve transition, and finally each object is regenerated with a shape prior so it keeps the original scene's pose and scale. The authors report that this pipeline yields scenes with natural layouts, complete compositional instances, and physical plausibility, and outperforms LLM-layout baselines on aesthetic score, image-text alignment, and user preference.","pith_inferences":["The same hierarchy could be applied recursively, so a generated object that itself contains smaller objects could be parsed and regenerated as a scene within a scene, provided the underlying native 3D generator can represent that level of detail.","The video-transition framing of amodal completion is not tied to furniture; the same dissolve-occlusion training recipe could be applied to people, vehicles, or dense vegetation by constructing shadow-aware video pairs for those categories.","Because the shape prior is injected as latent initialization rather than a model-specific constraint, the alignment mechanism could be ported to other native 3D generators beyond TRELLIS, which would make the pipeline more modular.","A testable extension is to add PBR material estimation into the regeneration stage, directly addressing the paper's own stated limitation of baked lighting and opening the output to standard rendering pipelines."],"forward_implications":["Users can generate editable 3D scenes from text without specifying a layout, bypassing LLM spatial planning and handcrafted placement rules.","Each object in the generated scene is produced as a complete, separable 3D entity, enabling individual manipulation, replacement, and simulation without disturbing the rest of the room.","Amodal completion formulated as a temporal video transition recovers occluded object shape and removes occlusion shadows better than static image inpainting or single-image diffusion, as measured on the COCO-A and BSDS-A benchmarks.","Spatial shape prior injection keeps regenerated objects aligned with the original scene, avoiding the canonical-orientation drift and Janus artifacts that appear when native 3D regeneration runs without alignment.","The full pipeline runs in about 12 minutes per scene, compared with one to two hours for SDS-optimization-based baselines, making interactive iteration practical."],"supporting_citations":[{"why":"TRELLIS is the native 3D generation model that creates the initial 3D Gaussian scene from the isometric image and is the target of the shape prior injection.","marker":"[70]"},{"why":"OmniSeg3D-GS provides the contrastive-learning-based 3D semantic segmentation that lifts 2D masks to per-Gaussian instance features for scene parsing.","marker":"[78]"},{"why":"EntitySeg supplies class-agnostic instance-level 2D segmentation masks over multi-view scene renders, which feed the 3D segmentation.","marker":"[55]"},{"why":"Stable Video Diffusion is the image-to-video base model fine-tuned into the amodal completion network that generates occlusion-to-complete transitions.","marker":"[7]"},{"why":"Pix2gestalt provides the amodal dataset construction approach and the zero-shot baseline the method compares against on COCO-A and BSDS-A.","marker":"[49]"},{"why":"The sparse-view large reconstruction model (InstantMesh/LRM) reconstructs initial geometry whose voxelized latent becomes the spatial shape prior for regeneration.","marker":"[72]"},{"why":"Objaverse supplies the filtered 3D object set used to synthesize the 468K shadow-aware training image pairs with rigid-body placement and path-traced lighting.","marker":"[15]"},{"why":"GALA3D is the LLM-layout scene generation baseline whose layout requirement and output quality are compared against HiScene.","marker":"[93]"},{"why":"DreamScene is the other layout-required baseline baseline, used for quantitative comparison and user study.","marker":"[29]"},{"why":"COCO-A with its amodal annotations is the primary benchmark on which the video-based amodal completion is evaluated against zero-shot baselines.","marker":"[94]"}],"fun_headline_variants":["Hierarchical 3D scenes from text, each object editable","Room as one object, then split into grab-ready parts","Isometric view unlocks full 3D scene generation","Scene generation that keeps every object complete","From one 3D room to fully editable objects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The single most load-bearing premise is that the object-centric native 3D generator TRELLIS, trained on single objects, also produces a clean, parseable 3D Gaussian scene when fed a room-level isometric image; the paper invokes this at scene initialization without validation at scene scale.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchical 3D scenes from text, each object editable","Room as one object, then split into grab-ready parts","Isometric view unlocks full 3D scene generation","Scene generation that keeps every object complete","From one 3D room to fully editable objects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000175,"raw_usage":{"total_tokens":1252,"prompt_tokens":878,"completion_tokens":374,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":298}},"tokens_in":494,"tokens_out":374,"duration_ms":3988,"temperature":1.0,"reasoning_tokens":298,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:14:34.917484+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of isometric room images with known ground-truth object instances and run TRELLIS scene initialization followed by OmniSeg3D-GS with EntitySeg masks; if overlapping objects such as chairs or table and pillows merge into single Gaussian groups and cannot be separated into the known instances, the decomposition-and-regeneration cascade fails at its first step.","supporting_citations":[{"cited_title":"Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning","cited_arxiv_id":null,"evidence_quote":"OmniSeg3D-GS provides the contrastive-learning-based 3D semantic segmentation that lifts 2D masks to per-Gaussian instance features for scene parsing."},{"cited_title":"pix2gestalt: Amodal segmentation by synthesizing wholes","cited_arxiv_id":null,"evidence_quote":"Pix2gestalt provides the amodal dataset construction approach and the zero-shot baseline the method compares against on COCO-A and BSDS-A."},{"cited_title":"Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling","cited_arxiv_id":null,"evidence_quote":"DreamScene is the other layout-required baseline baseline, used for quantitative comparison and user study."},{"cited_title":"1\": \"A wooden nightstand with a lamp","cited_arxiv_id":null,"evidence_quote":"COCO-A with its amodal annotations is the primary benchmark on which the video-based amodal completion is evaluated against zero-shot baselines."}],"review_version":1}