{"id":"9bea7415-145e-4791-99c0-f19fd3b89cea","arxiv_id":"2412.01801","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A two-stage latent diffusion method for text-guided 3D scene generation where a coarse semantic box layout is generated first and then used to condition high-fidelity geometry, enabling localized box-based editing.","lead":"SceneFactor generates 3D indoor scenes from text in two stages: first a coarse 3D layout of semantic boxes, then detailed geometry guided by that layout. The semantic box map can be edited with a few clicks, letting users add, remove, resize, or move objects without regenerating the whole scene.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Editing in Sec. 3.5 is underspecified: filling only R_S with Gaussian noise while keeping the rest of F_G clean is not a valid diffusion inpainting step, so the locality claim is unverified.","rationale":"The central claim is that users can locally edit a generated 3D scene by manipulating semantic boxes without re-synthesizing the whole scene. This hinges on the geometric diffusion model being able to synthesize a region R_S conditioned on the surrounding fixed latent grid. The paper's description of this step in Sec. 3.5 is not a complete algorithmic specification. In standard diffusion inpainting, including the paper's own outpainting in Sec. 3.4, the known region must be re-noised at every reverse step to keep it consistent with the noise level at the current timestep. The edit description omits this step and omits the starting timestep. If the reverse process begins at t=T with a clean fixed region, the input is a mixture of isotropic noise and a clean signal, which is far from what the model sees during training and likely leads to cross-boundary artifacts. If it begins at an intermediate t, the known region needs a matching noising schedule. The paper provides no experiments that directly measure geometry changes outside R_S; the only evidence is a perceptual question asked to 21 participants, which is too weak to establish locality. The convolutional attention window (size 3) compounds the issue by allowing information to bleed across the boundary. This is a load-bearing concern because without local editability the paper's claimed first contribution falls back to being a text-conditioned scene generator, which is less novel. The reader's weakest assumption focused on the locality/compositionality assumption; we refine this to the under-specification of the noising/inpainting process, which is closely related but more concrete. We therefore recommend keeping the conditional verdict, with the added explicit requirement that the authors provide the exact editing algorithm and quantitative boundary-consistency measurements before the central claim can be accepted.","tokens_in":20721,"tokens_out":8045,"duration_ms":81395,"concrete_test":"Re-derive the edit step using the paper's own RePaint equations (Eqs. 8-10): the outpainting procedure explicitly noises the known region (Eq. 8), but Sec. 3.5's edit description omits this. Implement two variants on 100 generated scene chunks: (A) as literally described, filling only R_S with Gaussian noise and keeping the complement clean during reverse diffusion; (B) with known-region noising at each reverse step per Eq. 8. Measure the Chamfer distance between pre-edit and post-edit geometry in a 1-voxel band outside R_S. If (A) produces artifacts or measurable geometry changes outside R_S that (B) does not, the description is incomplete and the locality claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 3.5 specifies each edit by 'fill[ing] only R_S in the grid F_G with Gaussian noise and re-generat[ing] geometry for it,' while holding the rest of the latent grid fixed. This is not a well-defined diffusion inverse problem. The paper does not state the starting timestep of the reverse process, the noise level of the known region at that timestep, or how the known region is incorporated during iterative denoising. If the process starts at t=T with the known region left as the clean decoded latent, the model input is isotropic Gaussian noise inside R_S and a clean signal outside—a distribution mismatch that likely causes severe boundary artifacts and invalidates the claimed locality. If instead the process starts at an intermediate timestep, the known region must be noised at each step (as in the paper's own outpainting equations, Eqs. 8-10) to remain consistent with the model's training distribution; this step is absent from Sec. 3.5. The only direct evidence for edit locality is a unary perceptual question ('did the scene remain unchanged outside the editing region?') answered by 21 participants, with no quantitative measure of geometry outside R_S. Given the convolutional attention with window size 3, edits can bleed across boundaries, so the central claim of localized editing without re-synthesis is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SceneFactor, a two-stage latent diffusion approach for text-guided 3D indoor scene generation. A semantic VQ-VAE encodes coarse semantic box layouts, and a text-conditioned diffusion model generates these layouts; a geometric VQ-VAE encodes truncated unsigned distance fields, and a second diffusion model synthesizes geometry conditioned on the semantic layout via spatial cross-attention with window-3 convolutional attention. Large scenes are produced by chunk-based outpainting, with RePaint-style inpainting equations for the known regions. The paper claims that this factorization enables localized editing: users manipulate semantic boxes (add, remove, replace, resize, move) and only the edited geometric region is re-synthesized. Experiments compare geometry quality and text consistency against several baselines and include a multi-part perceptual study.","tokens_in":20943,"tokens_out":7457,"duration_ms":73295,"significance":"If the claims hold, SceneFactor would be a notable advance: it targets editable large-scale text-to-3D-scene generation, and the factored semantic-proxy representation makes local editing conceptually simple. The paper's strengths include a clearly described two-stage pipeline, explicit RePaint-style outpainting equations (Eqs. 8-10), a large-scale training setup on 3D-FRONT/3D-FUTURE, and quantitative evaluations on geometry and text consistency, including a perceptual study. However, the central editing claim is currently not sufficiently supported: the exact diffusion-inpainting procedure for edits is underspecified, and the only direct evidence of edit locality is a small unary user study. The methodological issues in the editing evaluation are load-bearing because localized editing is the main stated contribution.","major_comments":[{"comment":"The editing protocol is underspecified in a way that directly affects the central claim. The text says to 'fill only R_S in the grid F_G with Gaussian noise and re-generate geometry for it,' but it does not state the starting timestep, the noise level of the known region at that timestep, or how the known region is resampled during the reverse process. If the reverse process starts at t=T with the known region held at clean latent values, the model input is a mixture of isotropic noise and clean signal, which is out-of-distribution and likely produces boundary artifacts; if it starts at an intermediate timestep, the known region must be noised at each step as in the paper's own outpainting equations (8)-(10). The manuscript should specify the algorithm precisely and verify that the known region is processed consistently with the diffusion training distribution.","section":"Sec. 3.5"},{"comment":"There is no quantitative measure of edit locality. The only evidence that the scene outside the editing region R_S remains consistent is a unary perceptual question answered by 21 participants. Because the geometric diffusion uses convolutional attention with window size 3 (Sec. 3.3), cross-boundary influence is mechanistically possible, so a numerical locality metric is needed — for example, Chamfer distance or 1-NNA between the geometry outside R_S before and after editing, or a comparison between localized editing and full re-synthesis. This evidence is necessary to support the claim of 'localized editing ... without requiring re-synthesis of the full scene.'","section":"Sec. 3.5 and Supplement Sec. 8"},{"comment":"The reported geometry and text-consistency metrics are point estimates with no error bars, standard deviations, or significance tests. Since diffusion sampling is stochastic and the test set has only 250 scenes, the claimed improvements over baselines could be within run-to-run variation. The authors should report results over multiple seeds (or at least repeated sampling) and, where relevant, a significance test.","section":"Tables 1, 3, 4, 9, 10"},{"comment":"The neural listener evaluation in Table 2 (and Table 8 in the supplement) uses the same BERT text encoder as SceneFactor for the language encoder. This can introduce a bias in favor of the proposed method, since the text representation is not encoder-independent. The CLIP score in Table 3 also uses the maximum over five rendered views per chunk; the paper should clarify whether the same rule is applied to all baselines and show sensitivity to the number and choice of views.","section":"Sec. 4.2 and Supplement Sec. 8"}],"minor_comments":[{"comment":"The text says the geometric features are translated 'from R1_S to R1_S' — presumably this should be 'from R1_S to R2_S'. Please fix the typo.","section":"Sec. 3.5, 'Moving an object'"},{"comment":"The caption says 'as independent chunks (left) and as chunks of outpainted 3D scenes (right)', but the table contains only independent-chunk columns; adjust the caption to match the table.","section":"Supplement, Table 6 caption"},{"comment":"The paper reports 3 million crops for VQ-VAE training and a 6000/250 train/test scene split, but it would help to state how many chunks are used for training the diffusion models and how the 250 test scenes are chunked.","section":"Sec. 4.1 and Supplement Sec. 6"},{"comment":"The v-parameterization notation is confusing: v_t is defined as sqrt(alpha_t) epsilon_t - sqrt(1-alpha_t) x_t, while Eq. (5) defines x_t = sqrt(alpha_bar_t) x_0 + sqrt(1-alpha_bar_t) epsilon. Please clarify the exact relation and ensure subscripts (alpha_t vs alpha_bar_t) are consistent.","section":"Sec. 3.3"},{"comment":"The BlockFusion comparisons are generated unconditionally, while the other baselines and SceneFactor are text-conditioned. Since the main comparison is against conditional generation, please state explicitly whether all geometry metrics are computed in the same conditional setting, or discuss why an unconditional BlockFusion comparison is informative.","section":"Sec. 4.3 and Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid pipeline paper and the editing idea is appealing, but the editing procedure as written is not a reproducible diffusion-inpainting algorithm, and the absence of a quantitative locality metric makes the central claim difficult to verify. I recommend major revision. I would also ask the authors to reconsider the 'first method' phrasing and to clarify why BlockFusion is evaluated unconditionally in the main comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know up front. The factored two-stage design—text to coarse 3D semantic boxes, then boxes to geometry—is a genuinely useful decomposition, and it buys something prior scene-level diffusion models lacked: a lightweight editing handle. But the paper's central editing claim is currently underspecified to the point of being unverified; that's the section a referee should press on.\n\nWhat's new: the latent semantic proxy is a real addition over BlockFusion, XCube, and SDFusion, which either generate text-to-geometry directly or need full re-synthesis for an edit. Box-based editing (add, remove, resize, move, replace) is a nice interface, and chunked outpainting with overlaps is a sensible way to get arbitrary scene sizes. Geometry metrics are decent, and the authors honestly asterisk the unconditional BlockFusion comparison.\n\nSoft spots: no error bars or significance tests on headline numbers, so the reported gains could be noise. The neural listener evaluation reuses SceneFactor's own text encoder as the language encoder, a real confound. And Sec. 3.5 says an edit is done by 'filling R_S with Gaussian noise and re-generating.' As written, that's not a valid diffusion inpainting step: the model expects noised latents at every timestep, and leaving the known region clean at t=T is a distribution mismatch likely to produce boundary artifacts. The paper has the correct RePaint-style formulas in Sec. 3.4 but never connects them to editing. If the authors intended the same known-region noising, they need to say so; if not, the locality claim is unsupported. The only direct locality evidence is a unary perceptual question from 21 participants, and the convolutional attention has a nonzero receptive field, so bleed is possible.\n\nWho this is for: anyone working on 3D scene diffusion or interactive content creation. The editing interface could be useful in practice, but needs a properly specified algorithm and a quantitative locality check first.\n\nRecommendation: send to review. The core idea is worth referee time, and the editing section is fixable if the authors are pushed to make the procedure concrete and measure the unchanged region directly.","headline":"Factored text-to-layout-to-geometry diffusion with box-based editing is a real step forward, but the editing procedure is underspecified and the locality claim needs hard evidence.","tokens_in":21523,"tokens_out":3270,"would_cite":false,"duration_ms":30656,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SceneFactor generates 3D scenes from text, then lets users edit them by moving boxes.","keywords":["3D scene generation","latent diffusion","text-guided generation","semantic layout","scene editing","VQ-VAE","outpainting","indoor scenes"],"falsifier":"Take a generated scene, edit one semantic box, fill only that box's geometric region with Gaussian noise, re-synthesize, and compare the decoded geometry outside the bounding box before and after the edit; if voxels outside the region change beyond the model's decoding noise, the locality assumption fails. A stronger check would compare the locally edited scene against a full re-synthesis with the same edited semantic map: the editing claim stands only if the two agree outside the edited box.","tokens_in":20480,"feed_emoji":"🛋️","tokens_out":5457,"duration_ms":46280,"temperature":0.7,"pith_summary":"SceneFactor tries to make large-scale 3D indoor scene generation from text both high-quality and locally editable. Its key move is to break generation into two stages: first synthesize a coarse semantic map of 3D boxes from the text, then synthesize geometry conditioned on that map. Because edits are performed on the box map, a user can add, remove, move, resize, or replace an object by manipulating two box corners, and only the edited region is re-synthesized. If the claim holds, this gives content creators a practical middle ground between writing a new text prompt and re-generating an entire scene.","feed_headline":"3D scenes from text become locally editable in a few clicks","feed_subtitle":"A semantic-box layer lets users add, move, or resize objects without re-synthesizing the scene.","key_machinery":"The load-bearing mechanism is the pair of aligned latent grids $f_S$ and $f_G$, produced by two 3D VQ-VAEs, together with the diffusion models defined over them. The semantic diffusion $\\Psi_S$ generates $f_S$ from text; the geometric diffusion $\\Psi_G$ generates $f_G$ from $f_S$ using convolutional attention with window size 3 so the geometry reads local semantic neighborhoods. Editing works by replacing the semantic box in the region $\\mathcal{R}_S$, filling the corresponding region of $f_G$ with Gaussian noise, and denoising only that region while the rest of the grid is held fixed. Chunk-wise outpainting with overlapping sliding windows extends the scheme to arbitrary scene sizes.","core_discovery":"The paper's central claim is that factoring text-to-scene generation through a proxy semantic space of 3D boxes makes 3D scene generation more tractable and editable. A semantic latent diffusion model consumes a text caption and produces a coarse box layout; a geometric latent diffusion model consumes that layout through spatial cross-attention and produces a truncated unsigned distance field for the geometry. The two latent grids are kept at equal resolution so they align exactly, and editing means changing boxes in the semantic map and re-running only the affected geometric region from Gaussian noise. The paper reports that this factored approach improves geometric quality and text adherence over direct text-to-geometry baselines, while enabling object addition, removal, replacement, rescaling, and movement without full-scene re-synthesis.","pith_inferences":["The editing procedure's locality could be stress-tested by measuring whether geometry outside the edited bounding box changes when only $\\mathcal{R}_S$ is re-noised; any measurable drift would indicate the effective receptive field leaks across the boundary.","A natural extension is to let the semantic stage produce open-vocabulary object categories instead of a closed 10-class vocabulary, which would broaden edit operations beyond categories seen in the training data.","The same factored semantic-geometry idea could apply to other structured outputs, such as outdoor scenes or articulated objects, wherever a coarse layout can mediate between language and dense geometry.","Linking text to semantic boxes opens a path to language-driven editing: a user could say 'make the sofa larger,' and a parser could perform the box manipulation automatically."],"forward_implications":["Users can add, remove, move, resize, or replace objects in a generated scene by editing two corners of a semantic box, with no full-scene re-synthesis.","Scene size is not fixed: chunk-by-chunk outpainting with overlapping windows yields arbitrarily sized rooms.","Decoupling layout from geometry lets the semantic stage align the scene to text before any geometric detail is committed, reducing floaters and incoherent arrangements.","Because the semantic map is explicit, edited results keep global consistency outside the edited region, provided the locality assumption in the edit procedure holds.","The same factored pipeline outperforms direct text-to-geometry diffusion baselines on geometric quality and text-consistency metrics in the reported experiments."],"supporting_citations":[{"why":"Supplies the denoising diffusion probabilistic model backbone used by both the semantic and geometric stages.","marker":"[25]"},{"why":"Provides the latent diffusion formulation that SceneFactor extends from images to 3D latent grids.","marker":"[52]"},{"why":"Serves as a text-conditioned latent 3D generation baseline and as the source of the v-parameterization used in the reverse process.","marker":"[11]"},{"why":"Supplies the VQ-VAE discrete latent compression that builds the semantic and geometric latent spaces.","marker":"[64]"},{"why":"Provides the inpainting-style denoising mechanism used for chunk outpainting and for re-synthesizing edited regions.","marker":"[40]"},{"why":"Contributes the 3D-FRONT indoor scene dataset used for training and evaluation.","marker":"[21]"},{"why":"Contributes the 3D-FUTURE furniture models that furnish the 3D-FRONT scenes.","marker":"[22]"},{"why":"Serves as the main large-scale 3D scene generation baseline compared in the experiments.","marker":"[70]"}],"fun_headline_variants":["Edit 3D scenes by tweaking semantic boxes, not re-synthesizing","Factored diffusion: text-to-3D with local box-based editing","Add, move, or resize 3D objects without full re-generation","Semantic proxy boxes make 3D editing as easy as dragging","Local 3D edits from text via factored diffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The editing procedure assumes that re-synthesizing only the edited region of the geometric latent grid, while holding the rest fixed, produces geometry that is consistent with the unchanged surroundings.","fun_headline_variants_meta":{"raw":{"variants":["Edit 3D scenes by tweaking semantic boxes, not re-synthesizing","Factored diffusion: text-to-3D with local box-based editing","Add, move, or resize 3D objects without full re-generation","Semantic proxy boxes make 3D editing as easy as dragging","Local 3D edits from text via factored diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000296,"raw_usage":{"total_tokens":1664,"prompt_tokens":836,"completion_tokens":828,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":452,"completion_tokens_details":{"reasoning_tokens":735}},"tokens_in":452,"tokens_out":828,"duration_ms":8272,"temperature":1.0,"reasoning_tokens":735,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:55:13.536811+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a generated scene, edit one semantic box, fill only that box's geometric region with Gaussian noise, re-synthesize, and compare the decoded geometry outside the bounding box before and after the edit; if voxels outside the region change beyond the model's decoding noise, the locality assumption fails. A stronger check would compare the locally edited scene against a full re-synthesis with the same edited semantic map: the editing claim stands only if the two agree outside the edited box.","supporting_citations":[{"cited_title":"SDFusion: Multimodal 3d shape completion, reconstruction, and generation","cited_arxiv_id":null,"evidence_quote":"Serves as a text-conditioned latent 3D generation baseline and as the source of the v-parameterization used in the reverse process."},{"cited_title":"3d-future: 3d fur- niture shape with texture","cited_arxiv_id":null,"evidence_quote":"Contributes the 3D-FUTURE furniture models that furnish the 3D-FRONT scenes."},{"cited_title":"Blockfusion: Expand- able 3d scene generation using latent tri-plane extrapolation","cited_arxiv_id":null,"evidence_quote":"Serves as the main large-scale 3D scene generation baseline compared in the experiments."}],"review_version":1}