{"id":"cadb2aee-a3ba-4498-930c-889414394c88","arxiv_id":"2412.00091","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"GraphCanvas3D generates and edits 3D scenes from text by iteratively optimizing a graph layout with feedback from a vision-language model.","lead":"The paper describes a system that turns a text description into a 3D scene by first building a graph of objects and their spatial relationships, then rendering views and asking a vision-language model to score and adjust the layout until it looks plausible. The same system claims to support interactive editing and time-varying scene changes through typed instructions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The edge-optimization loss in Eq. (2) is a function of discrete MLLM scores, yet Eq. (3) requires ∂Lij/∂fi; without a differentiable path from node features to scores, the gradient is undefined and the core optimization loop is not executable as written.","rationale":"The reader's REJECT verdict is appropriate, but the strongest reason is not primarily the noisy-MLLM concern; it is that the algorithm as written is formally incomplete. Section 3.3, Eq. (3), uses ∂Lij/∂fi. Lij is built from discrete MLLM scores, and since the MLLM outputs integer text responses to rendered images, and no differentiable connection between fi and scores is supplied, the partial derivative is undefined. The paper therefore cannot claim gradient-based updates without either a differentiable surrogate or a finite-difference scheme; neither appears in the main text or the appendix. This is an internal inconsistency, not a disagreement with community consensus, and it directly undermines the central claim that iterative optimization produces high-quality layouts. If the update rule is not specifiable, a reader cannot run the method, and any reported results cannot be attributed to the described mechanism. The reader's other concerns—placeholder citation entries for GALA3D and MVDream, and the circular MLLM evaluation—compound the problem, but the gradient-definition issue is the most load-bearing because it attacks the method's core rather than only its evaluation. The user study in Table 2 is less circular, but it does not rescue the method if the main optimization loop cannot be reproduced from the paper. My concern is distinct from but adjacent to the reader's weakest assumption: the reader focuses on score reliability and monotonicity, whereas I focus on the absence of any defined gradient even under the assumption that scores are reliable. Both point to the same section and to the same conclusion: the central optimization claim is unsupported, so the REJECT verdict should stand.","tokens_in":13570,"tokens_out":4393,"duration_ms":42588,"concrete_test":"Run the edge-optimization loop from the public repository (https://github.com/ILGLJ/Graph-Canvas) on a minimal scene, e.g., a lamp on a table, with a known suboptimal offset. Instrument the code to record, at each iteration, the MLLM scores s_k, the computed Lij, and the actual update applied to fi. Then verify (a) whether the update equals -η ∂Lij/∂fi for a differentiable surrogate (and if so, whether that surrogate matches MLLM behavior), and (b) whether Lij decreases monotonically and converges to the ground-truth layout. If the gradient in Eq. (3) cannot be computed or the loss does not decrease, the optimization claim fails. If the repository is empty or the loop is absent, that absence alone confirms the manuscript is not reproducible.","verdict_should_be":"REJECT","load_bearing_attack":"The central mechanism of GraphCanvas3D is the iterative edge-level optimization in Sec. 3.3. Equation (2) defines Lij = Σ_k w_k · f(s_k_ij), where each s_k_ij is an integer score in [-100,100] returned by ChatGPT-4o under Prompt 2. Equation (3) then updates the node feature vector as fi ← fi − η ∂Lij/∂fi. For this update to be computable, ∂Lij/∂fi must exist: the MLLM scores must be differentiable (or at least numerically well-defined) functions of the continuous attributes (x,y,z,scale,rotation). The paper supplies no such function: f(·) and w_k are unspecified, η is unspecified, and the MLLM is a discrete black-box text-to-score model. A discrete score does not yield a gradient with respect to fi unless one specifies finite differences, a differentiable surrogate, or a reward-to-gradient mapping; none is given. This is not merely the reader's point about noisy or non-monotonic scoring—even a perfectly accurate MLLM judge would not make Eq. (3) well-defined as written. If the actual implementation uses finite differences, it requires multiple MLLM queries per parameter per step, with explicit step sizes, tolerance, and query budgets, none of which are reported. If it uses MLLM-suggested parametric edits, then Eq. (3) is a misdescription of the algorithm. Either way, the claimed gradient-based optimization is unsupported by the manuscript, making the method irreproducible and the reported results unattributable to the described mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"GraphCanvas3D proposes a graph-structured, LLM/MLLM-driven framework for controllable 3D scene generation. Given a text prompt, an LLM constructs a scene graph whose nodes are object instances and edges are spatial relations; the method then renders multi-view images, queries an MLLM (ChatGPT-4o) for five integer scores per edge, and updates node features via a gradient-descent rule (Eq. 3). The final layout is rendered using Point-E and 3D Gaussian Splatting with MVDream/ControlNet. The paper reports improved CLIP and MLLM scores over five baselines, higher user-study ratings, and demonstrates object editing, addition/deletion, and 4D generation. The abstract and contributions emphasize usability, flexibility, and adaptability without retraining.","tokens_in":13928,"tokens_out":4185,"duration_ms":37607,"significance":"If the method operated as described, the graph-based, retraining-free formulation would be a useful contribution to controllable 3D scene generation, particularly the hierarchical edge/subgraph/graph optimization and the model-flexibility ablation. The paper provides template prompts in the appendix, a qualitative ablation study, and a transparent failure-case discussion, which are positive elements. However, the central optimization step is not mathematically well-defined as written, and the main automated metric is the same type of signal used for optimization, which makes the reported quantitative advantages difficult to interpret. The user study is a useful complement but cannot alone validate the method without a sound algorithmic description.","major_comments":[{"comment":"The gradient update is not well-defined. The loss Lij is a weighted sum of penalty functions f(s^k_ij) applied to integer scores produced by ChatGPT-4o; no differentiable mapping from the continuous node feature fi to these scores is provided. Consequently ∂Lij/∂fi does not exist as a mathematical object without additional assumptions such as finite differences, a surrogate model, or a specified stochastic estimator. The manuscript must either spell out the actual optimization algorithm (e.g., finite-difference step sizes, number of MLLM queries per parameter update, or a concrete mapping from scores to parameter edits) or correct the claim that the layout is obtained by gradient descent. As written, the core loop is not reproducible and the reported results cannot be attributed to the described mechanism.","section":"Section 3.3, Eqs. (2)-(3)"},{"comment":"The main automated evaluation metric, 'MLLM Score', is provided by a multimodal language model of the same kind (GPT-4o) that drives the optimization loop in Section 3.3. The system is therefore optimized against the same scoring signal that is used to evaluate it, which can inflate the apparent advantage; the baseline methods are not given any MLLM feedback loop. The paper should report the exact evaluation prompt, compare against an independent scoring model or human judgments with confidence intervals, and ideally measure the correlation between the MLLM score and layout accuracy. Without this, the quantitative superiority claim in Table 1 is not convincing.","section":"Section 4.1 and Table 1"},{"comment":"The claimed 4D scene generation capability is only described at a high level: 'an iterative process yields a temporal transformation sequence.' No details are given for how intermediate frames are produced, how temporal coherence is enforced, or how the 'state prompts' are turned into a 4D representation. Since 4D generation is listed as a contribution, this is a substantive omission that prevents verification and fair comparison.","section":"Section 3.6 and Figure 5"},{"comment":"The failure-case discussion states that 'objects move outside the camera's capture range during the optimization process, leading to errors in subsequent computations.' This is a known failure mode of the optimization loop, but the paper neither quantifies its frequency nor explains how scenes in the main results were filtered. A spatial optimization method should at least constrain objects to the view frustum or adapt the camera; otherwise the claimed robustness is not established.","section":"Supplementary Section 8 and Figure 8"}],"minor_comments":[{"comment":"The acronym 'MMLM' is used inconsistently; the correct term is MLLM (multimodal large language model).","section":"Abstract and Section 1"},{"comment":"Several references appear to be placeholders, e.g., [6] 'Jane Doe and John Smith' and [13] 'Alice Lee and Thomas Kim'; also [21] is cited for ChatGPT-4o but the listed reference is to the InstructGPT paper. These must be corrected before publication.","section":"References"},{"comment":"The implementation section reports the MVDream guidance scale but does not report the learning rate η, the edge-loss weights w_k, the penalty function f, or the convergence threshold appearing in Eqs. (2)-(3); these values are needed for reproducibility.","section":"Section 4 'Implementation details'"},{"comment":"The figure is dense and the labels for the five scores are hard to read; please enlarge the image and explicitly map each score to its corresponding optimization dimension (scale, x/y/z translation, rotation).","section":"Figure 3"},{"comment":"The user study reports mean ratings but does not include confidence intervals, inter-rater agreement, or significance tests; these should be provided to support the claimed superiority.","section":"Section 4.3, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's bibliographic integrity is a concern: several references are clearly placeholders and one citation does not match the claimed source. The core algorithmic issue in Section 3.3 is serious and likely requires the authors to reveal the actual implementation; if the method truly relies only on discrete MLLM scores, the gradient-descent formulation is not merely informal but formally undefined. The evaluation circularity with MLLM scores should be addressed before any claims of state-of-the-art performance are accepted. The 4D contribution is also far too underspecified. These issues are substantial but potentially fixable within a revision, so I recommend major_revision rather than rejection. The scope of the paper fits a vision/graphics venue if the methodology is clarified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the idea--MLLM-in-the-loop, hierarchical graph optimization for 3D layout--is worth a conversation, but the paper as written has two showstoppers: the references include fabricated entries, and the gradient update in Eq. (3) is not computable from anything in the paper.\n\nWhat's actually new: LayoutGPT and GALA3D already use LLMs to propose layouts; the new bit here is the iterative refinement loop where a multimodal LLM scores multi-view renders of edge, subgraph, and graph levels, and those scores drive adjustments to node position/scale/rotation. That is a legitimate extension. The paper also documents, in the supplement, a real failure case (objects leave the camera view and the loop breaks), which is more honest than most.\n\nWhat it does well: the system diagram is clear, the model-flexibility ablation shows the graph idea is not tied to a single LLM, and the user study, though small, asks sensible questions.\n\nWhere it falls apart:\n\n1. References [6] and [13] list 'Jane Doe' and 'Alice Lee' as authors of GALA3D and MVDream. Those papers exist, but their author lists are not Jane Doe/John Smith. This looks like placeholder citations that were never fixed, and it makes the baselines unverifiable. A serious referee cannot check the comparisons.\n\n2. Eq. (2) defines Lij as a weighted sum of penalty functions applied to integer scores from GPT-4o. Eq. (3) then updates fi by gradient descent on ∂Lij/∂fi. There is no differentiable mapping from the discrete scores to node features; no f, no wk, no η, no finite-difference scheme. The stress-test note is exactly right: the gradient is undefined as written. In practice the authors probably use a heuristic like 'score too positive => move left,' but the paper says gradient descent, and that claim is unsupported.\n\n3. The headline MLLM Score in Table 1 is produced by the same kind of model (GPT-4o) that runs the optimization loop. So the metric is not independent. CLIP Score is not sensitive to spatial relationships like 'on top of' versus 'next to.' The numbers, as presented, don't support the claim that the layouts are better by any objective measure.\n\nThe idea itself is salvageable. If the authors specify a concrete penalty/surrogate function, correct the citations, and evaluate with a held-out judge or human annotation with variance, this could become a decent paper. As it stands, I would not send it to referees; I'd send it back with a clear list of what has to change.","headline":"A potentially useful MLLM-in-the-loop layout idea, but fake references and an undefined gradient update make this unpublishable as written.","tokens_in":14505,"tokens_out":3986,"would_cite":false,"duration_ms":33373,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GraphCanvas3D claims that a hierarchical scene graph, iteratively refined by multimodal-LLM feedback on four rendered views, yields controllable 3D scenes and beats five text-to-3D baselines without retraining.","keywords":["3D scene generation","scene graph","multimodal LLM","layout optimization","3D Gaussian Splatting","controllable generation","4D scene generation","text-to-3D"],"falsifier":"A direct test: render a series of layouts that move a lamp in small increments toward the center of a table, and record the MLLM's left-right score each step; if the score does not monotonically approach zero as the lamp approaches the correct position, the gradient signal can be wrong. Alternatively, replace the MLLM's five scores with random values from the same range for the same number of iterations; if the layouts are indistinguishable from those produced with real feedback, the scores are not driving the improvement.","tokens_in":13336,"feed_emoji":"🎨","tokens_out":5980,"duration_ms":45865,"temperature":0.7,"pith_summary":"The paper tries to establish that controllable 3D scene generation can be achieved by representing the scene as a hierarchical graph—objects as nodes with position, scale, and rotation, spatial relations as edges—and then optimizing that graph in a loop driven by a multimodal LLM. The LLM inspects four rendered views of a pair of objects, scores how well they match the intended relation, and those scores are converted into gradient-like update signals. Because the graph is the only state, users can add, remove, or move objects by editing the graph and re-running the optimization. The paper reports that this approach outperforms five existing text-to-3D methods on CLIP score, MLLM score, and a 67-participant user study, and that it extends to 4D scenes with temporal changes.","feed_headline":"Beat five baselines with MLLM-scored scene graphs for 3D layout","feed_subtitle":"A graph of objects and relations is iteratively refined by a vision-language critic's scores, enabling on-the-fly edits and 4D scenes.","key_machinery":"The operating mechanism is the scene graph together with the edge-level optimization loop. Nodes carry a feature vector (position, scale, rotation); edges are the spatial relation and are optimized one at a time. For each edge, the method renders the two connected objects from four viewpoints, queries the MLLM with a fixed prompt to get five numerical scores, converts scores through a penalty function into a loss, and performs a gradient-descent-like update of the source node's feature vector. Subgraph and graph-level passes repeat the same pattern at larger scope, aggregating locally consistent groups and then adjusting them to keep the entire scene coherent.","core_discovery":"The central claim is that spatial coherence in multi-object 3D scenes can be achieved by representing the scene as a hierarchical graph and applying iterative, edge-wise optimization whose loss is provided by a multimodal LLM rather than by a hand-designed objective. Each object is a node with attributes; each spatial relation is a directed edge; edges are optimized by rendering four views, having the MLLM output five scores in [-100, 100] for scale, left-right, front-back, up-down, and yaw rotation, and mapping those scores through a penalty function into a weighted loss. The paper asserts that this MLLM-as-critic loop aligns layouts with human spatial intuition, and that the graph representation makes the framework programmable and extensible: objects can be added, removed, or repositioned, and temporal prompts yield 4D sequences, all without retraining.","pith_inferences":["If MLLM scores prove reliable, the same graph-scoring loop could serve as a general 3D editing interface for embodied agents, letting a robot re-plan a scene layout from a natural-language request without re-training.","The approach's ceiling is set by the MLLM's spatial judgment: four views of a scene may be insufficient for subtle relations or occlusions, so a testable extension is to feed more views or explicit depth images and see whether scores become more consistent.","The paper's failure case (objects moving outside the camera view) suggests a practical hardening: dynamically re-aiming cameras or constraining object positions to a view-frustum budget would make the loop more robust, but this is not addressed in the current design.","Because the loss is purely relative to what the MLLM considers good, the method could transfer layout priors across room types or object sets simply by changing the scene prompt, with no dataset collection needed."],"forward_implications":["Users can edit, add, or delete objects in an existing 3D scene through natural-language instructions, with the graph re-optimized to preserve coherence.","The framework supports 4D scene generation: prompts describing a transformation produce a sequence of state prompts that drive iterative graph optimization over time.","The framework is model-agnostic: ablations show the same graph-based pipeline works with GPT-4o, GPT-3.5, Claude, and LLaMA 3.2 as the language model, with Point-E or 3DGS plus MVDream as the 3D generator.","Ablation without edge, subgraph, or graph optimization produces visibly worse layouts, which the paper takes as evidence that each hierarchy level contributes to coherence.","Relying on MLLMs to produce interpretable, directional scores means the optimization signal is available off-the-shelf, avoiding task-specific retraining."],"supporting_citations":[{"why":"LayoutGPT provides the CSS-like prompt format and baseline layout generation that GraphCanvas3D compares against and uses for initial scene attribute estimation.","marker":"[8]"},{"why":"GALA3D is the main baseline; GraphCanvas3D adopts its 3D Gaussian Splatting parameter settings and benchmarks against it.","marker":"[6]"},{"why":"Point-E is the 3D generative model that produces point-cloud initializations for each object's 3D Gaussian Splatting process.","marker":"[20]"},{"why":"MVDream is used for high-quality object rendering and texturing, and also serves as a comparison baseline.","marker":"[13]"},{"why":"DreamFusion is cited as the source of the CLIP score evaluation protocol for text-to-3D alignment.","marker":"[24]"},{"why":"ControlNet provides the scene-rendering stage that keeps style and integration consistent across objects.","marker":"[37]"},{"why":"Cited as the ChatGPT-4o language model that performs graph construction and MLLM scoring; the actual reference text describes instruction-following pretraining.","marker":"[21]"}],"fun_headline_variants":["AI critic scores graph relations for real-time 3D scene control","GraphCanvas3D: MLLM-guided 3D scenes without retraining","Steer 3D scenes via graph nodes and MLLM-scored edges","No retraining needed: MLLM critic on graph drives 3D edits","4D scene generation with graph-based MLLM optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole optimization rests on the assumption that a multimodal LLM, looking at four rendered views, gives consistent and directionally correct numerical scores for each spatial adjustment, so that 'the lamp is a bit too far left' becomes a reliable gradient.","fun_headline_variants_meta":{"raw":{"variants":["AI critic scores graph relations for real-time 3D scene control","GraphCanvas3D: MLLM-guided 3D scenes without retraining","Steer 3D scenes via graph nodes and MLLM-scored edges","No retraining needed: MLLM critic on graph drives 3D edits","4D scene generation with graph-based MLLM optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000336,"raw_usage":{"total_tokens":1856,"prompt_tokens":938,"completion_tokens":918,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":820}},"tokens_in":554,"tokens_out":918,"duration_ms":8280,"temperature":1.0,"reasoning_tokens":820,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:19:29.984883+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test: render a series of layouts that move a lamp in small increments toward the center of a table, and record the MLLM's left-right score each step; if the score does not monotonically approach zero as the lamp approaches the correct position, the gradient signal can be wrong. Alternatively, replace the MLLM's five scores with random values from the same range for the same number of iterations; if the layouts are indistinguishable from those produced with real feedback, the scores are not driving the improvement.","supporting_citations":[{"cited_title":"Layoutgpt: Compositional visual plan- ning and generation with large language models","cited_arxiv_id":null,"evidence_quote":"LayoutGPT provides the CSS-like prompt format and baseline layout generation that GraphCanvas3D compares against and uses for initial scene attribute estimation."},{"cited_title":"Gala3d: Generative adversar- ial layout arrangement in 3d spaces","cited_arxiv_id":null,"evidence_quote":"GALA3D is the main baseline; GraphCanvas3D adopts its 3D Gaussian Splatting parameter settings and benchmarks against it."},{"cited_title":"Mvdream: Multi-view consis- tent 3d object generation from single-view images","cited_arxiv_id":null,"evidence_quote":"MVDream is used for high-quality object rendering and texturing, and also serves as a comparison baseline."},{"cited_title":"Training language models to follow instructions with human feedback","cited_arxiv_id":null,"evidence_quote":"Cited as the ChatGPT-4o language model that performs graph construction and MLLM scoring; the actual reference text describes instruction-following pretraining."}],"review_version":1}