{"id":"5810bb35-b154-4937-a7e4-fb2ed44d78fb","arxiv_id":"2505.04718","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"LayouSyn is a text-to-layout pipeline using a lightweight open-source LLM for object extraction and an aspect-aware diffusion Transformer for bounding-box generation, reporting SOTA on NSR-1K and COCO-GR layout metrics.","lead":"This paper presents a system that turns a sentence like \"two men posing on a street\" into a picture plan: a list of objects and the boxes where each one goes. The system uses a small free language model to list the objects and a diffusion model to place them, aiming to replace paid AI services for this step.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NSR-1K spatial prompts are used for training and then evaluated on the same benchmark with no documented split, so the headline spatial-reasoning SOTA may reflect memorization rather than generalization.","rationale":"The paper describes a reasonable two-stage pipeline and includes useful ablations (noise-schedule scaling, LDiT components, DDIM steps), but the experiment that backs the abstract's strongest claim has a training/evaluation overlap. The reader's weakest_assumption identifies this, and the manuscript itself exposes it: Section 4.1 trains on NSR-1K spatial, Section 4.4 evaluates on NSR-1K, and no split is described anywhere. This is a correctness risk, not just a norm violation, because a diffusion model conditioned on prompt tokens can partially memorize the layout positions for the 738 training prompts. The numerical-reasoning portion is less affected, but the abstract does not separate the two. I agree with the reader's assessment and would keep the REJECT verdict; the issue is addressable with a clean held-out evaluation, so a revised version could still establish the claim.","tokens_in":14454,"tokens_out":7182,"duration_ms":68409,"concrete_test":"Retrain LayouSyn under the same recipe but with the NSR-spatial subset removed from the training mixture (GRIT pretraining plus COCO-GR finetuning only), then rerun the spatial-reasoning rows of Table 3 on the NSR-1K benchmark. If spatial accuracy remains around 92.58%, the training overlap is not the driver; if it falls toward the LayoutGPT / Llama-3.1-8B baselines (87.07 / 86.64), the published result is contaminated. The authors should also release an explicit NSR-1K train/test split and report the held-out row.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 states: 'We use the NSR-1K spatial dataset proposed in LayoutGPT [12] to train our model for understanding the spatial relationship between objects present in the scene. The dataset contains 738 prompts describing four spatial relations: above, below, left, and right between two objects in the scene.' Section 4.4 then reports results on the NSR-1K benchmark and highlights '92.58% accuracy in spatial reasoning.' The paper never documents a train/test split, a held-out subset, or any exclusion of NSR-spatial prompts from training. Because NSR-1K was introduced as a zero-shot evaluation benchmark for LLM-based planners (LayoutGPT), training on it and then reporting state-of-the-art spatial numbers is circular. The numerical-reasoning column is less directly contaminated because only the spatial subset is mentioned in Section 4.1, but the abstract's SOTA claim bundles both benchmarks, and the spatial half is where the largest margin over baselines appears. This is the load-bearing weakness for the central claim: if the spatial accuracy is memorized, the comparison against proprietary-LLM baselines is not evidence of open-vocabulary generalization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"LayouSyn is a two-stage text-to-layout pipeline: a lightweight open-source LLM (Llama-3.1-8B) extracts object descriptions with counts, and an aspect-aware diffusion Transformer (LDiT) generates bounding boxes conditioned on the prompt, the object descriptions, and the aspect ratio. The paper evaluates on COCO-GR layout quality (L-FID), on the NSR-1K spatial and numerical reasoning benchmark, and in two applications (LLM initialization and object addition to images). The central claim is that LayouSyn achieves state-of-the-art spatial and numerical reasoning while using only lightweight open-source components, making controllable image generation cheaper and more transparent than proprietary-LLM-based approaches.","tokens_in":14652,"tokens_out":9382,"duration_ms":89322,"significance":"If the spatial and numerical results were obtained on properly held-out data, the contribution would be meaningful: an open-vocabulary, transparent, and cost-effective alternative to proprietary-LLM layout generators. The COCO-GR L-FID comparison, the noise-schedule scaling analysis, and the ablations that separate description-set generation from layout generation are informative and show careful engineering. However, the central spatial-reasoning claim is currently undermined by benchmark contamination, and the numerical SOTA claim is only partially supported by the reported metrics. Until the evaluation is corrected, the magnitude of the reported gains cannot be taken as evidence of open-vocabulary generalization.","major_comments":[{"comment":"The model is trained on the NSR-1K spatial dataset (738 prompts, Sec. 4.1) and then evaluated on the NSR-1K benchmark (Sec. 4.4, Table 3) with no documented train/test split or exclusion of these prompts from training. The NSR-1K benchmark was introduced as a zero-shot evaluation set for LLM-based planners, so the reported 92.58% spatial accuracy is not evidence of open-vocabulary generalization unless a held-out portion is shown. This also makes the comparison against zero-shot LayoutGPT baselines unfair, since LayouSyn has seen the spatial prompts during training. Please provide a proper split, or retrain without NSR-spatial and report the resulting spatial accuracy; this is a necessary condition for the abstract's spatial SOTA claim.","section":"Sec. 4.1 / Sec. 4.4 / Table 3"},{"comment":"The abstract and Section 1 claim state-of-the-art numerical reasoning, but Table 3 shows that on the numerical GLIP accuracy metric LayouSyn (56.17) and LayouSyn-GRIT (56.20) are below LayoutGPT (GPT-4o-mini) (57.96). The numerical SOTA claim is therefore only supported for Precision/Recall/Accuracy, not for the GLIP-based metric; please qualify the claim or explain why the GLIP column should not be considered when assessing numerical reasoning.","section":"Sec. 4.4, Table 3"},{"comment":"The proof of the noise-schedule scaling is inconsistent as written. Eq. (5) defines the normalized process with a sqrt(tilde-alpha_t) coefficient, but Eq. (6) sets tilde-alpha_t equal to that coefficient itself (s sqrt(alpha_t)/sqrt((s^2-1)alpha_t+1)). The variance parameter corresponding to Eq. (3) is s^2 alpha_t/((s^2-1)alpha_t+1), not the expression in Eq. (6). Please correct the theorem or align the notation between Eq. (3) and the appendix.","section":"Appendix A.1, Eq. (6)"}],"minor_comments":[{"comment":"The L-FID evaluation uses the first 8,700 captions from the COCO-GR validation set; please specify whether this is a random sampled subset and report the subset construction, since an ordered prefix may introduce selection bias.","section":"Sec. 4.3"},{"comment":"The table note says 'shaded columns represent metrics computed directly on the generated layouts', but it is unclear which columns are shaded in monochrome printing; please make the notation explicit.","section":"Table 3"},{"comment":"The L-FID improvement from cross-attention and modulation is small (2.82 to 2.74); please report variance or statistical significance before claiming that these architectural components are responsible for the improvement.","section":"Table 7"},{"comment":"The affiliation 'Tsingua University' appears to be a typo for 'Tsinghua University'.","section":"Page 2, affiliations"}],"recommendation":"major_revision","confidential_remarks":"The reader's reject recommendation is understandable, but the central flaw is fixable in principle by re-running the evaluation with a held-out split or by retraining without NSR-spatial and comparing against baselines under the same data conditions. If the corrected spatial numbers collapse, a reject recommendation would be appropriate. I would also ask the editor to weigh the abstract's overstatement of numerical SOTA, since Table 3's GLIP column already contradicts it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper has a real contribution: a two-stage text-to-layout pipeline using a small open-source LLM for description-set extraction and an aspect-aware diffusion Transformer (LDiT) that conditions on global prompt, local descriptions, and aspect ratio. The LDiT block with cross-attention between global and local tokens is a genuine architectural novelty, and the noise-schedule scaling derivation in the appendix is formal and correct. The COCO-GR dataset construction is also useful, and the two applications (LLM initialization, object addition) are plausible and well-motivated.\n\nThe soft spot is exactly the one flagged: Section 4.1 trains on the NSR-1K spatial dataset (738 prompts), and Section 4.4 reports SOTA on the NSR-1K benchmark with no documented train/test split. That is a load-bearing circularity for the spatial reasoning claim. The numerical reasoning column is less directly contaminated because only the spatial subset is mentioned for training, but the abstract's SOTA claim bundles both, and the spatial margin is where the big gains appear. Without a held-out spatial test set, the 92.58% spatial accuracy cannot be taken as evidence of open-vocabulary generalization. The finetuned Llama baseline is trained on the same data, so the comparison against zero-shot LLM baselines is misleading.\n\nThe L-FID comparison also has some ambiguity: LayoutGPT is given COCO-GR training captions as in-context exemplars, and the evaluation is on the first 8,700 validation captions without specifying the split of COCO-GR. That is a minor concern relative to the NSR issue, but worth clarifying.\n\nThe rest of the paper holds up: the ablations are informative, the fixed-description-set comparison shows the layout model itself contributes, and the appendix proof is correct. I don't see equation-level circularity or hidden parameters beyond the usual diffusion sampling choices.\n\nAs written, the central claim is not supported. But this is fixable with a clean held-out evaluation and a corrected abstract. The paper deserves a serious referee, not a desk reject. Send it to review, ask for a documented split on NSR-1K, and let the authors revise.","headline":"A solid architectural contribution undermined by a training/evaluation overlap on NSR-1K that makes the headline spatial SOTA unsupported as written.","tokens_in":15215,"tokens_out":2879,"would_cite":false,"duration_ms":27731,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small open LLM plus an aspect-aware diffusion Transformer generates scene layouts from text","keywords":["text-to-layout generation","open-vocabulary layout generation","diffusion transformer","spatial reasoning","numerical reasoning","controllable image generation","NSR-1K"],"falsifier":"Compare the NSR-1K prompts used in training against the benchmark's evaluation prompts; if any overlap exists, retrain on a disjoint subset and measure spatial accuracy. A large drop from the reported 92.58% would show the result is memorization rather than generalization.","tokens_in":14203,"feed_emoji":"🖼️","tokens_out":8148,"duration_ms":69891,"temperature":0.7,"pith_summary":"This paper tries to establish that open-vocabulary scene layout generation does not need proprietary large language models. It proposes a two-stage pipeline, LayouSyn, in which a small open-source Llama model turns a text prompt into a set of object names with counts, and an aspect-ratio-aware diffusion Transformer turns those names into bounding boxes. The paper reports that this pipeline beats proprietary-LLM baselines on the NSR-1K spatial and numerical reasoning benchmark and lowers Layout-FID on a grounded COCO dataset, while using only an 18M-parameter denoiser. If the result holds, controllable image generation becomes cheaper and more transparent, and object parsing can be cleanly separated from geometric placement.","feed_headline":"Open tools beat proprietary LLMs at scene layout generation","feed_subtitle":"A small Llama model and an 18M-parameter diffusion Transformer place objects that obey spatial and counting rules.","key_machinery":"The load-bearing component is the layout diffusion Transformer (LDiT) block. It concatenates bounding-box tokens and T5-encoded object-description tokens, modulates the description tokens with the same adaptive layer norm used for scalar conditions, and lets description tokens attend to the global prompt through an added cross-attention layer, so local object information is aligned with the scene-level text. The architecture also makes generation aspect-ratio aware by normalizing box coordinates to [-1,1] and feeding the ratio W/H as a scalar condition. A second mechanism is the scaled noise schedule of Eq. (3), which slows the destruction of box-coordinate information in early diffusion steps; the paper uses scale s=2.0 and classifier-free guidance 2.0 after ablating both.","core_discovery":"On its own terms, LayouSyn's central claim is that decomposing text-to-layout into a language step and a geometry step, with a diffusion Transformer trained directly on bounding-box coordinates, achieves better spatial and counting accuracy than layouts written entirely by a proprietary LLM. The language step is a prompted open-source Llama-3.1-8B model that outputs a JSON description set of noun phrases and counts. The geometry step is an 18M-parameter LDiT denoiser that adds noise to box coordinates, normalizes coordinates by layout size for aspect-ratio-agnostic training, conditions on the aspect ratio and timestep via adaLN, and aligns local description embeddings with the global T5 prompt embedding through cross-attention and modulation. A scaled noise schedule with s=2.0 keeps low-dimensional box information from being destroyed too early. The paper reports 92.58% spatial accuracy and 95.14% numerical accuracy on NSR-1K, L-FID 3.07 on COCO-GR, and improvements over GPT-based LayoutGPT with equal or fewer sampling steps.","pith_inferences":["Editorial inference: if the NSR-1K prompts used for training are also present in the reported evaluation set, the spatial-accuracy numbers may partly reflect memorization; a disjoint held-out split would settle this.","Editorial inference: the scaled noise schedule should transfer to other low-dimensional diffusion targets where information is destroyed quickly, such as keypoints or object-detection boxes.","Editorial inference: the paper's separation of noun-phrase parsing from geometric placement suggests a modular recipe in which compact specialist models can replace proprietary LLM components in other controllable-generation pipelines.","Editorial inference: LLM initialization followed by DDIM inversion may work as a general refinement operator for coarse structured predictions beyond layouts, such as 3D boxes or scene graphs."],"forward_implications":["Open-vocabulary layout generation works with a lightweight open-source LLM in the loop; the paper reports that Llama-3.1-8B description sets give lower L-FID than GPT-3.5 or GPT-4o-mini description sets.","The layout denoiser alone contributes most of the gain: when given the same description sets as LayoutGPT, LayouSyn cuts L-FID by up to 50.1%.","Coarse layouts from any LLM can be refined by DDIM inversion in about 15 steps, improving spatial accuracy over the LLM's raw output.","The same model can be used for automated object addition: detect existing objects, let LayouSyn complete the layout for the new object, and inpaint it with GLIGEN.","Because the noise schedule is built into alpha_t rather than into input normalization, the method is usable with standard stable training at any aspect ratio."],"supporting_citations":[{"why":"provides the NSR-1K spatial and counting benchmark, the LayoutGPT baselines, and the comparison metrics the paper reports against.","marker":"[12]"},{"why":"supplies the diffusion Transformer (DiT) block architecture that the LDiT denoiser is built from.","marker":"[36]"},{"why":"supplies the DDPM forward process and the noise-prediction training objective used to train the layout denoiser.","marker":"[18]"},{"why":"supplies Llama-3.1-8B, the lightweight open-source LLM used to extract description sets from prompts.","marker":"[11]"},{"why":"supplies the grounded image-text data construction recipe used to create COCO-GR and the GRIT pretraining set.","marker":"[37]"},{"why":"supplies the Sentence-T5 encoder that embeds each object description into a token for the diffusion Transformer.","marker":"[32]"},{"why":"supplies GLIGEN, used to render layouts into images for evaluation and to inpaint objects in the object-addition pipeline.","marker":"[26]"},{"why":"supplies the FID metric used as the layout-quality score (L-FID).","marker":"[16]"},{"why":"supplies GroundingDINO, used to detect object boxes in captioned images for COCO-GR and to detect existing objects for object addition.","marker":"[29]"}],"fun_headline_variants":["Open-source pipeline beats proprietary LLMs on scene layouts","Diffusion transformer sets new bar for layout reasoning","Lightweight models nail spatial and counting tasks","Lay-Your-Scene: open tools, SOTA layout generation","Efficient open-source layout model outperforms GPT-based"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central performance claim assumes the NSR-1K benchmark is a held-out test set, yet the paper trains on the NSR-1K spatial prompts and never documents a train/test split before reporting spatial accuracy on that same benchmark.","fun_headline_variants_meta":{"raw":{"variants":["Open-source pipeline beats proprietary LLMs on scene layouts","Diffusion transformer sets new bar for layout reasoning","Lightweight models nail spatial and counting tasks","Lay-Your-Scene: open tools, SOTA layout generation","Efficient open-source layout model outperforms GPT-based"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1453,"prompt_tokens":930,"completion_tokens":523,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":455}},"tokens_in":546,"tokens_out":523,"duration_ms":5567,"temperature":1.0,"reasoning_tokens":455,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:22:38.759441+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the NSR-1K prompts used in training against the benchmark's evaluation prompts; if any overlap exists, retrain on a disjoint subset and measure spatial accuracy. A large drop from the reported 92.58% would show the result is memorization rather than generalization.","supporting_citations":[{"cited_title":"The llama 3 herd of models, 2024","cited_arxiv_id":null,"evidence_quote":"supplies Llama-3.1-8B, the lightweight open-source LLM used to extract description sets from prompts."},{"cited_title":"Kosmos-2: Grounding multimodal large language models to the world, 2023","cited_arxiv_id":null,"evidence_quote":"supplies the grounded image-text data construction recipe used to create COCO-GR and the GRIT pretraining set."},{"cited_title":"Hall, Daniel Cer, and Yinfei Yang","cited_arxiv_id":null,"evidence_quote":"supplies the Sentence-T5 encoder that embeds each object description into a token for the diffusion Transformer."},{"cited_title":"Gligen: Open-set grounded text-to-image generation, 2023","cited_arxiv_id":null,"evidence_quote":"supplies GLIGEN, used to render layouts into images for evaluation and to inpaint objects in the object-addition pipeline."}],"review_version":1}