{"id":"840bf189-6a09-4f8e-958f-73f11b448c90","arxiv_id":"2501.03490","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"SceneBooth keeps a provided subject image untouched and paints a new background around it, guided by a caption, object labels, and a predicted scene layout.","lead":"SceneBooth generates a new background around an input subject photo instead of regenerating the subject, so the subject's appearance stays pixel-identical. This could make personalized image generation more reliable for product advertising, selfie backgrounds, and catalog images where exact subject fidelity matters.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Subject-fidelity advantage is partly by construction (subject is pasted, never regenerated), and Table I's random-placement protocol confounds layout quality with PaintNet quality; the headline outperformance needs a matched-layout rerun.","rationale":"The reader's weakest-assumption entry focuses on the clean, unoccluded subject-image input and aspect-ratio constraints, which is a real limitation acknowledged in Section V and Fig. 8. However, the more load-bearing issue for the paper's central claim is that the quantitative comparison does not isolate the method's learned contribution. Because SceneBooth preserves the subject by pasting the exact input pixels, its subject-fidelity metrics are near-ceiling by construction; this is not a flaw in the method per se, but it means the claimed superiority over subject-generating baselines is not evidence of model skill. The random-placement protocol for baselines in Table I additionally conflates layout quality with background harmonization, so the headline 'significantly outperforms' is not supported by the reported numbers. The GT-layout ablation (Table II) is a step in the right direction and gives some independent support for PaintNet, but it still has an input asymmetry. A matched-layout, matched-conditioning rerun, plus a trivial paste baseline, would settle whether the claimed advantages are real. Therefore the existing CONDITIONAL verdict is appropriate, with the condition being a controlled re-evaluation rather than a mere addition of error bars.","tokens_in":15108,"tokens_out":3036,"duration_ms":32678,"concrete_test":"Run a controlled comparison on the COCO test set using ground-truth layouts: paste the subject crop into its GT box at the GT scale, then evaluate (a) SceneBooth with this layout, (b) ControlNet-inpaint and StableDiffusion-inpaint with the subject pre-pasted in the same box and prompted to inpaint only the complement, and (c) a trivial baseline that pastes the subject into a background generated by unmodified Stable Diffusion without subject conditioning. Report FID, CLIP-I, DINO, and human preference with 95% confidence intervals over multiple seeds. If SceneBooth's CLIP-I/DINO margin over (b) or (c) is negligible, the subject-fidelity advantage is constructional; if its FID margin over (b) collapses, the Table I result is an artifact of random placement.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that SceneBooth 'significantly outperforms baseline methods in terms of subject preservation, image harmonization and overall quality' (abstract; Section IV.B). The evidence for this claim is weakened by two coupled issues. First, Equations (2)-(3) show that the output subject is exactly R&P(Isub, l), rescaled and pasted, and the mask compositing never regenerates subject pixels. Consequently, the CLIP-I and DINO gains in Table I largely reflect a property of the task formulation, not the learned model: any method that pastes the input subject would inherit this advantage, and comparing against inpainting baselines that must synthesize a new subject does not test SceneBooth's components. Second, the Table I evaluation protocol deliberately places baseline subjects at random positions with 20% scale variation (Section IV.B note 2), while SceneBooth uses a learned layout. This confounds the layout module's contribution with the background-painting module's harmonization ability, so FID and human-preference differences cannot be attributed to PaintNet. The ground-truth-layout ablation in Table II is fairer, but it still compares PaintNet (with ControlNet subject conditioning) against GLIGEN variants without subject conditioning, so it does not cleanly isolate ControlNet or the instance-mask training contribution. The paper's own limitations (Section V, Fig. 8) also restrict the clean-subject assumption, but the evaluation mismatch is the more load-bearing threat to the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SceneBooth, a two-stage framework for subject-preserved text-to-image generation. Given a subject image, object phrases, and a scene caption, MCLayoutDM first generates a scene layout that determines the subject's position and scale along with boxes for other objects; PaintNet then synthesizes a background using a latent diffusion model conditioned on the layout, the caption, the object phrases, and the subject image injected via ControlNet and Gated Self-Attention. The final output composites the pasted subject with the generated background using a binary mask. The method is evaluated on COCO 2017 against inpainting baselines (StableDiffusion-inpaint and ControlNet-inpaint) and through ablations of ControlNet, mask strategy, attention type, and visual embedding. The headline claim is that SceneBooth significantly outperforms baselines in subject preservation, image harmonization, and overall quality.","tokens_in":15410,"tokens_out":2697,"duration_ms":26684,"significance":"If the evaluation were fully convincing, SceneBooth would make a useful contribution by showing that avoiding subject resynthesis altogether is a practical route to high-fidelity customization, complementary to DreamBooth-style finetuning. The paper is clearly written, the two modules are sensible, and the open-world examples suggest the framework generalizes beyond COCO categories. The ablations (Tables II-V) provide useful evidence about the design choices, and the authors honestly disclose limitations in Section V. However, the central comparative claim is weakened by an evaluation protocol that conflates layout quality with background-painting quality and by subject-fidelity metrics that partly reflect the copy-and-paste formulation rather than the learned components. The contribution is still credible and potentially publishable, but the experimental evidence needs reworking before the strong superiority claim can be accepted.","major_comments":[{"comment":"The random-placement protocol for the inpainting baselines, stated in note (2) of Section IV.B, confounds the layout module's contribution with the background-painting module's contribution. Baselines receive subjects at random positions with 20% scale variation while SceneBooth uses its learned layout, so the FID and human-preference differences cannot be attributed to PaintNet alone. The ground-truth-layout ablation in Table II is fairer, but it compares PaintNet only against GLIGEN variants, not against the same inpainting baselines used in Table I. I recommend rerunning Table I with all methods given identical layouts (either ground-truth or SceneBooth-generated), so that the comparison isolates background-painting quality from layout prediction.","section":"Section IV.B, Table I"},{"comment":"The CLIP-I and DINO advantages in Table I are partly by construction. Equation (3) shows that the output subject pixels are exactly the rescaled and pasted input subject, never regenerated by the diffusion model. Any method that pastes the input subject without any learned background generation would inherit most of this subject-fidelity benefit, so these metrics do not measure the learned model's contribution to subject preservation. I suggest adding a paste-only baseline (subject pasted onto a blank or trivial background) and reporting subject-fidelity metrics restricted to the background region, or otherwise quantifying the added value of PaintNet beyond the copy-and-paste operation.","section":"Equations (2)-(3), Table I"},{"comment":"The ablation intended to isolate the effect of ControlNet does not do so cleanly. GLIGEN-repaint and GLIGEN-inpaint receive no visual information about the subject, whereas PaintNet receives the subject image through ControlNet. The large differences in CLIP-I, DINO, and human preference could therefore be due to the mere presence of subject appearance conditioning rather than to ControlNet specifically. A stronger ablation would give the GLIGEN variants access to the subject's visual features through some other mechanism (for example, concatenating the subject image as an extra conditioning channel), or would compare PaintNet to a variant that replaces ControlNet with a different image-conditioning adapter while keeping the same information available.","section":"Section IV.C, Table II"},{"comment":"The paper's own limitations section, together with Fig. 8, shows that the framework assumes a clean, unoccluded, single subject whose bounding box is compatible with its aspect ratio (also stated in Section III.A). When these assumptions are violated, the paste-and-blend strategy leads to visible seams or irrational layouts, which undermines the general harmonization claim. This scope restriction should be stated as a central limitation in the abstract or introduction, and the paper should discuss or at least acknowledge potential mitigations (e.g., matting, subject-hole inpainting) rather than leaving the issue as a peripheral failure case.","section":"Section V, Limitations"}],"minor_comments":[{"comment":"The text refers to 'Figure 5.7' when discussing open-world generation, but the actual figure is Fig. 7; please correct the cross-reference.","section":"Section IV.D"},{"comment":"The token selection operation TS is used in Eq. (12) but never defined before its first use; please define it in the text or in the equation caption.","section":"Section III.D, Eq. (12)"},{"comment":"The dataset name is written as 'COCO2017' in several places; use 'COCO 2017' for consistency with the reference [22].","section":"Section IV.A"},{"comment":"References [6] and [29] both cite the same DisenBooth paper; please consolidate or differentiate them.","section":"References"},{"comment":"The extension to Max. IoU @ k is described as detailed in supplementary materials, but no supplementary material is included in the arXiv submission; please include the detail or remove the reference.","section":"Section IV.A"},{"comment":"The human-evaluation metrics Pquality, Pfidelity, Pobjf, and Ppromptf are introduced in the text but not defined in the table caption; adding a one-line definition would improve readability.","section":"Table I caption"}],"recommendation":"major_revision","confidential_remarks":"The reader's concern about circularity is not, in my assessment, a correctness problem: the use of LayoutDM as the backbone is disclosed, and the contribution is incremental but legitimate. The load-bearing issue is the evaluation protocol: the Table I comparison is unfair to the inpainting baselines because of random subject placement, and the subject-fidelity metrics partially measure the copy-and-paste operation rather than the learned model. These concerns are fixable with a matched-layout rerun and a paste-only baseline. The paper's scope is a reasonable fit for a computer vision venue, though the novelty relative to LayoutDM plus ControlNet/GLIGEN is modest."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SceneBooth is a serious attempt at a genuinely different task: instead of regenerating a subject like DreamBooth, it pins the subject pixels and paints the background around them. That reframing is the paper's real contribution, and it largely works. The two-stage design is clean, the instance-mask training for the ControlNet branch is a sensible new twist, and the qualitative examples show coherent blends. The authors also deserve credit for stating their input assumptions and limitations plainly in Section V.\n\nThe soft spot is the evaluation, and it is load-bearing. The baseline comparison in Table I gives the inpainting methods random positions and 20% scale jitter because they cannot infer layout, while SceneBooth uses its learned layout. That confounds layout quality with PaintNet's background-painting quality, so the FID and human-preference gaps cannot be attributed to the painting module. Worse, the headline fidelity metrics (CLIP-I, DINO) are partly won by construction: Equations (2)-(3) show the subject is pasted, never regenerated, so any method that pastes the subject would inherit that advantage. The claim that SceneBooth \"significantly outperforms\" on subject preservation is therefore not tested against an honest baseline.\n\nThe ablation in Table II with ground-truth layouts is fairer and supports the ControlNet contribution, but it still compares against GLIGEN variants without subject conditioning, so it does not cleanly isolate ControlNet. I would like to see a matched-layout rerun of the inpainting baselines, or at least a version where the baselines receive the same bounding box, plus error bars and significance tests. The code and data splits are promised but not yet available.\n\nThis is not a paper that should be rejected for its core idea; the problem is real and the direction is worth pursuing. But the current evidence does not support the strong outperform claim, and the evaluation needs reworking. A serious referee could help the authors get there.\n\nRecommendation: accept for peer review, with the expectation of requested revisions on the evaluation protocol.","headline":"Solid task reframing with a promising two-stage design, but the headline outperformance claim is undermined by an unfair baseline protocol and must be re-evaluated.","tokens_in":15939,"tokens_out":2196,"would_cite":true,"duration_ms":19994,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SceneBooth claims that fixing the subject image and generating only a layout-guided background preserves subject fidelity better than methods that regenerate the subject from a learned representation.","keywords":["subject-preserved text-to-image generation","layout generation","diffusion models","latent diffusion","ControlNet","gated self-attention","subject fidelity","background inpainting"],"falsifier":"Run SceneBooth on a subject image that includes background clutter or a pose that conflicts with the generated layout, such as a lying person assigned a standing-object box; if the output shows a visible cut-out seam around the subject or an implausible object relation, the claimed harmonization is falsified for that input class. A simpler quantitative check is to feed the same subject image with and without a tight, background-free crop and compare DINO and CLIP-I scores along with boundary artifact rates.","tokens_in":14901,"feed_emoji":"🎨","tokens_out":5303,"duration_ms":46816,"temperature":0.7,"pith_summary":"SceneBooth is a framework for subject-preserved text-to-image generation: given one subject image, a scene caption, and phrases describing the other objects, it keeps the subject pixels exactly as provided and generates only the surrounding scene. The paper's central claim is that this \"fix the subject, paint the background\" strategy sidesteps the fidelity-diversity trade-off that plagues subject-driven methods such as DreamBooth, so logos, textures, and fine details survive unchanged. The framework first predicts a scene layout that positions and scales the subject and other objects, then paints a background that harmonizes with the pasted subject using a latent diffusion model with layout and visual control. If the claim holds, personalized image generation for products, selfies, and illustrations becomes a matter of background synthesis rather than subject resynthesis.","feed_headline":"SceneBooth preserves subjects by never regenerating them","feed_subtitle":"A layout-guided diffusion model paints only the background, keeping logos and fine details pixel-identical.","key_machinery":"Two components carry the argument. MCLayoutDM is a diffusion-based layout denoiser built on LayoutDM, extended with a cross-attention layer so that a CLIP text embedding of the caption and a CLIP visual embedding of the randomly scaled and padded subject image guide the denoising of object bounding boxes. PaintNet is a latent diffusion model built on LDM and Stable Diffusion, into which ControlNet injects a conditioning image of the subject pasted on a blank canvas and gated self-attention layers inject grounding tokens derived from object phrases and bounding boxes; its defining design choice is training with instance masks rather than random masks, so the model learns that the known region is a complete subject.","core_discovery":"On its own terms, the paper argues that precise subject preservation is better achieved by construction than by representation learning: rather than embedding the subject into the prompt and regenerating it, SceneBooth treats the input subject image as fixed foreground and treats text-to-image generation as a conditional background-painting problem. The layout module conditions a diffusion layout denoiser on CLIP text and image features, producing bounding boxes for the subject and every background object; the painting module then feeds a canvas with the subject pasted at its predicted box into a latent diffusion model via ControlNet, while gated self-attention layers inject layout tokens. Training PaintNet with instance masks, where the known region is the whole subject rather than a random hole, teaches the model that the unmasked area is a complete subject whose surroundings must blend with it. The reported outcome is that SceneBooth beats inpainting-based baselines on FID, CLIP-I, and DINO, and wins the majority of human preference judgments on quality, fidelity, object-phrase match, and caption alignment.","pith_inferences":["The paper's own limitations section implies that its central claim is conditional on the subject image being clean: adding a segmentation step that strips background pixels before pasting is a direct, testable extension that should remove the reported occlusion and seam failures.","Because the method assumes exactly one preserved subject, a natural multi-subject extension is to run layout generation per object and composite several fixed foregrounds one at a time; the paper does not attempt this.","The comparison set is inpainting models rather than subject-driven personalization models, so the broadest inference, that fixed pixels are a stronger fidelity prior than learned subject embeddings, needs a direct head-to-head with DreamBooth-style methods under identical prompts to be confirmed."],"forward_implications":["If the claim is correct, subject fidelity no longer competes with prompt diversity: the subject is literally the input pixels, so logos and fine textures cannot drift.","Object-level layout prediction makes counting and positional relationships explicit, which the paper's human evaluations show standard CLIP-T alignment scores underweight.","Because the background painter inherits a large pretrained latent diffusion model, SceneBooth can place unseen subjects, such as a Hello Kitty doll, into novel scenes without retraining for that subject.","Editing the subject's bounding box acts as a drag control, repositioning the subject while preserving its appearance.","The measured gap between random-mask and instance-mask training suggests the same painting module could serve other \"complete foreground, synthesize surroundings\" tasks."],"supporting_citations":[{"why":"Supplies the diffusion-based layout generation backbone that MCLayoutDM extends with multimodal conditioning.","marker":"[19]"},{"why":"Provides the pretrained CLIP text and image encoders used to build the multimodal embeddings.","marker":"[20]"},{"why":"Supplies ControlNet, the adapter that injects the pasted-subject conditioning image into PaintNet.","marker":"[14]"},{"why":"Supplies the gated self-attention layers and grounding tokens that inject layout information into the diffusion model.","marker":"[21]"},{"why":"Supplies the latent diffusion model and Stable Diffusion weights that PaintNet builds on.","marker":"[15]"},{"why":"Supplies the COCO dataset used for training and evaluation.","marker":"[22]"},{"why":"Serves as the representative subject-driven generation method whose fidelity failures motivate the subject-preserved setting.","marker":"[2]"},{"why":"Provides the masked denoising inpainting strategy that the paper contrasts with its instance-mask training approach.","marker":"[17]"}],"fun_headline_variants":["SceneBooth paints your background, never your subject","Keep the subject, regenerate the world: SceneBooth","Background-only diffusion: SceneBooth preserves subject detail","Layout-guided painting: SceneBooth avoids subject redraw","SceneBooth locks the subject, then dresses the scene"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes the user supplies a single, clean, unoccluded subject image whose aspect ratio fits its predicted bounding box, plus object phrases for every other scene object; if the subject carries background pixels, is partly occluded, or is squeezed into a mismatched box, the pasted foreground cannot blend cleanly and the harmonization claim breaks down.","fun_headline_variants_meta":{"raw":{"variants":["SceneBooth paints your background, never your subject","Keep the subject, regenerate the world: SceneBooth","Background-only diffusion: SceneBooth preserves subject detail","Layout-guided painting: SceneBooth avoids subject redraw","SceneBooth locks the subject, then dresses the scene"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000281,"raw_usage":{"total_tokens":1693,"prompt_tokens":1001,"completion_tokens":692,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":613}},"tokens_in":617,"tokens_out":692,"duration_ms":6687,"temperature":1.0,"reasoning_tokens":613,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:53:02.752351+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SceneBooth on a subject image that includes background clutter or a pose that conflicts with the generated layout, such as a lying person assigned a standing-object box; if the output shows a visible cut-out seam around the subject or an implausible object relation, the claimed harmonization is falsified for that input class. A simpler quantitative check is to feed the same subject image with and without a tight, background-free crop and compare DINO and CLIP-I scores along with boundary artifact rates.","supporting_citations":[{"cited_title":"Layoutdm: Transformer-based diffusion model for layout generation,","cited_arxiv_id":null,"evidence_quote":"Supplies the diffusion-based layout generation backbone that MCLayoutDM extends with multimodal conditioning."},{"cited_title":"Gligen: Open-set grounded text-to-image generation,","cited_arxiv_id":null,"evidence_quote":"Supplies the gated self-attention layers and grounding tokens that inject layout information into the diffusion model."}],"review_version":1}