{"id":"0b8466c2-e78f-4e0d-ab67-9a01456528c1","arxiv_id":"2412.00427","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FreeCond adjusts only the image and mask inputs of Stable Diffusion Inpainting, improving prompt adherence and mask fitting without training or extra compute.","lead":"A training-free tweak to the input conditions of Stable Diffusion Inpainting improves how well the model follows complex text prompts and mask shapes. The method filters the image condition and scales the mask condition, showing large gains on a new benchmark but much smaller gains on existing ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 60% CLIP gain is measured on FCIBench after tuning FreeCond's four hyperparameters on FCIBench; without a held-out split or fixed configuration, the 'free lunch' generalization claim is not established.","rationale":"I read the claim as: a training-free, input-only modification gives large gains on hard inpainting and is compatible with any SDI-based model. What must be true is that the reported gains are not artifacts of benchmark-specific tuning or of evaluation-protocol changes. The reader's conditional verdict captures this well, and I agree with that assessment. The strongest evidence in favor is the consistent, if small, gains on COCO and BrushBench across multiple backbones in Tab. 2, and the CI-indicator analysis is suggestive rather than definitive. However, the headline 60%/58% numbers come from FCIBench, which the authors constructed. The ablation study in Fig. 10 selects α, β, γ, t_fc using the same benchmark, with no held-out split. The 60% number is a relative increase from a very low baseline (11.45 to 18.27), so it is less impressive in absolute terms; the absolute gains on standard benchmarks are below 1 CLIP point. In addition, the mask scaling M_fc with β > 0 acts as a soft dilation, and Sec. 3.3 already showed dilation improves CLIP/IoU on hard masks; without a dilation baseline in Tab. 2, the novel component is not isolated. The paper's own footnote says the arXiv version is abbreviated and defers full settings and code to a GitHub repository, so the per-baseline hyperparameter choices in Tab. 2 cannot be independently audited from the manuscript. These are addressable by a held-out split and code release, so the verdict should remain CONDITIONAL rather than REJECT. The concern is about under-supported generalization, not about the plausibility of the mechanism or the integrity of the authors.","tokens_in":15221,"tokens_out":7125,"duration_ms":72139,"concrete_test":"Split FCIBench's 600 pairs into a tuning set and a held-out set. Select (α, β, γ, t_fc) using only the tuning set, then recompute the SDI and SDXL FCIBench CLIP columns of Tab. 2 on the held-out set. If the held-out CLIP gain approaches the reported 60%, the headline is robust; if it falls to the ~1% level seen on COCO/BrushBench, the headline is an artifact of on-benchmark hyperparameter selection. As a secondary check, include a simple binary-dilation baseline for M_c in the same comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is the 60% CLIP-score improvement of SDI on FCIBench (Abstract; Tab. 2, right column). The support is undercut by selection on the evaluation set: the four FreeCond hyperparameters (α, β, γ, t_fc) are chosen via ablations shown in Fig. 10, and those ablations report CLIP/IoU on the same three benchmarks including FCIBench. No held-out split, no error bars, and no pre-registered configuration are provided. Because the FCIBench baseline CLIP is extremely low (11.45), the 60% relative gain is a 6.8-point absolute change, while the gains on COCO and BrushBench are only 0.15 and 0.05 CLIP points; the headline is therefore dominated by the self-built benchmark. A second confound: M_fc = α M_c + β (1 − M_c) with β > 0 is a soft dilation of the mask, and Sec. 3.3 already shows that dilating the mask improves CLIP/IoU on exactly the hard-instruction regime. No baseline using simple mask dilation is included in Tab. 2, so the specific contribution of the proposed condition modulation is not isolated. The abbreviated arXiv text also defers the full settings and code to an unverifiable external repository, so the per-baseline hyperparameter choices in Tab. 2 cannot be checked.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes Stable Diffusion Inpainting's instruction-following deficiency, attributing it to training bias from random masking and to the model's reliance on the image condition. It proposes FreeCond, a training-free modification of the input conditions: low-pass filtering the image latent in early diffusion steps (Eq. 3) and scaling the mask condition as M_fc = α M_c + β (1 − M_c) (Eq. 4). The authors argue that this aligns cross-attention features with the model's learned bias and improves prompt-adherence and mask-fitting. They introduce FCIBench and report improved CLIP and IoU across SDI, ControlNet Inpainting, HD-Painter, PowerPaint, BrushNet, and SDXL, including a headline 60% CLIP gain for SDI and 58% for SDXL on FCIBench.","tokens_in":15567,"tokens_out":4246,"duration_ms":38350,"significance":"If the central claim holds, FreeCond would be a useful plug-in for SDI-based inpainting models: it requires no training, no additional computation, and is compatible with existing baselines. The paper also contributes an interpretability analysis based on a channel influence indicator and proposes a benchmark with complex prompts and varied mask types. However, the current evidence is not fully convincing: the headline improvement is measured on the authors' own benchmark after selecting the four FreeCond hyperparameters on that benchmark, the FCIBench baseline is unusually low, and the proposed mask modulation is not compared against simple mask dilation. These issues are addressable, so the work is promising but needs revision.","major_comments":[{"comment":"The four FreeCond hyperparameters α, β, γ, and t_fc are selected through ablations that report CLIP and IoU on the same three benchmarks, including FCIBench, that produce the headline numbers. No held-out split, no pre-registered configuration, and no error bars are provided. The reported 60% CLIP gain for SDI on FCIBench (Tab. 2) is therefore partly a fitted-prediction result. Please report performance with a configuration fixed before evaluation, or use cross-validation, and give variance estimates across the 600 prompts.","section":"Sec. 5.3, Fig. 10"},{"comment":"On FCIBench, the SDI baseline CLIP score is 11.45 and FreeCond raises it to 18.27; on COCO and BrushBench the same baseline changes by only +0.15 and +0.05 CLIP points. The headline 60% relative gain is thus dominated by the self-built benchmark with an unusually low baseline. Please report absolute gains on standard benchmarks and justify why FCIBench is the appropriate basis for the central claim, or moderate the headline claim accordingly.","section":"Tab. 2"},{"comment":"M_fc = α M_c + β (1 − M_c) with β > 0 is a soft dilation or offset of the mask condition. Since Sec. 3.3 already shows that enlarging the mask improves CLIP and IoU in the hard-instruction regime, a baseline that simply dilates the input mask, without the low-pass image modification, is needed. Tab. 2 does not include such a baseline, so the specific contribution of the FreeCond modulation cannot be isolated from the already-known effect of mask dilation.","section":"Sec. 4.2, Eq. (4); Sec. 3.3"},{"comment":"The claim that low-pass filtering z_fc in early steps largely preserves the background is not well supported by the reported metrics: in Tab. 2, SDI's PSNR drops from 25.95 to 24.79 on COCO, from 27.26 to 26.73 on BrushBench, and from 25.54 to 24.58 on FCIBench when FreeCond is applied. Please quantify background preservation more directly, for example by reporting PSNR and LPIPS on the unmasked region only, and temper the 'minimal impact' claim accordingly.","section":"Sec. 4.1, Eq. (3); Tab. 2"},{"comment":"The claim that the first 80 channels of the cross-attention query are specifically adapted to the mask is based on a single visualization of ΔCI for one example. Since Sec. 4.2's design of M_fc relies on this claim, please provide statistics over multiple prompts, masks, and timesteps, or otherwise weaken the architectural claim to an observation that motivated the heuristic.","section":"Sec. 3.4, Fig. 7"}],"minor_comments":[{"comment":"There is a typo: 'SDXLI' should be 'SDXL'.","section":"Abstract"},{"comment":"There is a typo: 'insturction' should be 'instruction'.","section":"Sec. 4.1"},{"comment":"The typesetting of the normalization term in the CI definition is garbled ('1P j ¯Mj H×W /16X j=1'); please rewrite the equation clearly.","section":"Eq. (2)"},{"comment":"The paper says the full version and code are available in an external repository; for a journal submission, the full experimental settings (per-baseline hyperparameters, exact LPF implementation, and IoU computation details) should be included in the paper or a stable supplement so that the reported numbers are verifiable.","section":"Footnote 1 and Sec. 5.1"},{"comment":"Reference [35] is a duplicate of reference [34]; please remove the duplicate and renumber.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is appealing and the analysis is suggestive, but the current arXiv text is abbreviated and relies on an external repository for critical details. The main risk is that the headline improvement is a selection artifact on the authors' own benchmark. I would encourage the editor to require a held-out validation protocol and a mask-dilation baseline as conditions for acceptance, and to ask for the full version and code release as part of the revised submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FreeCond is a training-free input-condition tweak for SDI-based inpainting that looks like it helps when prompts are complex and unrelated to the image context. That is the useful bit. The analysis of how the mask shifts cross-attention channels (their CI indicator) is a new diagnostic, and the plug-in nature—just modifying z_c and M_c—means it can bolt onto any SDI backbone. The qualitative results in Fig. 1 and 2 are convincing that something real is happening.\n\nThe soft spot is the evidence behind the '60%' headline. That number comes from FCIBench, their own benchmark, and the four hyperparameters (α, β, γ, t_fc) are tuned on the same benchmark via ablations in Fig. 10. No fixed configuration is given for Table 2, and there are no error bars or held-out split. Because the baseline CLIP on FCIBench is 11.45, the relative gain is a 6.8-point absolute change, while COCO and BrushBench gains are under 0.2 points. That suggests the headline is less a property of the method and more a property of the benchmark.\n\nA second issue: the mask modification in Eq. 4 with β>0 is a soft dilation, and Sec. 3.3 already shows dilation helps. There is no mask-dilation-only baseline in Table 2, so you cannot tell if the method is better than simply dilating the mask. The low-pass filtering of the image condition is the genuinely new piece, but its contribution is not isolated.\n\nMinor: they disable the NSFW detector in BrushBench, which changes the evaluation protocol for baselines. The CI analysis is heuristic, based on a single example. Nothing here is fatal, and the paper is honest that excessive parameter tweaking degrades quality.\n\nMy take: with fixed hyperparameters, a dilation-only baseline, and results on a held-out set, this could be a solid contribution. As it stands, the claims outrun the evidence. I'd send it to review, but with a clear demand for those additions.","headline":"A plausible training-free inpainting plug-in whose headline result is inflated by tuning on the same benchmark used for evaluation.","tokens_in":16096,"tokens_out":3188,"would_cite":true,"duration_ms":32664,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free tweak to Stable Diffusion inpainting inputs can lift prompt-following by up to 60%.","keywords":["text-guided inpainting","Stable Diffusion","training-free","cross-attention","classifier-free guidance","input conditioning","FCIBench","instruction following"],"falsifier":"Take a held-out set of complex inpainting prompts not used in the paper's ablations, freeze FreeCond's hyperparameters at the values reported, and measure the CLIP-score gain over the plain SDI baseline; if the gain drops to the ~1% level observed on COCO and BrushBench rather than the claimed ~60%, the 'free lunch' framing fails. A second check is to measure LPIPS or PSNR on the preserved background with and without the low-pass filter to see whether the assumed background preservation holds.","tokens_in":15011,"feed_emoji":"🎨","tokens_out":5065,"duration_ms":45075,"temperature":0.7,"pith_summary":"The paper argues that Stable Diffusion Inpainting (SDI) models fail to follow complex prompts when the surrounding image context is unrelated, because the model was trained with random masking and leans on the image condition to stay harmonious. It locates the mechanism in cross-attention: the mask input shifts query-key features so the model paints inside the mask, and the image condition supplies context that can override the prompt. From this it derives FreeCond, a two-part, training-free modification of the network's input conditions—low-pass filtering the image condition in early diffusion steps and scaling the mask condition—that it claims improves prompt adherence and mask fitting on any SDI-based model with no extra computation. The paper also introduces FCIBench, a 600-pair benchmark with precise, rough, and multiple masks paired with complex prompts unrelated to the image, and reports up to a 60% CLIP-score gain for SDI and 58% for SDXL on it.","feed_headline":"Input tweak lifts SD inpainting CLIP scores by 60%","feed_subtitle":"FreeCond only low-pass filters the image and scales the mask; no retraining or extra compute.","key_machinery":"The load-bearing object is the cross-attention layer in the SDI UNet, where query features $Q$ from the concatenated noise, mask, and image latents meet prompt key and value tokens $K, V$. The paper defines a Channel Influence Indicator $\\mathrm{CI}(Q, M, k, i)$ to show that the mask condition shifts attention channels, particularly the first 80 channels, toward informative prompt tokens. FreeCond's machinery is the pair of input modulations: Eq. (3) low-pass filters the image latent at early diffusion steps to reduce contextual interference, and Eq. (4) scales the mask latent to intensify the cross-attention feature shift. Together they make the modified input conditions carry the work that would otherwise require retraining.","core_discovery":"The paper's central claim is that SDI's instruction-following weakness is a conditioning bias, not a capacity problem, and can be corrected by modulating only the inputs to the denoising UNet. Concretely, replacing the image condition with $z_{fc} = \\mathrm{LPF}(z_c, \\gamma)$ for early timesteps $t \\ge t_{fc}$ removes high-frequency context that would otherwise dominate generation, while replacing the mask condition with $M_{fc} = \\alpha M_c + \\beta(1 - M_c)$ amplifies the cross-attention feature shift that directs prompt influence into the masked region. The resulting noise predictor $\\hat{\\epsilon}_\\theta(z_t, z_{fc}, M_{fc}, t, p)$ is argued to be a more general form of classifier-free guidance. The paper reports consistent gains in CLIP and IoU scores across five SDI-based baselines, with the largest gains on the new hard benchmark, FCIBench, including a 60% CLIP-score increase over the original SDI and a 1% increase over the existing SOTA.","pith_inferences":["I infer that the same input-condition modulation could transfer to other conditional diffusion pipelines, such as depth-to-image or image editing, since the paper's mechanism is about conditioning dominance in cross-attention rather than about inpainting per se; the paper does not test this.","I infer that the headline 60% figure should be read as benchmark-specific: if the four hyperparameters are fixed and applied to a held-out collection of complex prompts, the gain may shrink toward the single-digit percentages seen on COCO, because the paper tuned those values on FCIBench itself.","A natural testable extension is to make $\\alpha$, $\\beta$, $\\gamma$, and $t_{fc}$ adaptive to prompt complexity or mask size, since the paper reports fixed values and notes that over-adjustment degrades image quality.","I infer that the method's real-world niche is interactive editing tools where users give detailed prompts on cluttered backgrounds; for simple object replacement, users should expect no visible change."],"forward_implications":["Any SDI-based inpainting model can be upgraded by wrapping its inputs in FreeCond, with no retraining and no extra compute.","The method improves prompt adherence and mask fitting at the same time, rather than trading one for the other as classifier-free guidance scaling does.","FCIBench gives the community a harder evaluation setting in which complex prompts are unrelated to the image context, exposing weaknesses that COCO and BrushBench hide.","On easy benchmarks such as COCO and BrushBench the gains are small (about 1% or less), so FreeCond's value is concentrated in hard, context-unrelated inpainting requests.","FreeCond also improves SOTA methods like BrushNet and PowerPaint by about 1% CLIP on FCIBench, suggesting the plug-in is complementary to training-based mask-fitting solutions."],"supporting_citations":[{"why":"Supplies the SDI backbone whose training bias and cross-attention behavior FreeCond modifies.","marker":"[26]"},{"why":"Defines classifier-free guidance, the noise-prediction framework FreeCond generalizes and the w scale used in ablations.","marker":"[9]"},{"why":"Provides the observation that low-frequency structure forms early and high-frequency details later, motivating the timestep-dependent low-pass filter.","marker":"[28]"},{"why":"BrushNet, the SOTA inpainting baseline whose metrics and comparisons FreeCond must beat.","marker":"[13]"},{"why":"HD-Painter, an SDI-based method FreeCond is shown to improve.","marker":"[20]"},{"why":"PowerPaint, another SDI-based method FreeCond is shown to improve.","marker":"[42]"},{"why":"CLIP, the instruction-following metric that carries the headline gain.","marker":"[24]"},{"why":"COCO, the source of evaluation pairs and the surrogate analysis for the SDI training mask distribution.","marker":"[16]"},{"why":"SDXL, the model used to show zero-shot improvement of FreeCond.","marker":"[23]"}],"fun_headline_variants":["Free lunch: input-only fix boosts inpainting CLIP 60%","No-compute tweak gives SDI inpainting a 60% CLIP lift","Mask scaling + image filtering: 60% CLIP gain, zero retrain","FreeCond: adjust mask & image to leap 60% in CLIP score"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim rests on the assumption that the four FreeCond hyperparameters, chosen by ablations on the same FCIBench benchmark that produces the reported 60% gain, continue to work well on other inpainting prompts; a related assumption is that low-pass filtering the image condition in early steps leaves the background intact enough for preservation metrics to hold.","fun_headline_variants_meta":{"raw":{"variants":["Free lunch: input-only fix boosts inpainting CLIP 60%","No-compute tweak gives SDI inpainting a 60% CLIP lift","Mask scaling + image filtering: 60% CLIP gain, zero retrain","FreeCond: adjust mask & image to leap 60% in CLIP score"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000299,"raw_usage":{"total_tokens":1740,"prompt_tokens":967,"completion_tokens":773,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":694}},"tokens_in":583,"tokens_out":773,"duration_ms":7396,"temperature":1.0,"reasoning_tokens":694,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:23:29.625681+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of complex inpainting prompts not used in the paper's ablations, freeze FreeCond's hyperparameters at the values reported, and measure the CLIP-score gain over the plain SDI baseline; if the gain drops to the ~1% level observed on COCO and BrushBench rather than the claimed ~60%, the 'free lunch' framing fails. A second check is to measure LPIPS or PSNR on the preserved background with and without the low-pass filter to see whether the assumed background preservation holds.","supporting_citations":[{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the SDI backbone whose training bias and cross-attention behavior FreeCond modifies."},{"cited_title":"Freeu: Free lunch in diffusion u-net","cited_arxiv_id":null,"evidence_quote":"Provides the observation that low-frequency structure forms early and high-frequency details later, motivating the timestep-dependent low-pass filter."},{"cited_title":"Microsoft coco: Common objects in context","cited_arxiv_id":null,"evidence_quote":"COCO, the source of evaluation pairs and the surrogate analysis for the SDI training mask distribution."}],"review_version":1}