{"id":"0f9f45a8-33f1-4bc7-98d2-21e2b5f7987e","arxiv_id":"2505.13740","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CompLift accepts or rejects generated samples by computing lift scores from conditional and unconditional denoising errors, improving compositional alignment without retraining.","lead":"CompLift is a training-free rejection sampler that uses the diffusion model's own denoising error to check whether an image matches each part of a composition. It improves compositional generation in CLEVR and text-to-image tasks, at a small cost in inference.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains may rest on hyperparameters (t=928, τ=250) chosen on the evaluation sets; a validation-split rerun is needed before the central claim of a generalizable lift-score criterion is established.","rationale":"The reader's weakest assumption identified both the unexamined ELBO approximation and the task-specific constants (α=0.9, t=928, τ=250). I agree with the CONDITIONAL verdict: the paper proposes a plausible, training-free resampling idea with consistent gains across three settings, and the appendix adds useful ablations (CAS variant, TIFA, minCLIP, human-labeled ROC/PR). These are real evidence that the score carries signal. However, the most load-bearing threat to the central claim is that the criterion was not evaluated under honest hyperparameter selection. For CLEVR and T2I, the key constants are chosen by looking at the evaluation data, so the reported improvements may not transfer. This does not warrant rejection: the validation-split test I propose is inexpensive and would either confirm the finding or bound it. I therefore keep the reader's verdict unchanged.","tokens_in":25054,"tokens_out":9156,"duration_ms":91710,"concrete_test":"Re-run the CLEVR and text-to-image experiments with a strict split: select t and τ using only a validation portion (e.g., 50% of position combinations for CLEVR; 50% of prompts for T2I), then evaluate the chosen configuration on the held-out test portion. Report accuracy/CLIP/ImageReward and their standard errors for the baseline, cached CompLift, and vanilla CompLift. If the gains over Composable Diffusion are not statistically significant on the held-out split, the central claim that CompLift generalizes is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the training-free lift-score criterion improves compositional alignment. The decisive weakness is that the criterion's free parameters are selected on the evaluation tasks themselves. In Section 6.2, t=928 is chosen by inspecting the accuracy curve in Figure 3 on the CLEVR test combinations; in Section 5.1, τ=250 is set as the median activated-pixel count over the test images, and Section 5.1 says the median was chosen after testing the 25th and 75th percentiles. The 2D task also uses an ad hoc unconditional definition ϵθ(xt,∅)=0.9·ϵθ(xt,c) (Section 6.1). Because these constants are fitted to the same benchmarks on which improvements are reported, Tables 2-3 and Figure 9 do not yet demonstrate a generalizable criterion. This is compounded by the paper's own Appendix A: 'we will lose the theoretical guarantee of CompLift using ELBO estimation.' No sensitivity analysis or validation-based selection is provided, so the reported gains could reflect test-set tuning rather than a robust property of lift scores.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CompLift, a training-free rejection/resampling criterion for compositional generation with diffusion models. The lift score log p(x|c)/p(x) is approximated by the difference between two ELBO-based denoising errors, with and without condition c (Eq. 3). The Compose function combines per-condition lift scores according to product, mixture, or negation algebra (Table 1). An optimized cached variant reuses intermediate predictions from Composable Diffusion and classifier-free guidance (Algorithm 2) to reduce the computational overhead. Experiments are reported on 2D synthetic distributions, the CLEVR Position dataset, and the Attend-and-Excite text-to-image benchmark, with accuracy, Chamfer distance, FID, CLIP, and ImageReward metrics. The central claim is that lift scores, computed only from the original diffusion model, significantly improve compositional generation without additional training or external modules.","tokens_in":25295,"tokens_out":4361,"duration_ms":38417,"significance":"If the criterion is robust, CompLift would be a useful, model-agnostic, training-free addition to compositional generation, and the cached variant shows a concrete path to near-zero extra inference cost under the Composable Diffusion framework. The paper provides a broad experimental sweep, including new TIFA and minCLIP results in the rebuttal appendix, a controlled comparison against a CAS-like whole-prompt scorer (Appendix G.3), and an ablation on the Attend-and-Excite generator (Table 8). The strengths are the simplicity of the criterion, the caching optimization, and the breadth of evaluation across three very different settings. However, the empirical significance is currently limited because key hyperparameters are selected on the evaluation tasks themselves, and because the theoretical link between the ELBO difference and the lift score is not established quantitatively, as the paper itself concedes in Appendix A.","major_comments":[{"comment":"The timestep t=928 is chosen by inspecting the accuracy curve in Figure 3, which is computed on the CLEVR Position evaluation combinations. The same combinations are then used for the reported results in Figure 9 and Table 5. This makes the reported accuracy gains potentially a consequence of test-set selection rather than a general property of the lift-score criterion. Please rerun the evaluation with the timestep selected on a validation split of position combinations (or with a principled timestep-sampling distribution), and report both the validation-selected and test results.","section":"Section 6.2, Figure 3"},{"comment":"The activated-pixel threshold tau=250 is set as the median activated-pixel count 'among all images,' and the paper states that tests at the 25th and 75th percentiles showed the median works best. Because these images are the test images of the Attend-and-Excite benchmark, the threshold is fitted to the evaluation set. Table 3 therefore does not by itself establish that a fixed tau generalizes. Please define a validation-based selection procedure (e.g., a separate set of prompts and images) and report the sensitivity of Table 3 to tau.","section":"Section 5.1, Eq. (4)"},{"comment":"The 2D experiments define the unconditional prediction as epsilon_theta(xt, empty)=0.9 * epsilon_theta(xt, c), a task-specific choice that changes the lift criterion itself. The paper calls this 'an effective strategy' but provides no derivation or sensitivity analysis. Since the large gains in Table 2 (e.g., product accuracy rising from 56.5 to 99.5) depend on this alpha, the 2D results do not currently support the claimed generalizability of the lift-score criterion. Please provide a proper unconditional estimate or, failing that, a sensitivity study over alpha.","section":"Section 6.1"},{"comment":"The approximation in Eq. (3) replaces log p(x|c)/p(x) with a difference of ELBO terms. Because the ELBO is a lower bound, the sign of Eq. (3) is not guaranteed to match the sign of the true lift score. The paper itself notes in Appendix A that 'we will lose the theoretical guarantee of CompLift using ELBO estimation,' but no estimate of the approximation error or of false-accept/false-reject rates is provided. Since the acceptance threshold is exactly zero, a bias or overlap analysis (e.g., extending the Figure 17 score histograms to the text-to-image setting) is needed to connect the theoretical framing to the empirical procedure.","section":"Eq. (3) and Appendix A"},{"comment":"The variance-reduction substitution of the true noise epsilon by epsilon_theta(zt, c_compose) in Eq. (5) is heuristic. Figure 6 shows a qualitative reduction in noise, but there is no quantitative variance comparison, no bias analysis, and no comparison with alternative variance-reduction baselines. Because Eq. (5) is the core of the text-to-image results, the paper should report the variance of Eq. (3) versus Eq. (5), and show that acceptance decisions are stable across the number of trials T.","section":"Section 5.2, Eq. (5)"}],"minor_comments":[{"comment":"The word 'sysmatic' should be 'systematic'.","section":"Section 7"},{"comment":"The code URL 'rainorangelemon.github.io/complift' lacks a protocol and appears as plain text; please format it as a proper hyperlink.","section":"Abstract/Code Link"},{"comment":"The x-axis label 'Timestep Used (1 Trial)' is ambiguous; please clarify that the ELBO is estimated with a single sampled timestep.","section":"Figure 3"},{"comment":"The loop variable is ck but the appended score is written as liftj(x0|ci), which is confusing; please unify the notation.","section":"Algorithm 1, line 7"},{"comment":"Reusing the symbol lift for a pixel-count criterion with units of pixels is confusing after Eq. (3) defines a scalar score; please introduce a distinct symbol such as N_act(x, ci).","section":"Section 5.1, Eq. (4)"},{"comment":"The appendix titled 'Q&A during Rebuttal' contains reviewer-facing language ('We appreciate the feedback from anonymous reviewers'); this text is not appropriate for a published manuscript and should be removed or rewritten.","section":"Appendix H"}],"recommendation":"major_revision","confidential_remarks":"The paper's Appendix H appears to be leftover from the review process and should be removed from any archival version. More substantively, the admission in Appendix A that the theoretical guarantee is lost under ELBO estimation is central to the paper's validity and should be addressed in the main text, not relegated to an appendix. The key fix that would make the central claim convincing is a validation-based protocol for choosing t, tau, and alpha, reported alongside the current test-set-selected numbers. If the authors can supply such a rerun, the paper could become acceptable; without it, the reported gains remain vulnerable to the objection that the criterion is tuned on the benchmarks."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a useful paper, not a paradigm shift. The core idea—estimate per-condition lift via denoising error differences and accept or reject samples accordingly—works across three settings, and the paper is honest about its limitations. The novelty is real: applying lift scores compositionally, the activated-pixel count for object existence, and the caching variant that makes overhead near zero under Composable Diffusion. The experiments are thorough: 2D synthetic with product/mixture/negation, CLEVR position, and text-to-image on SD 1.4/2.1/XL, plus extra ablations in the appendix (TIFA, minCLIP, gain over a perfect selector, CAS-variant comparison). That is a solid empirical package.\n\nThe soft spot is real, though. The decisive weakness is the selection of free hyperparameters on the evaluation tasks: t=928 is picked from the accuracy curve in Figure 3 on CLEVR test combinations; tau=250 is set as the median activated-pixel count on the test images; alpha=0.9 in the 2D task is ad hoc. Without a validation split or sensitivity analysis, the reported gains could reflect test-set tuning. The appendix's admission that the theoretical guarantee is lost for ELBO estimation (Appendix A) does not help. The ELBO approximation in Eq. (3) is also unexamined as an estimator of the log-likelihood ratio, and Eq. (5)'s use of the composed prediction as reference introduces a bias toward samples the model already favors—a legitimate circularity concern, though not fatal.\n\nThat said, the central empirical claim probably holds. The improvements are consistent across backbones and metrics, and the relative gain over a perfect best-of-n selector (Table 10) makes the CLIP numbers look less trivial. The paper is clearly written, the related work is properly cited, and the authors flag their own limitations rather than hide them.\n\nWho is this for? Researchers working on training-free guidance or rejection sampling for diffusion models, and practitioners who want a cheap filter for missing objects. It deserves a serious referee: the tuning issue is fixable in revision, and the tool is likely to be used. I would send it to peer review and ask for validation-based selection of alpha, tau, and t, plus an error analysis of the lift estimator.","headline":"CompLift is a genuinely useful training-free rejection filter for compositional diffusion with broad experiments, but its headline gains rest on hyperparameters tuned on the evaluation sets, so the generalizable-criterion claim needs a validation-split rerun.","tokens_in":25787,"tokens_out":2503,"would_cite":true,"duration_ms":22500,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A diffusion model can grade its own output: reject images that its own denoiser does not reconstruct better when told the prompt, and compositional prompt-following improves, with no retraining and no external verifier.","keywords":["compositional generation","diffusion models","lift scores","rejection sampling","training-free methods","text-to-image synthesis","ELBO approximation","resampling criterion"],"falsifier":"Two checks would settle the claim. First, measure the variance of the Eq. (3) estimator across trials on a fixed image: if the standard error of the lift score is comparable to its mean, the accept/reject boundary is dominated by estimation noise rather than condition alignment. Second, freeze all constants before seeing the test distribution — pick $\\tau$ on held-out validation prompts from object classes not in the benchmark and sample the timestep by the training distribution rather than the single best observed value — and re-run the text-to-image experiments; if the CLIP and minCLIP gains largely disappear, the reported improvement is attributable to the tuned constants rather than to the lift-score principle itself.","tokens_in":24834,"feed_emoji":"🖼️","tokens_out":19546,"duration_ms":145567,"temperature":0.7,"pith_summary":"This paper tries to establish that a diffusion model can check its own output against a prompt: before trusting an image, ask the model whether knowing the condition actually lowers its reconstruction error. The lift score — the expected gap between unconditional and conditional denoising error — is positive exactly when the condition is doing work, and per-condition scores composed by AND/OR/NOT logic decide whether the whole prompt is satisfied. The authors report that this training-free acceptance rule improves condition alignment over the Composable Diffusion baseline across 2D synthetic compositions, CLEVR position tasks, and text-to-image generation with three Stable Diffusion backbones, at near-zero extra cost once denoising predictions are cached. If the claim holds, any conditional diffusion model gains a built-in test-time self-critic, without an external scorer, a verifier, or additional training.","feed_headline":"Reject images the model can't verify against prompt, lifting accuracy","feed_subtitle":"Denoising error with and without the prompt flags missing objects, boosting composition accuracy without retraining.","key_machinery":"The load-bearing object is the lift score, $\\operatorname{lift}(x|c) \\approx \\mathbb{E}_{t,\\epsilon}\\big[\\|\\epsilon - \\epsilon_\\theta(x_t,\\emptyset)\\|^2 - \\|\\epsilon - \\epsilon_\\theta(x_t,c)\\|^2\\big]$ (Eq. 3): the expected gap between unconditional and conditional denoising error, which is positive precisely when the condition improves reconstruction of the sample. Around this quantity the paper assembles three further pieces. The Compose function converts any prompt algebra into conjunctive normal form and combines the signs of per-condition lift scores into a single boolean accept/reject verdict, covering product, mixture, and negation. The image-space estimator (Eq. 5) replaces the random noise $\\epsilon$ with the composed-prompt prediction $\\epsilon_\\theta(z_t, c_{\\mathrm{compose}})$, which is assumed to share the model's bias and therefore cancel it, sharply reducing per-pixel variance. The activated-pixel count (Eq. 4) then detects object presence as the number of latent pixels with positive lift minus a threshold $\\tau$. Finally, the cached variant (Algorithm 2) reuses the denoising predictions made during ordinary generation, so under classifier-free guidance with Composable Diffusion the number of extra forward passes is zero.","core_discovery":"The central claim is that the lift score, $\\log p(x|c) - \\log p(x)$, approximated by the difference between unconditional and conditional denoising losses (Eq. 3), is a dependable per-condition test of whether a generated sample matches a condition. A product prompt such as 'a black car and a white clock' is accepted only when every object's lift is positive; mixtures and negations are composed from the same per-condition scores by logical rules. The paper reports large accuracy gains from applying this filter: from 56.5% to 99.9% on 2D product compositions, from 78.7% to 90.3% at five simultaneous constraints on CLEVR, and consistent CLIP and ImageReward improvement in text-to-image across SD 1.4, 2.1, and XL, with an older backbone plus the filter approaching the next generation's vanilla performance. For text-to-image, the estimator is stabilized by comparing the composed-prompt denoising prediction against the unconditional and per-condition predictions (Eq. 5), and object presence is read off as the count of latent pixels with positive lift above a threshold (Eq. 4). The paper also states the scope of its claims explicitly: the criterion inherits the quality of the underlying generator, and OR/NOT algebra is tested on 2D data but not on text-to-image.","pith_inferences":["The per-condition scores carry more than a verdict: the non-positive lift identifies which object is missing, so the filter could drive targeted repair (resampling or editing only the failing object) instead of discarding the whole image; the paper does not pursue this.","Because the constants $\\alpha = 0.9$, $t = 928$, and $\\tau = 250$ are tuned on the evaluation tasks themselves, the sharpest test of the principle would fix those constants before seeing any test data and measure whether the gains persist.","The recipe is domain-agnostic: any diffusion model with conditionable denoising — audio, video, molecules — inherits the same accept/reject check without retraining, extending the paper's stated future direction toward video and music generation.","The reported Pearson correlations between activated-pixel count and CLIP score (0.68–0.86) suggest the lift criterion could serve as a reward signal for self-training or reinforcement-learning fine-tuning, a use the paper does not explore."],"forward_implications":["Any diffusion model that can be run both with and without a condition inherits a free self-verifier: when classifier-free guidance and composable sampling are already in use, the cached version adds zero extra forward passes.","On the CLEVR position task the accuracy gap widens as the prompt grows harder (78.7% to 90.3% at five constraints), so the criterion degrades much more gracefully than the baseline as compositional load increases.","The filter transfers across backbones: SD 1.4 plus CompLift approaches vanilla SD 2.1, and SD 2.1 plus CompLift approaches vanilla SDXL on parts of the Attend-and-Excite benchmark, meaning part of a newer model's advantage can be recovered at test time.","Rejection trims diversity: CLEVR FID worsens modestly as accuracy improves, so the method trades sample variety for prompt fidelity."],"supporting_citations":[{"why":"Supplies the ELBO-based likelihood approximation and the shared-noise estimator that CompLift's lift-score formula (Eq. 3) is built on.","marker":"(Li et al., 2023)"},{"why":"Provides the Composable Diffusion baseline, the CFG-style score combination, and the CLEVR pretrained model that CompLift is evaluated against.","marker":"(Liu et al., 2022)"},{"why":"Origin of the lift concept in data mining that the paper adapts to log-likelihood ratios for the acceptance criterion.","marker":"(Brin et al., 1997)"},{"why":"The DDPM denoising objective whose ELBO gap is the proxy for $\\log p(x|c) - \\log p(x)$.","marker":"(Ho et al., 2020)"},{"why":"Importance-sampled training that dictates matched timestep sampling for the CLEVR model, leading to the single-timestep choice $t = 928$.","marker":"(Nichol & Dhariwal, 2021)"},{"why":"Defines the 2D synthetic distribution setup and supplies the energy-based MCMC baselines (ULA, MALA, HMC, U-HMC) that CompLift is compared with.","marker":"(Du et al., 2023)"},{"why":"Provides the Attend-and-Excite benchmark prompts and the CLIP/ImageReward evaluation protocol for text-to-image, plus the generator used in one ablation.","marker":"(Chefer et al., 2023)"},{"why":"Classifier-free guidance, the source of the unconditional predictions and the guidance scale used in every experiment and in the cached variant.","marker":"(Ho & Salimans, 2022)"},{"why":"The whole-prompt Condition Alignment Score used in the controlled ablation that isolates the advantage of per-condition composition.","marker":"(Hong et al., 2024)"},{"why":"SAM2 segmentation serves as the generalizable position verifier that labels whether each CLEVR condition is satisfied.","marker":"(Ravi et al., 2024)"}],"fun_headline_variants":["Lift scores reject samples failing condition checks in diffusion models","No-training filter uses lift scores to improve compositional diffusion outputs","Score each condition: lift scores fix composition in diffusion generation","Diffusion composition improved by lift-score verification without retraining","Lift-score gate checks every object then accepts the image in diffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the difference in denoising error with and without the condition, averaged over a small number of Monte Carlo trials, is an accurate and unbiased measure of whether the condition truly matches the image, and it leans on several constants tuned on the evaluation tasks themselves ($\\alpha = 0.9$ in 2D, $t = 928$ on CLEVR, $\\tau = 250$ for text-to-image).","fun_headline_variants_meta":{"raw":{"variants":["Lift scores reject samples failing condition checks in diffusion models","No-training filter uses lift scores to improve compositional diffusion outputs","Score each condition: lift scores fix composition in diffusion generation","Diffusion composition improved by lift-score verification without retraining","Lift-score gate checks every object then accepts the image in diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1370,"prompt_tokens":937,"completion_tokens":433,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":350}},"tokens_in":553,"tokens_out":433,"duration_ms":4593,"temperature":1.0,"reasoning_tokens":350,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:10:20.133564+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Two checks would settle the claim. First, measure the variance of the Eq. (3) estimator across trials on a fixed image: if the standard error of the lift score is comparable to its mean, the accept/reject boundary is dominated by estimation noise rather than condition alignment. Second, freeze all constants before seeing the test distribution — pick $\\tau$ on held-out validation prompts from object classes not in the benchmark and sample the timestep by the training distribution rather than the single best observed value — and re-run the text-to-image experiments; if the CLIP and minCLIP gains largely disappear, the reported improvement is attributable to the tuned constants rather than to the lift-score principle itself.","supporting_citations":[{"cited_title":"D., and Tsur, S","cited_arxiv_id":null,"evidence_quote":"Origin of the lift concept in data mining that the paper adapts to log-likelihood ratios for the acceptance criterion."}],"review_version":1}