{"id":"7e114179-5881-492d-8402-9975175f02a9","arxiv_id":"2502.04646","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A score-based generative model can be reused without retraining to sample from arbitrary weighted versions of its base distribution by adding an approximated guidance term during reverse diffusion.","lead":"This paper proposes a training-free method to draw samples from a target distribution proportional to the product of a known base distribution and a given weight function, using a pretrained score-based generative model for the base distribution. The method approximates the diffusion guidance term with a Tweedie mean estimate and a finite-difference correction, then runs a reverse diffusion process, with applications to rare-event sampling and image control.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (14) and Eq. (16) misapply the chain rule: the Tweedie estimate in Eq. (13) has Jacobian [I+(1−αbar)H]/√αbar, so the guidance should be divided by √αbar, not multiplied by it.","rationale":"The reader's weakest assumption concerned the accuracy of the first-order Taylor expansion in Eq. (11). That is a real and unresolved issue: the conditional variance of X0 given Xt is large for most of the reversed trajectory, and no accumulation bound is provided. However, the chain-rule error identified here is more fundamental because it is prior to any approximation: the proposed SDE is not the (even approximate) score of q_t that the derivation claims. The factor error can be checked by elementary differentiation, so it is not a matter of interpretation or missing baselines. If the equations are corrected, the experiments and Theorem 1 would need to be re-run; if the released code already uses the corrected factor, the paper still needs to state and analyze that SDE. Either way, the current manuscript's central construction is internally inconsistent. I therefore disagree with the reader's identification of the weakest assumption and recommend REJECT for the submitted version; a corrected derivation with rerun experiments could support a future submission.","tokens_in":13446,"tokens_out":12945,"duration_ms":116595,"concrete_test":"Independently re-derive Eq. (14) by differentiating Eq. (13) with respect to x and applying the chain rule; the Jacobian contains 1/√αbar, so the RHS of Eq. (14) should be divided by √αbar. For a numerical confirmation, set p_0 = N(0,I_d), l(x) = exp(a·x), so that q_0 = N(a,I_d) and ∇log q_t is known in closed form; evaluate the paper's Eq. (16) and the corrected chain-rule expression at αbar = 0.5 and compare. Also inspect the released code to determine whether it implements Eq. (16) as printed or the corrected factor.","verdict_should_be":"REJECT","load_bearing_attack":"The most load-bearing problem is an algebraic error in the derivation of the proposed SDE, not the Taylor truncation in Eq. (11). Eq. (13) defines x0hat = (x + (1−αbar)∇log p_t(x))/√αbar. Differentiating this expression with respect to x gives the Jacobian J = [I + (1−αbar)H_{p_t}(x)]/√αbar. The chain rule therefore gives ∇_x log l(x0hat) = J^T g = [I + (1−αbar)H_{p_t}(x)] g / √αbar, where g = ∇_{x0hat} log l(x0hat). Eq. (14) instead has [I + (1−αbar)H_{p_t}(x)] √αbar g, i.e., the factor √αbar is inverted. The same error appears in Eq. (16), whose first-order guidance term is √αbar g rather than g/√αbar; the finite-difference Hessian term in Eq. (16) is also not what Eq. (14) plus Eq. (15) would produce, differing by another factor of √αbar. Consequently, the reverse SDE actually analyzed and run in the experiments is not the score of the importance-sampling distribution, even when the first-order Taylor approximation in Eq. (11) is exact. This invalidates the central derivation and makes Theorem 1, which bounds the gap to this incorrectly defined score, not a guarantee about the intended sampling task.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free importance sampling scheme for score-based generative models. Given a pretrained score function for a base density p and a differentiable positive weight function l, the authors derive an approximate backward SDE that they claim samples from q(x) ∝ l(x)p(x). The method replaces the intractable time-dependent score of q by the score of p plus a Tweedie-based guidance term, with a Hessian-vector product approximated by a finite difference. Empirical evaluations are reported on synthetic targets, on a neural-compression distortion task for CSI data, on CelebA with a classifier weight, and on StableCascade with a frequency-based weight. The high-level idea is attractive because a single pretrained SGM could in principle serve many importance weights without retraining.","tokens_in":13831,"tokens_out":8003,"duration_ms":85556,"significance":"If the derivation were correct, this would be a useful and scalable contribution: it would extend the applicability of pretrained SGMs to arbitrary differentiable importance weights, including neural-network weights, with no per-weight training. The paper includes code, a reasonable breadth of experiments, and visible effort to compare against density-estimation baselines on synthetic tasks. The main theoretical claim, Theorem 1, is intended to bound the score approximation error in terms of the conditional variance of the initial state, which is a sensible quantity. However, the central derivation contains an algebraic error that changes the guidance term by a factor of sqrt(alpha_bar), so the SDE actually implemented in the experiments is not the score approximation derived in the paper. Other promised components—the uncertainty-aware scheduler and the Stable Diffusion XL speedup results in the abstract—are absent from the body. The significance of the contribution depends on correcting these issues; the underlying idea remains promising.","major_comments":[{"comment":"There is a chain-rule error in the guidance term. Let s = ∇_x log p_t(x), g = ∇_{x̄0} log l(x̄0), and H be the Hessian of log p_t at x. Differentiating x̄0 = (x + (1−ᾱ)s)/√ᾱ gives ∂x̄0/∂x = [I + (1−ᾱ)H]/√ᾱ, so the correct chain rule is ∇_x log l(x̄0) = [I + (1−ᾱ)H]g/√ᾱ. Equation (14) instead has √ᾱ in the numerator. This is not a localized typo: the first-order term in the SDE (16) is √ᾱ g rather than g/√ᾱ, and the Hessian term in (16) has coefficient (1−ᾱ)/√ᾱ, which is also inconsistent with Eq. (14) as printed. Consequently, the backward SDE analyzed and simulated in Section 4 is not the score approximation derived from Eqs. (12)–(15), and Theorem 1, even if proved, would not certify that the implemented algorithm samples q(x) ∝ l(x)p(x). The algebra must be corrected and the experiments rerun with the corrected guidance.","section":"§3.2, Eq. (14) and Eq. (16)"},{"comment":"The derivation of Eq. (12) uses a first-order Taylor expansion of l(X′_0) around the conditional mean, while Theorem 1 assumes that l is approximately equal to its second-order Taylor expansion at the same point. These are different approximations, so the theorem does not bound the score approximation actually used in the method. In addition, the proof of Theorem 1 is not included in the manuscript, although the text states that the gap is established. The bound is also pointwise in x and t, with an error controlled by the conditional variance of X′_0 given X′_t; no argument shows that this pointwise score gap remains small along the reverse-time SDE trajectory or that the accumulated discretization error is controlled. This leaves the central theoretical guarantee unsupported.","section":"§3.2–§3.3, Eq. (11) and Theorem 1"},{"comment":"The abstract and introduction identify an \"uncertainty-aware scheduler that dynamically adjusts the guidance strength based on a temporal analysis of approximation error\" as a key component of the method, but no scheduler is defined, derived, or evaluated anywhere in Sections 3–4. The proposed SDE is presented as fixed, and the experiments do not describe any scheduler. Either the scheduler should be specified and studied, or the claim should be removed from the abstract and contributions.","section":"Abstract and Sections 3–4"},{"comment":"The abstract claims that the framework is validated on Stable Diffusion XL with 1.2× to 4.7× speedups while matching or outperforming state-of-the-art baselines. The body does not contain a Stable Diffusion XL experiment: Section 4.3, Exp-(b) uses StableCascade, and Table 2 reports importance-weight values rather than speedups or comparisons with baselines. The abstract's empirical claims need to be reconciled with the actual experimental content.","section":"Abstract vs. Section 4.3"}],"minor_comments":[{"comment":"There is a typo: \"convolutional nueral architecture\" should read \"convolutional neural architecture.\"","section":"§4.2"},{"comment":"The text refers to \"Table 4.1\" but the table caption reads \"Table 1\"; the numbering should be made consistent.","section":"§4.1"},{"comment":"The experimental section does not report the number of discretization steps, the schedule of β(t), or the value of ϵ used in the finite-difference approximation, which are needed to reproduce the results without relying solely on the source code.","section":"§4"},{"comment":"The finite-difference identity should be stated with the sign convention made explicit: ∇ log p_t(x + ϵg) ≈ ∇ log p_t(x) + ϵ H g, which is the form used in the subsequent derivation.","section":"§3.2, Eq. (15)"}],"recommendation":"major_revision","confidential_remarks":"The chain-rule error in Eq. (14)/(16) is decisive as submitted: the implemented SDE is not sampling the stated importance distribution. I recommend major revision rather than rejection because the error is localized in principle and the core idea is sensible. The authors should correct the factor, rerun the experiments, supply the missing proof of Theorem 1, and align the abstract with the body. If the corrected experiments do not reproduce the reported behavior, the manuscript should be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the core derivation in Eqs. (14)–(16) has an algebraic error. The Tweedie estimate x̂0 = (x + (1−ᾱ)∇log p_t)/√ᾱ has Jacobian J = (I + (1−ᾱ)H_p)/√ᾱ. The gradient of log l(x̂0) w.r.t. x should therefore be Jᵀg = (I + ...)g / √ᾱ. The paper writes it with √ᾱ in the numerator. That is inverted. It propagates into the SDE in Eq. (16), so the reverse process being run is not the score of the importance distribution, even when the first-order Taylor approximation is exact. I checked the stress-test note and it holds.\n\nWhat is genuinely new: the problem is well chosen. Using a single pretrained SGM for many weight functions l without retraining is useful, and the proposed pipeline—Tweedie plus guidance plus a finite-difference Hessian correction—is a real design. The synthetic and large-scale examples (Stable Cascade frequency control, CSI failure-mode analysis) demonstrate that the method works in practice. The paper is clearly written and honest about the Taylor approximation being a heuristic.\n\nWhere it falls apart: the chain-rule factor error is load-bearing, not cosmetic. Theorem 1 bounds a gap to the wrongly defined score, so the stated guarantee does not apply to the intended q. The Taylor expansion of l around the conditional mean is also heuristic; its error is controlled by the conditional variance, which is large over most of the reverse diffusion, and the paper does not bound the accumulated error over the SDE. The experiments compare against trained normalizing flows but omit the most relevant baselines—Diffusion Posterior Sampling and classifier guidance—which use essentially the same guidance mechanism. The proof of Theorem 1 is missing, and the ϵ step size is left unspecified.\n\nBottom line: the idea is worth pursuing, and the experiments suggest that, with the factor fixed, the method would reduce to a variant of DPS with a finite-difference Hessian. That is incremental but useful. As submitted, the central derivation is invalid. It deserves a serious referee because the problem is important and the fix is straightforward, but it should not be accepted until the SDE is corrected and the missing baselines are added.\n\nRecommendation: engage in review rather than desk-reject, but with a strong expectation of major revision. For your reading group, it is a maybe—good as a case study in how an algebraic slip can invalidate a derivation.","headline":"A promising training-free importance sampling method, but the central chain-rule derivation in Eqs. (14)–(16) inverts a √α factor, so the proposed SDE is not actually the score of the target distribution.","tokens_in":14266,"tokens_out":3534,"would_cite":false,"duration_ms":35575,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One pretrained diffusion model samples any reweighted distribution.","keywords":["importance sampling","score-based generative models","diffusion models","training-free guidance","Tweedie's formula","backward SDE","reweighted sampling"],"falsifier":"For a closed-form base distribution and a strongly nonlinear weight function $l$, compute the true importance-sampling distribution $q$ by numerical integration, run the proposed backward SDE, and measure the Jensen–Shannon divergence between generated samples and $q$; if the divergence grows with the curvature of $l$ as predicted by the conditional-variance term, the approximation's practical limit is identified. More directly, track the conditional variance $\\mathbb{E}[\\|X'_0 - \\bar{x}'_{0|x,t}\\|^2 \\mid X'_t = x]$ along the reverse trajectory; if it stays large for a substantial fraction of the trajectory, the bound in Theorem 1 predicts a large score gap at those times, revealing where the method fails.","tokens_in":13290,"feed_emoji":"🎯","tokens_out":4400,"duration_ms":35973,"temperature":0.7,"pith_summary":"This paper claims that importance sampling—drawing from a distribution proportional to a base distribution times a weight function—can be done with an already-trained score-based generative model, without any additional training. The key step is an approximation of the target distribution's time-dependent score: it is the base score plus a guidance term built from Tweedie's formula and the gradient of the weight function, with a Hessian correction computed cheaply by finite differences. The authors derive a backward SDE whose solutions approximately follow the reweighted distribution, and they show empirically on synthetic data, CSI compression, CelebA, and Stable Diffusion XL that the method matches or beats baselines that require per-weight training while being faster. The paper is significant because it makes task-adaptive sampling from a single pretrained model practical for any differentiable importance weight.","feed_headline":"One pretrained diffusion model samples any reweighted distribution","feed_subtitle":"Tweedie-based guidance reuses one pretrained score to sample from l(x)p(x) for any differentiable l.","key_machinery":"Tweedie's formula for the conditional mean $\\bar{x}'_{0|x,t} = \\frac{1}{\\sqrt{\\bar{\\alpha}(t)}}\\left(x + (1 - \\bar{\\alpha}(t))\\nabla_x \\log p_t(x)\\right)$; a first-order Taylor expansion of $l$ around this mean; and a finite-difference approximation of the Hessian-vector product using score evaluations at perturbed points. These combine to give the approximated score $\\nabla_x \\log \\tilde{q}_t(x)$ and the backward SDE (16), which is the object that actually generates samples.","core_discovery":"The central claim is that the score of the importance-sampling distribution $q_t(x)$ can be approximated as $\\nabla_x \\log p_t(x) + \\nabla_x \\log l(\\bar{x}'_{0|x,t})$, where $\\bar{x}'_{0|x,t}$ is the Tweedie conditional mean of the initial sample given the noisy state. To make this computable without Hessians, the paper replaces the Hessian-vector product in the chain rule by a finite-difference score evaluation, yielding the closed-form guidance SDE in Eq. (16). Theorem 1 bounds the gap between true and approximated scores by a term proportional to the conditional variance of $X'_0$ given $X'_t$, which vanishes as $t \\to 0$, and by a term involving the Hessian of $\\log l$ and the Lipschitz constant of $H_{p_t}$. The paper argues, and demonstrates empirically, that this approximation is accurate enough for sampling, and that because only the base score and $l(x)$ are needed, the method is training-free and scales to large foundation models.","pith_inferences":["The paper bounds the score gap at each timestep but does not integrate this bound over the reverse trajectory; a plausible extension is that the accumulated error stays controlled when the conditional variance term decays quickly, but this is not proven.","The Hessian-free guidance term resembles classifier guidance; a testable extension is to compare both on identical base models to see when the explicit guidance term is necessary.","For highly nonlinear or non-smooth $l$, the first-order Taylor expansion could break, and a stress test with adversarial $l$ would reveal the practical limits of the approximation."],"forward_implications":["Any differentiable importance weight $l(x)$ can be used for sampling with a single pretrained model, with no per-weight training.","The same base SGM can serve multiple weighted-sampling tasks simultaneously, since the guidance is computed on the fly from $l$.","In large-scale text-to-image models, the method adds a new control dimension beyond prompts, such as generating images with particular frequency content.","In model analysis, it can efficiently find rare high-distortion inputs, improving inverse model analysis and failure-mode identification.","The overhead over standard sampling is one backpropagation through $l$ and one extra score evaluation per step, which is far cheaper than training a new model."],"supporting_citations":[{"why":"Supplies the DDPM base score model used throughout the experiments.","marker":"[1]"},{"why":"Establishes the forward/backward SDE framework and score-matching that the method builds on.","marker":"[3]"},{"why":"Provides the reverse-time diffusion SDE that the proposed sampling process solves.","marker":"[21]"},{"why":"Gives Tweedie's formula for the conditional mean used in Eq. (13).","marker":"[28]"},{"why":"Uses Tweedie's approach for posterior sampling, a direct predecessor of the guidance idea.","marker":"[19]"},{"why":"The StableCascade foundation model used for large-scale text-to-image experiments.","marker":"[31]"}],"fun_headline_variants":["Score-based sampling for any reweighted distribution","Training-free weighted sampling with pretrained scores","One score model samples all reweighted targets","Efficient weighted sampling without Hessians"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The first-order Taylor expansion of the weight function around the Tweedie conditional mean is accurate enough that the approximation error stays small throughout the reverse trajectory.","fun_headline_variants_meta":{"raw":{"variants":["Score-based sampling for any reweighted distribution","Training-free weighted sampling with pretrained scores","One score model samples all reweighted targets","Efficient weighted sampling without Hessians"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00014,"raw_usage":{"total_tokens":1184,"prompt_tokens":995,"completion_tokens":189,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":145}},"tokens_in":611,"tokens_out":189,"duration_ms":2141,"temperature":1.0,"reasoning_tokens":145,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T21:58:32.735228+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a closed-form base distribution and a strongly nonlinear weight function $l$, compute the true importance-sampling distribution $q$ by numerical integration, run the proposed backward SDE, and measure the Jensen–Shannon divergence between generated samples and $q$; if the divergence grows with the curvature of $l$ as predicted by the conditional-variance term, the approximation's practical limit is identified. More directly, track the conditional variance $\\mathbb{E}[\\|X'_0 - \\bar{x}'_{0|x,t}\\|^2 \\mid X'_t = x]$ along the reverse trajectory; if it stays large for a substantial fraction of the trajectory, the bound in Theorem 1 predicts a large score gap at those times, revealing where the method fails.","supporting_citations":[{"cited_title":"Diffusion posterior sampling for general noisy inverse problems,","cited_arxiv_id":null,"evidence_quote":"Uses Tweedie's approach for posterior sampling, a direct predecessor of the guidance idea."},{"cited_title":"Würstchen: An efficient architecture for large-scale text-to-image diffusion models,","cited_arxiv_id":null,"evidence_quote":"The StableCascade foundation model used for large-scale text-to-image experiments."}],"review_version":1}