{"id":"cf6c243e-0e9b-4d10-9bb4-0236e103b8ea","arxiv_id":"2606.22521","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Divergence-induced weighted denoising replaces MSE loss in diffusion models to improve robustness under data contamination, with Hellinger yielding explicit exponential weights.","lead":"The paper replaces the standard MSE denoising loss in diffusion models with a nonlinear weighting derived from f-divergences to create a robust training objective. This could allow generative models to train more reliably on real-world datasets that contain errors or contamination.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Lognormal likelihood-ratio assumption under Gaussian reverse kernel is the load-bearing step for reducing f-divergence to 1D denoising-error function.","rationale":"The reader's weakest_assumption directly identifies the single theoretical step whose validity determines whether the strongest_claim holds; confirming or refuting the lognormal scalar parameterization would settle the central justification without requiring new experiments.","tokens_in":1726,"tokens_out":346,"duration_ms":24308,"concrete_test":"Starting from two isotropic Gaussians N(μ, Σ) and N(μ+δ, Σ) in the reverse-kernel dimension, derive the exact log likelihood ratio, confirm it is normal, and verify that both mean and variance of the log-ratio are functions of a single scalar (‖δ‖ or equivalent); if either fails, recompute the f-divergence integral to check whether it remains univariate in the residual norm.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that, under DDPM's Gaussian reverse-kernel, the per-step likelihood ratio is lognormal and parameterized solely by a scalar mismatch. This permits the conditional f-divergence to collapse to a univariate function of the denoising residual, whose derivative then supplies the explicit influence weight. If the log-ratio is not exactly normal, or if its parameters depend on the vector direction of the mismatch rather than a scalar (e.g., its norm), the reduction fails and the claimed unification to bounded-influence weighted denoising (Hellinger yielding exponential weights) does not follow from the f-divergence construction. The empirical FID improvement under contamination would then be an ad-hoc nonlinear loss rather than a consequence of the stated theory.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript claims that replacing the standard MSE denoising loss in DDPM with a nonlinear transformation induced by an f-divergence yields a robust training objective. Under the Gaussian reverse-kernel, each per-step likelihood ratio is lognormal parameterized by a scalar mismatch, reducing the conditional f-divergence to a univariate function of the denoising error; its derivative supplies a residual-space influence weight. Bounded-influence divergences (Hellinger, negative exponential) suppress outliers, with Hellinger giving explicit exponential weights. On CIFAR-10 with 30% contamination, the method (NED) reduces FID from 93.0 (KL) to 77.5 and outperforms Huber and clipped MSE.","tokens_in":1915,"tokens_out":569,"duration_ms":19733,"significance":"If the lognormal reduction holds exactly, the work supplies a principled derivation of influence functions for diffusion training from the f-divergence family, directly linking to robust M-estimation. The empirical FID gain under contamination indicates practical utility for noisy data regimes with only modest overhead. The unification of training objectives via local divergences is a potential strength if the central assumption is verified.","major_comments":[{"comment":"The reduction of the conditional f-divergence to a one-dimensional function of the denoising error (and the subsequent derivation of explicit influence weights) rests on the claim that, under the DDPM Gaussian reverse kernel, the per-step likelihood ratio is exactly lognormal and parameterized solely by a scalar mismatch. No explicit derivation, expansion of the log-ratio, or verification that the parameters depend only on the scalar norm (rather than vector direction) is supplied; if this property fails, the claimed unification to bounded-influence weighted denoising does not follow from the f-divergence construction.","section":"Theoretical construction of local divergences"},{"comment":"Table or figure reporting FID scores under 30% contamination lists 93.0 (KL) to 77.5 (NED) without error bars, standard deviations, or the number of independent training runs, preventing assessment of whether the improvement over Huber and clipped MSE is statistically reliable or reproducible.","section":"Empirical results on CIFAR-10"}],"minor_comments":[{"comment":"The abstract introduces the acronym NED without expansion; the full name of the proposed method should appear on first use.","section":"Abstract"},{"comment":"The weighting function obtained as the derivative of the induced divergence is described in prose but lacks an explicit numbered equation that readers can cite when comparing to standard influence functions in robust statistics.","section":"Derivation of influence weights"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed comments. We address each major point below and will revise the manuscript accordingly to strengthen both the theoretical presentation and the empirical reporting.","responses":[{"response":"We agree that the manuscript states the lognormal property of the per-step likelihood ratio but does not supply the explicit derivation or verification that the parameters depend only on the scalar mismatch norm. In the revised version we will add a dedicated subsection that expands the log-ratio under the Gaussian reverse kernel, derives the lognormal form, and confirms independence from vector direction. This addition will make the reduction to a univariate function of the denoising error fully rigorous and directly support the subsequent influence-weight construction.","revision_made":"yes","referee_comment":"[Theoretical construction of local divergences] The reduction of the conditional f-divergence to a one-dimensional function of the denoising error (and the subsequent derivation of explicit influence weights) rests on the claim that, under the DDPM Gaussian reverse kernel, the per-step likelihood ratio is exactly lognormal and parameterized solely by a scalar mismatch. No explicit derivation, expansion of the log-ratio, or verification that the parameters depend only on the scalar norm (rather than vector direction) is supplied; if this property fails, the claimed unification to bounded-influence weighted denoising does not follow from the f-divergence construction."},{"response":"We concur that the absence of error bars, standard deviations, and the number of independent runs limits the ability to judge statistical reliability. The current numbers reflect single-run results. In the revision we will repeat the CIFAR-10 experiments under 30% contamination with at least five independent random seeds, report means and standard deviations, and include error bars in the relevant table/figure.","revision_made":"yes","referee_comment":"[Empirical results on CIFAR-10] Table or figure reporting FID scores under 30% contamination lists 93.0 (KL) to 77.5 (NED) without error bars, standard deviations, or the number of independent training runs, preventing assessment of whether the improvement over Huber and clipped MSE is statistically reliable or reproducible."}],"tokens_in":1437,"tokens_out":465,"duration_ms":17319,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core contribution is a construction that starts from standard DDPM reverse kernels and f-divergences, then derives per-step influence weights that turn the usual MSE objective into a weighted version. Bounded-influence choices like Hellinger produce an explicit exponential weight on large residuals. On CIFAR-10 with 30% contamination this drops FID from 93.0 to 77.5 and beats Huber and clipped MSE.\n\nThe reduction itself is the part that stands out: each local divergence collapses to a univariate function of the denoising error whose derivative supplies the weight. That step is not a routine extension of prior robust-loss work on diffusion.\n\nThe load-bearing assumption is that the per-step likelihood ratio is exactly lognormal and parameterized only by a scalar mismatch. If the ratio deviates from lognormality or if the mismatch carries directional dependence, the collapse to a one-dimensional weighting function does not follow and the claimed unification to robust M-estimation weakens. The abstract states the claim but gives no derivation or verification that the property holds exactly under the Gaussian kernel.\n\nEmpirically the numbers are concrete, yet the report mentions no error bars, no multiple random seeds, and no ablation on the contamination model itself. Those gaps make it hard to judge how stable the improvement is.\n\nThe work is aimed at practitioners who already train diffusion models and need a low-overhead way to down-weight bad samples. A reader who cares about robust generative modeling or connections between f-divergences and influence functions will find a usable recipe.\n\nIt deserves a serious referee. The empirical signal is there and the framing is new enough that the theory should be checked rather than desk-rejected.","headline":"The paper reduces f-divergences to per-sample weights on the DDPM denoising loss and shows FID gains under contamination, but the lognormal reduction step needs verification.","tokens_in":2389,"tokens_out":419,"would_cite":false,"duration_ms":11883,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Replacing the standard MSE loss in diffusion models with a nonlinear f-divergence transformation yields a weighted denoising objective that improves robustness to data contamination.","keywords":["diffusion models","robust training","f-divergence","data contamination","denoising loss","influence function","weighted training"],"falsifier":"An empirical measurement showing that the actual likelihood ratios under DDPM sampling deviate substantially from the predicted lognormal form for typical denoising errors would undermine the one-dimensional reduction of the f-divergence.","tokens_in":2631,"feed_emoji":"🛡️","tokens_out":658,"duration_ms":23458,"temperature":0.7,"pith_summary":"The paper establishes that diffusion model training can be reframed as summing local f-divergences derived from the Gaussian reverse process. This leads to an objective where each sample's contribution is weighted by the derivative of the divergence with respect to its denoising error. Bounded divergences like Hellinger automatically downweight samples with large errors. The result is a simple surrogate loss that outperforms standard MSE and other robust losses on contaminated datasets. This approach connects diffusion training directly to principles from robust statistics.","feed_headline":"Divergence weights replace MSE for robust diffusion training","feed_subtitle":"Hellinger-induced exponential weights cut FID by over 15 points on CIFAR-10 with 30 percent contamination","key_machinery":"divergence induced weighted denoising, in which the derivative of the local f-divergence with respect to the denoising residual serves as an influence weight on each training sample","core_discovery":"Under the Gaussian reverse-kernel structure of DDPM, each per-step likelihood ratio follows a lognormal distribution parameterized by a scalar mismatch, so the conditional f-divergence at each step reduces to a one-dimensional function of the denoising error. Summing these local divergences yields a training objective that unifies diffusion training as divergence induced weighted denoising, where the derivative of the induced divergence acts as a residual-space influence weight that controls the contribution of each sample. Bounded-influence divergences suppress large error samples, with Hellinger yielding an explicit exponential weight.","pith_inferences":["Similar weighting could be applied in other score-based generative models that rely on denoising objectives.","Testing the method on datasets with different types of contamination would reveal the scope of robustness gains.","The connection to M-estimation suggests potential for using other robust loss functions from statistics in diffusion training."],"forward_implications":["Bounded-influence f-divergences such as Hellinger and negative exponential suppress the contribution of samples with large denoising errors.","Hellinger divergence produces an explicit exponential weighting function.","The method reduces FID from 93.0 to 77.5 on CIFAR-10 with 30% contamination compared to standard KL-based training.","The approach requires only small additional computational overhead over standard diffusion training."],"fun_headline_variants":["Divergence weights supplant MSE in diffusion training","f-Divergence induces exponential robust weighting","Weighted denoising via bounded f-divergences","Hellinger divergence controls sample influence in DDPM","Local divergence reduces to denoising error weighting"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The per-step likelihood ratios in the diffusion reverse process follow a lognormal distribution controlled by a single scalar mismatch parameter.","fun_headline_variants_meta":{"raw":{"variants":["Divergence weights supplant MSE in diffusion training","f-Divergence induces exponential robust weighting","Weighted denoising via bounded f-divergences","Hellinger divergence controls sample influence in DDPM","Local divergence reduces to denoising error weighting"]},"model":"grok-4.3","cost_usd":0.004326,"raw_usage":{"total_tokens":2178,"prompt_tokens":680,"num_sources_used":0,"completion_tokens":67,"cost_in_usd_ticks":43262000,"prompt_tokens_details":{"text_tokens":680,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1431,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":680,"tokens_out":67,"duration_ms":12931,"temperature":1.0,"reasoning_tokens":1431,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T09:42:25.605794+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An empirical measurement showing that the actual likelihood ratios under DDPM sampling deviate substantially from the predicted lognormal form for typical denoising errors would undermine the one-dimensional reduction of the f-divergence.","supporting_citations":[],"review_version":1}