{"id":"e2aa7882-e03e-4903-961c-2f0f1b342889","arxiv_id":"2412.19992","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A stochastic-start ODE sampler for diffusion bridge models avoids the singular start of the probability-flow ODE and beats prior samplers with fewer neural network evaluations.","lead":"This paper introduces a fast sampling algorithm for diffusion bridge models, which generate images from corrupted inputs. It starts with a stochastic step and then solves an ordinary differential equation, cutting computation while improving image quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3 justifies q_post by a conditional KL that omits b_t^2 Cov(X0|y) from the true marginal start; if this covariance is non-negligible at the chosen tau, the deterministic PF-ODE inherits an under-dispersed initial condition and the method's theoretical basis is incomplete.","rationale":"The reader's weakest_assumption identifies essentially the same issue: q_post is justified by a conditional KL and ignores the b_τ^2 Cov(X0|y) term in the true marginal q_{τ|y}. I agree that this is the most load-bearing theoretical soft spot. The central claim is that the limited performance of pure ODE samplers is caused by the PF-ODE singularity at T and that posterior sampling at the first reverse step fixes it. For that claim to hold, the distribution placed at time τ must be close to the true marginal q_{τ|y}, because the subsequent Heun integration is deterministic and cannot resample from the missing covariance. Theorem 3 does not establish this closeness; it establishes only that q_post is the best Gaussian approximation to the conditional transition kernel averaged over X0, which is a different object. The concern is not merely a disagreement with consensus: it is an internal gap between the theorem's objective and the sampler's requirement. The proposed test settles the question directly by measuring the omitted covariance term and by re-running the same sampler with a start that includes it. If the covariance term is negligible, the theorem's omission is harmless and the verdict should remain conditional only on the other empirical issues (ablation, error bars, code). If the term matters, the theoretical justification is incomplete and the method's success needs a different explanation, which would strengthen the case for a conditional or revised verdict. Since the reader's verdict is already CONDITIONAL, my read does not change it; it sharpens the specific condition that should be checked.","tokens_in":12188,"tokens_out":10637,"duration_ms":117700,"concrete_test":"Estimate the conditional covariance Cov(X0|y) on the I2SB and DDBM validation sets (e.g., by drawing multiple clean samples conditioned on each y, or by training a lightweight variance predictor), and compute R = b_τ^2 Tr Cov(X0|y) / (d c_τ^2) at the actual τ used with N=20 and N=15. Then modify Algorithm 1 to sample the start from N(a_τ y + b_τ \\hat{X}_0^{(T)}, c_τ^2 I + b_τ^2 \\widehat{Cov}(X0|y)) instead of q_post, keeping every later Heun step fixed. If FID does not materially change, the missing covariance is negligible and the concern does not land; if FID improves, the paper's start distribution is suboptimal and Theorem 3's objective is the wrong one. A complementary check is to replace the stochastic start with its deterministic mean a_τ y + b_τ \\hat{X}_0^{(T)}; if FID stays competitive, the stochasticity itself is not the source of the reported gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the stochastic start distribution q_post in Eq. (14), whose justification rests on Theorem 3 (Appendix A.2). That theorem solves min_q E_{X0~qdata(.|y)} KL(q(Xτ|y) || q_{τ|0,y}(Xτ|X0,y)) under a Gaussian constraint. The objective averages over clean images but each KL is conditional on a single X0; its minimizer has mean a_τ y + b_τ \\hat{X}_0^{(T)} and covariance c_τ^2 I. The distribution actually needed to initialize the PF-ODE (9) is the marginal q_{τ|y}(Xτ|y) = E_{X0|y}[q_{τ|0,y}(Xτ|X0,y)], whose covariance is c_τ^2 I + b_τ^2 Cov(X0|y). Because the remaining sampler is deterministic, any under-dispersion of the initial condition is not corrected by the ODE and propagates to the output. The paper provides no bound showing b_τ^2 Cov(X0|y) is negligible at the chosen τ (e.g., by choosing τ close enough to T that b_τ is small). Comparing q_post to q_EM in Theorem 3 does not resolve this, since q_EM is also not the marginal. Consequently, the empirical gains could derive from Heun's solver or the particular choice of τ rather than from the posterior-sampling mechanism that is the paper's central contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes ODES3, a training-free ODE sampler for diffusion bridge models. It observes that the PF-ODE for diffusion bridges has a singular drift at the terminal time T, while the reverse SDE is well-defined. To initialize the ODE trajectory, the method samples X_τ from a Gaussian posterior q_post defined by plugging the data predictor's estimate of the clean image into the transition kernel (Eq. 14), then integrates the PF-ODE from τ to 0 with Heun's second-order solver. Experiments on super-resolution, JPEG restoration, and two image-to-image translation tasks show improved FID with 28–38 NFEs compared to SDE or hybrid samplers using 100+ NFEs.","tokens_in":12522,"tokens_out":7517,"duration_ms":66366,"significance":"If the theoretical justification were complete, this would be a valuable contribution: it is a simple, training-free acceleration applicable to pretrained bridge models, with clean singularity theorems (Thms 1–2) and a correct conditional-KL computation (Thm 3, App. A.2). The empirical results are strong and the method is fully reproducible given the released pretrained models. However, the central theoretical claim—that the stochastic start q_post is the appropriate marginal for initializing the PF-ODE—has a gap, because Theorem 3 optimizes a conditional KL rather than the marginal KL. This needs to be addressed before the method's theoretical basis is solid.","major_comments":[{"comment":"The distribution q_post(X_τ|y) = q_{τ|0,y}(X_τ | \\hat{X}_0^{(T)}, y) has covariance c_τ^2 I, but the true marginal start needed by the PF-ODE (9) is q_{τ|y}(X_τ|y) = E_{X_0|y}[q_{τ|0,y}(X_τ|X_0,y)], whose covariance is c_τ^2 I + b_τ^2 Cov(X_0|y). Theorem 3 solves min_q E_{X_0}[KL(q || q_{τ|0,y}(·|X_0,y))] under a Gaussian constraint; this objective does not include the between-X_0 covariance term, so the optimal covariance is c_τ^2 I. Because the subsequent ODE is deterministic, under-dispersion of the initial condition is never corrected. The paper provides no bound on b_τ^2 Cov(X_0|y) at the chosen τ (e.g., by choosing τ close enough to T that b_τ is small) and no experiment measuring the sensitivity to this term. This is a load-bearing gap, as the stochastic start is the paper's main contribution.","section":"Section III-B, Eq. (14), Appendix A.2"},{"comment":"The comparison to q_EM in Theorem 3 does not resolve the marginal issue. q_EM is also not the marginal q_{τ|y}; inequality (15) compares expected conditional KLs, and both distributions could be poor approximations to the marginal. A correct justification would either optimize KL(q || q_{τ|y}) directly, or show that b_τ^2 Cov(X_0|y) is negligible for the chosen schedule and tasks.","section":"Theorem 3 and Eq. (15)"},{"comment":"No ablation isolates the effect of the stochastic start. The reported gains could in principle come from the use of Heun's solver or from the specific choice of τ rather than from posterior sampling. I recommend an ablation comparing (i) pure Euler/Heun ODE from a collapsed start (e.g., mean a_τ y + b_τ \\hat{X}_0^{(T)} without noise), (ii) q_EM stochastic start, and (iii) the proposed q_post start, all with the same solver and time schedule. This would directly support the causal claim in the abstract and Section I.","section":"Section IV, Algorithm 1"}],"minor_comments":[{"comment":"The phrase 'To proof Theorem 3' should be 'To prove Theorem 3'; similarly, 'inequity (15)' should be 'inequality (15)'.","section":"Appendix A.2"},{"comment":"The name 'Frechet' should be spelled 'Fréchet'.","section":"Abstract and Introduction"},{"comment":"The comment 'SDE sampling at T' on line 4 is misleading; the step is posterior sampling, not SDE discretization. Consider renaming it to 'Posterior sampling at T'.","section":"Algorithm 1"},{"comment":"The footnote that baseline results are taken from DDBM and DBIM without exact NFE should be stated more prominently in the main text, since a reader might otherwise treat those rows as directly comparable in NFE.","section":"Table III"}],"recommendation":"major_revision","confidential_remarks":"The main concern is the marginal-covariance gap: the paper's theory overstates the optimality of q_post as the start of the PF-ODE. The self-citation [24] is not load-bearing. The paper is otherwise well written and the empirical contribution is solid."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is the first paper I've seen that explains why pure ODE samplers are blurry on bridge models—the PF-ODE drift is singular at t=T—and it patches the start with a posterior-sampling step before running Heun's solver. The empirical numbers are impressive: 2.7–4.2x speedup with better FID on four tasks. But the theoretical justification for the stochastic start has a hole that the authors should be asked to fix.\n\nWhat's genuinely new: Theorems 1 and 2 cleanly separate the reverse-SDE drift (finite) from the PF-ODE drift (singular). The idea of using the pretrained data predictor to jump from the corrupted image y to a plausible intermediate X_tau is clever and training-free. The paper is clear, the math is mostly careful, and the comparisons against I2SB and DDBM are fair.\n\nWhere it gets soft: the stress-test note is right. Theorem 3 solves a KL minimization per X0, in expectation, but each KL is conditional on a specific X0. The minimizer has covariance c_tau^2 I, whereas the true marginal q_tau|y has covariance c_tau^2 I + b_tau^2 Cov(X0|y). Since the rest of the sampler is deterministic, an under-dispersed initial condition propagates. The paper gives no bound on b_tau^2 Cov(X0|y) at the chosen tau, and the comparison to q_EM doesn't rescue the argument because q_EM is also not the marginal. On top of that, there is no ablation isolating the stochastic start. The gains could be coming from Heun's solver, the schedule, or the point estimate \\hat{X}_0^(T) rather than from posterior sampling per se. No error bars, no code, and the time schedule details are vague.\n\nThese are fixable, not fatal. The singularity identification is a real contribution, and the empirical claim is plausible. I'd want an ablation where the stochastic start is replaced by a one-step Euler from y, and a quantitative check on the covariance gap. I'd also want error bars and code.\n\nFor whom: anyone working on fast sampling for diffusion bridges or Doob's-h-transform models. It's a solid workshop or conference paper after revision. Send it to a serious referee; the main missing piece is the marginal-KL analysis.","headline":"A clever, well-written sampler for diffusion bridges with a genuine theoretical gap in the stochastic-start justification; worth peer review with requests for ablations.","tokens_in":13021,"tokens_out":3092,"would_cite":true,"duration_ms":29191,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that pure ODE samplers in diffusion bridge models fail because the probability flow ODE is singular at the start, and fixes it with a Gaussian stochastic start followed by Heun's second-order solver.","keywords":["diffusion bridge model","probability flow ODE","ODE sampler","posterior sampling","Heun's method","image restoration","image translation","stochastic start"],"falsifier":"Compute the empirical covariance of X_τ from many independent draws of the reverse SDE at time τ and compare it with the covariance $c_τ^{2}$ I used in the stochastic start; a large excess variance would indicate the start is under-dispersed and the deterministic trajectory is biased.","tokens_in":12005,"feed_emoji":"⚡","tokens_out":6741,"duration_ms":54169,"temperature":0.7,"pith_summary":"Diffusion bridge models generate images by starting from a corrupted input rather than pure noise, but they have typically required slow stochastic (SDE) samplers. This paper argues that deterministic ODE samplers underperform on these models because the probability flow ODE is singular at the very first step, where the process is pinned to the corrupted image. The proposed sampler, ODES3, sidesteps the singularity by drawing the first intermediate state from a Gaussian posterior centered near the predicted clean image, then integrates the well-behaved ODE with Heun's second-order solver. On super-resolution, JPEG restoration, and two image-translation tasks, this training-free recipe matches or beats prior samplers while using 28–38 neural function evaluations instead of 100 or more.","feed_headline":"Stochastic start lets diffusion bridges use fast ODE solvers","feed_subtitle":"ODES3 beats 100-step SDE samplers with 28–38 network evaluations, no retraining.","key_machinery":"The paper's central object is the probability flow ODE (PF-ODE) of a diffusion bridge model, whose drift term combines a learned score with an explicit h-transform term. Theorems 1 and 2 characterize its boundary behavior: the drift is well-defined at the terminal time for the reverse SDE but singular for the PF-ODE. To bypass the singularity, the sampler uses a stochastic start defined by the Gaussian posterior q_{τ|0,y}(X_τ | \\hat{X}$_0^{{(T)}}$, y), where \\hat{X}$_0^{{(T)}}$ = D_θ(y, y, T) is the trained data predictor's estimate of the clean image. This single draw replaces the first Euler step; then Heun's second-order method, an explicit two-stage Runge–Kutta scheme, integrates the PF-ODE.","core_discovery":"The central claim is that the limited performance of pure ODE samplers in diffusion bridge models is caused by the singular behavior of the PF-ODE at the start of the generative process. The paper shows that the non-linear drift in the PF-ODE has no finite limit as time approaches the terminal point, while the reverse SDE's drift is well-defined. To jump over the singularity, it introduces posterior sampling at the first reverse step: X_τ is drawn from q_{τ|0,y}(X_τ | \\hat{X}$_0^{{(T)}}$, y), a Gaussian whose mean uses the data predictor's estimate of the clean image and whose covariance is the analytic $c_τ^{2}$ I. The paper proves this start is closer in KL divergence to the exact transition distribution than a single Euler–Maruyama step. After the jump, Heun's second-order solver integrates the PF-ODE from τ to 0, yielding the reported gains in FID and speed.","pith_inferences":["The stochastic start uses a covariance that ignores the spread of clean images around the predicted mean; a natural testable extension is to inflate the start covariance or learn it, which might help tasks with highly ambiguous degradations.","The choice of the jump time τ is taken from the pretrained model's schedule; tuning τ directly could trade off singularity avoidance against the distance the ODE must integrate.","The same stochastic-start idea could be applied to other conditional generation settings where an ODE endpoint is singular, such as inpainting or colorization.","Since Theorem 3 optimizes a conditional KL, the start is optimal only within the Gaussian family; a mixture-based start might do better when the posterior is multimodal."],"forward_implications":["ODES3 matches or beats the original samplers of I2SB and DDBM on all four tasks with far fewer neural function evaluations.","The method requires no additional training and works with already-pretrained diffusion bridge models.","It reduces FID by 9–18% on restoration tasks and by 11–70% on translation tasks compared to prior samplers.","Inference time per image drops by roughly 2.7× on restoration tasks and 3.6–4.2× on translation tasks."],"supporting_citations":[{"why":"Supplies the denoising diffusion bridge model formulation and the hybrid sampler whose inefficiency motivates the paper.","marker":"[8]"},{"why":"Provides the pretrained I2SB bridge models and the restoration tasks used for evaluation.","marker":"[7]"},{"why":"Unified framework for diffusion bridge models that gives the forward SDE and transition kernels used in the derivations.","marker":"[22]"},{"why":"Introduces Heun's second-order solver for fast PF-ODE sampling in diffusion models, which the paper adapts.","marker":"[3]"},{"why":"Standard reference for Heun's second-order method used in the integration step.","marker":"[13]"},{"why":"Foundational connection between the reverse SDE and the probability flow ODE that underlies the setup.","marker":"[1]"}],"fun_headline_variants":["ODE sampler with stochastic start speeds diffusion bridge inference","Diffusion bridge ODE sampler: stochastic start enables fast sampling","High-order ODE solving for diffusion bridges via posterior start","Stochastic start jump makes high-order ODE samplers work for bridges","Heun's solver with stochastic start beats SDE for diffusion bridges"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that sampling X_τ from a single Gaussian centered at the predicted clean image, with covariance $c_τ^{2}$ I, is a faithful stand-in for the true marginal posterior, whose extra spread from the unknown clean image is ignored.","fun_headline_variants_meta":{"raw":{"variants":["ODE sampler with stochastic start speeds diffusion bridge inference","Diffusion bridge ODE sampler: stochastic start enables fast sampling","High-order ODE solving for diffusion bridges via posterior start","Stochastic start jump makes high-order ODE samplers work for bridges","Heun's solver with stochastic start beats SDE for diffusion bridges"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000638,"raw_usage":{"total_tokens":2957,"prompt_tokens":979,"completion_tokens":1978,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":1892}},"tokens_in":595,"tokens_out":1978,"duration_ms":13761,"temperature":1.0,"reasoning_tokens":1892,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:41:22.900439+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the empirical covariance of X_τ from many independent draws of the reverse SDE at time τ and compare it with the covariance $c_τ^{2}$ I used in the stochastic start; a large excess variance would indicate the start is under-dispersed and the deterministic trajectory is biased.","supporting_citations":[{"cited_title":"Consistency Diffusion Bridge Models","cited_arxiv_id":"2410.22637","evidence_quote":"Unified framework for diffusion bridge models that gives the forward SDE and transition kernels used in the derivations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Standard reference for Heun's second-order method used in the integration step."}],"review_version":1}