Guidance with Spherical Gaussian Constraint for Conditional Diffusion
Reviewed by Pithpith:FQJUEWBMopen to challenge →
read the original abstract
Recent advances in diffusion models attempt to handle conditional generative tasks by utilizing a differentiable loss function for guidance without the need for additional training. While these methods achieved certain success, they often compromise on sample quality and require small guidance step sizes, leading to longer sampling processes. This paper reveals that the fundamental issue lies in the manifold deviation during the sampling process when loss guidance is employed. We theoretically show the existence of manifold deviation by establishing a certain lower bound for the estimation error of the loss guidance. To mitigate this problem, we propose Diffusion with Spherical Gaussian constraint (DSG), drawing inspiration from the concentration phenomenon in high-dimensional Gaussian distributions. DSG effectively constrains the guidance step within the intermediate data manifold through optimization and enables the use of larger guidance steps. Furthermore, we present a closed-form solution for DSG denoising with the Spherical Gaussian constraint. Notably, DSG can seamlessly integrate as a plugin module within existing training-free conditional diffusion methods. Implementing DSG merely involves a few lines of additional code with almost no extra computational overhead, yet it leads to significant performance improvements. Comprehensive experimental results in various conditional generation tasks validate the superiority and adaptability of DSG in terms of both sample quality and time efficiency.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures
URGE performs unbiased inference-time scaling for diffusion models by attaching multiplicative path weights from Girsanov estimation and resampling trajectories, with a proven equivalence to prior particle-wise SMC schemes.
-
Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Models
Under a Gaussian prior assumption, zero-shot diffusion posterior samplers for inverse problems admit closed-form spectral representations that enable a new parameter-selection framework balancing perceptual quality an...
-
FlowLPS: Langevin-Proximal Sampling for Flow-based Inverse Problem Solvers
FlowLPS perturbs flow-model estimates with Langevin steps then applies proximal refinement to balance fidelity and perceptual quality on linear inverse problems.
-
Saving Foundation Flow-Matching Priors for Inverse Problems
FMPlug adapts foundation flow-matching models into practical priors for inverse problems by combining instance-guided warm-start with sharp Gaussianity regularization, showing superior results on image restoration and...
-
A Survey on Diffusion Models for Inverse Problems
A survey that introduces taxonomies for categorizing pre-trained diffusion model methods applied to inverse problems and analyzes their connections and challenges.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.