Recognition: unknown
It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Models
read the original abstract
Contemporary text-to-image models exhibit a surprising degree of mode collapse, as can be seen when sampling several images given the same text prompt. Previous work has attempted to address this issue by steering the model using guidance mechanisms, or by generating a large pool of candidates and refining them. In this work, we take a different direction and aim for diversity in generations via noise optimization. Specifically, we show that a simple noise optimization objective can mitigate mode collapse while preserving the fidelity of the base model. We also analyze the frequency characteristics of the noise and show that alternative noise initializations with different frequency profiles can improve both optimization and search. Our experiments demonstrate that noise optimization yields superior results in terms of generation quality and diversity.
This paper has not been read by Pith yet.
Forward citations
Cited by 3 Pith papers
-
STRIDE: Training-Free Diversity Guidance via PCA-Directed Feature Perturbation in Single-Step Diffusion Models
STRIDE boosts diversity in one-step diffusion models by injecting PCA-aligned pink noise into transformer features while preserving text alignment and quality.
-
Couple to Control: Joint Initial Noise Design in Diffusion Models
Coupled initial noises in diffusion models, with designed dependence but unchanged marginal Gaussians, improve generated image diversity on Stable Diffusion variants while preserving quality and alignment.
-
Diverse Sampling in Diffusion Models with Marginal Preserving Particle Guidance
EDDY adds diversity to diffusion-model samples by using kernel-based anti-symmetric pairwise drifts that preserve marginal distributions via Fokker-Planck symmetries, with practical approximations for expensive cases.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.