REVIEW 10 cited by
Golden Noise for Diffusion Models: A Learning Framework
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Text-to-image diffusion model is a popular paradigm that synthesizes personalized images by providing a text prompt and a random Gaussian noise. While people observe that some noises are ``golden noises'' that can achieve better text-image alignment and higher human preference than others, we still lack a machine learning framework to obtain those golden noises. To learn golden noises for diffusion sampling, we mainly make three contributions in this paper. First, we identify a new concept termed the \textit{noise prompt}, which aims at turning a random Gaussian noise into a golden noise by adding a small desirable perturbation derived from the text prompt. Following the concept, we first formulate the \textit{noise prompt learning} framework that systematically learns ``prompted'' golden noise associated with a text prompt for diffusion models. Second, we design a noise prompt data collection pipeline and collect a large-scale \textit{noise prompt dataset}~(NPD) that contains 100k pairs of random noises and golden noises with the associated text prompts. With the prepared NPD as the training dataset, we trained a small \textit{noise prompt network}~(NPNet) that can directly learn to transform a random noise into a golden noise. The learned golden noise perturbation can be considered as a kind of prompt for noise, as it is rich in semantic information and tailored to the given text prompt. Third, our extensive experiments demonstrate the impressive effectiveness and generalization of NPNet on improving the quality of synthesized images across various diffusion models, including SDXL, DreamShaper-xl-v2-turbo, and Hunyuan-DiT. Moreover, NPNet is a small and efficient controller that acts as a plug-and-play module with very limited additional inference and computational costs, as it just provides a golden noise instead of a random noise without accessing the original pipeline.
Forward citations
Cited by 10 Pith papers
-
LatSearch: Latent Reward-Guided Search for Faster Inference-Time Scaling in Video Diffusion
LatSearch improves video diffusion quality and efficiency by scoring intermediate latents with a trained reward model and performing reward-guided resampling plus final pruning.
-
FastInit: Fast Noise Initialization for Temporally Consistent Video Generation
A single-pass learned noise predictor, trained to imitate FreeInit's outputs, gives temporally more consistent text-to-video generation at near-zero added inference cost.
-
Canonical Latent Representations in Conditional Diffusion Models
Projecting out the top Jacobian singular directions in a conditional diffusion model's latent space yields class prototypes that, when used for distillation, improve classifier robustness and generalization.
-
NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Models
NoiseAR replaces the fixed Gaussian starting noise of a diffusion model with a learned, text-conditioned, autoregressive prior over noise patches and reports modest improvements in text-to-image metrics.
-
Breaking Latent Prior Bias in Detectors for Generalizable AIGC Image Detection
Optimizing the initial latent noise of a diffusion model creates on-manifold adversarial examples that, added to training, make AIGC detectors generalize much better to unseen generators.
-
Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation
Post-hoc distillation with a PDE-residual loss on final samples avoids the Jensen gap and yields one-step physics-constrained generation.
-
Scaling Image and Video Generation via Test-Time Evolutionary Search
Evolutionary search over denoising trajectories improves image and video generation quality and diversity as test-time compute increases, without retraining the generative model.
-
A Minimalist Method for Fine-tuning Text-to-Image Diffusion Models
A one-step RL method learns a prompt-conditioned initial noise distribution for a frozen diffusion model, improving scores on the training reward models, with the largest gains at low inference steps.
-
Straighten Viscous Rectified Flow via Noise Optimization
VRFNO claims state-of-the-art one- and few-step image generation by straightening rectified flow trajectories, but its sampler relies on real images from the dataset.
-
Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations
Ctrl-Z Sampling improves text-to-image outputs by adaptively rolling back and re-exploring when a reward model flags a quality plateau, at roughly 3 to 9 times the usual compute.
Discussion (0). Sign in to comment.