REVIEW 15 cited by
Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The ability to collect a large dataset of human preferences from text-to-image users is usually limited to companies, making such datasets inaccessible to the public. To address this issue, we create a web app that enables text-to-image users to generate images and specify their preferences. Using this web app we build Pick-a-Pic, a large, open dataset of text-to-image prompts and real users' preferences over generated images. We leverage this dataset to train a CLIP-based scoring function, PickScore, which exhibits superhuman performance on the task of predicting human preferences. Then, we test PickScore's ability to perform model evaluation and observe that it correlates better with human rankings than other automatic evaluation metrics. Therefore, we recommend using PickScore for evaluating future text-to-image generation models, and using Pick-a-Pic prompts as a more relevant dataset than MS-COCO. Finally, we demonstrate how PickScore can enhance existing text-to-image models via ranking.
Forward citations
Cited by 15 Pith papers
-
JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models
JAGG replaces per-step gradient backpropagation in diffusion GRPO with two endpoint backward passes joined by timestep-weighted interpolation, giving ~2x backward-pass savings at modest quality cost.
-
Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training
SURE learns sample-adaptive variance in a latent reward model and uses that variance to weight dense post-training feedback, improving image and video diffusion alignment in reported experiments.
-
Latent Reward Registers for Diffusion Preference Alignment
Prepending learnable register tokens to a frozen DiT yields dense latent reward estimates, enabling faster on-policy distillation and training-free guided sampling that improve preference alignment.
-
NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning
NormGuard, a hinge penalty on excess velocity norm during RL post-training of flow models, improves perceptual quality and realism without sacrificing reward.
-
Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions
Z-Reward trains a 27B reasoning teacher VLM on score distributions via GDSO and distills it via RISD into a 9B student, reaching 89.6% and 88.6% human preference accuracy with 41.3% optimization gain over SFT baseline.
-
Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics
Prototypicality bias: common text-to-image metrics systematically prefer plausible-but-wrong images over correct non-prototypical ones; PROTOSCORE mitigates but does not eliminate the failure.
-
MaskAttn-SDXL: Controllable Region-Level Text-To-Image Generation
MaskAttn-SDXL adds token-conditioned spatial gating to SDXL cross-attention to sparsify irrelevant token-to-location bindings and improve region-level controllability without retraining or inference edits.
-
AU-IQA: A Benchmark Dataset for Perceptual Quality Assessment of AI-Enhanced User-Generated Content
AU-IQA is a new 4,800-image benchmark showing existing quality models, mainly those trained on ordinary user content, only partially predict human ratings of AI-enhanced photos.
-
HPSv3: Towards Wide-Spectrum Human Preference Score
HPSv3, trained on the new 1.08M-pair HPDv3 dataset, reaches 76.9% pairwise preference accuracy on its own test set and Spearman 0.94 against human model rankings, and is used to iteratively refine generated images (CoHP).
-
Transition Matching: Scalable and Flexible Generative Modeling
Transition Matching unifies flow matching and continuous autoregressive generation as discrete-time Markov processes, with three variants that improve text-to-image quality and speed.
-
Instant Preference Alignment for Text-to-Image Diffusion Models
An MLLM-driven, training-free pipeline extracts preference keywords from a reference image and modulates diffusion cross-attention at global and regional levels for instant, multi-round preference-aligned image generation.
-
ImageReFL: Balancing Quality and Diversity in Human-Aligned Diffusion Models
ImageReFL combines base-model early diffusion steps with a real-image-based fine-tuning objective to improve the quality-diversity trade-off in reward-aligned text-to-image generation.
-
$I^2G$: Generating Instructional Illustrations via Text-Conditioned Diffusion
A pairwise-conditioned diffusion model generates instructional illustrations from procedural text and is finetuned with a text-image alignment reward.
-
Inference-Time Alignment Control for Diffusion Models with Reinforcement Learning Guidance
Blending a base diffusion model with its RL-finetuned version at sampling time lets users dial alignment strength, with the blend weight corresponding to the KL-regularization coefficient beta/w.
-
Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences
SmPO-Diffusion improves diffusion-model preference alignment with reward-model soft labels and ReNoise inversion, reporting higher human-preference scores and up to 26x lower training cost than Diffusion-KTO.
Discussion (0). Continue with ORCID to comment.