Pith. sign in

REVIEW 5 cited by

A Dense Reward View on Aligning Text-to-Image Diffusion with Preference

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.08265 v2 pith:464BD74N submitted 2024-02-13 cs.CV

Shentao Yang , Tianqi Chen , Mingyuan Zhou This is my paper · ORCID
classification cs.CV
keywords preferencediffusiongenerationrewardaligningalignmentchaindense
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Aligning text-to-image diffusion model (T2I) with preference has been gaining increasing research attention. While prior works exist on directly optimizing T2I by preference data, these methods are developed under the bandit assumption of a latent reward on the entire diffusion reverse chain, while ignoring the sequential nature of the generation process. This may harm the efficacy and efficiency of preference alignment. In this paper, we take on a finer dense reward perspective and derive a tractable alignment objective that emphasizes the initial steps of the T2I reverse chain. In particular, we introduce temporal discounting into DPO-style explicit-reward-free objectives, to break the temporal symmetry therein and suit the T2I generation hierarchy. In experiments on single and multiple prompt generation, our method is competitive with strong relevant baselines, both quantitatively and qualitatively. Further investigations are conducted to illustrate the insight of our approach.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    UDM-GRPO is the first RL integration for uniform discrete diffusion models, using final clean samples as actions and forward-process trajectory reconstruction to raise GenEval accuracy from 69% to 96% and OCR accuracy...

  2. Latent Reward Registers for Diffusion Preference Alignment

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Prepending learnable register tokens to a frozen DiT yields dense latent reward estimates, enabling faster on-policy distillation and training-free guided sampling that improve preference alignment.

  3. ShortFT: Diffusion Model Alignment via Shortcut-based Fine-Tuning

    cs.CV 2025-07 conditional novelty 6.0 of 10

    ShortFT fine-tunes Stable Diffusion by backpropagating reward gradients through a distilled few-step shortcut denoising chain, improving alignment scores over DRaFT-LV and DRTune.

  4. RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    RePrompt uses RL-trained reasoning traces to enhance text-to-image prompts, boosting spatial composition and counting scores across FLUX, SD3, and PixArt-Σ while keeping image generators fixed.

  5. Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences

    cs.CV 2025-06 conditional novelty 4.0 of 10

    SmPO-Diffusion improves diffusion-model preference alignment with reward-model soft labels and ReNoise inversion, reporting higher human-preference scores and up to 26x lower training cost than Diffusion-KTO.

Pith tools