Pith. sign in

REVIEW 8 cited by

Re-Imagen: Retrieval-Augmented Text-to-Image Generator

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2209.14491 v3 pith:DX7SWPYQ submitted 2022-09-29 cs.CV cs.AIcs.LG

Wenhu Chen , Hexiang Hu , Chitwan Saharia , William W. Cohen This is my paper
classification cs.CVcs.AIcs.LG
keywords entitiesre-imagentextimagesretrievalimagegeneratingmodel
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Research on text-to-image generation has witnessed significant progress in generating diverse and photo-realistic images, driven by diffusion and auto-regressive models trained on large-scale image-text data. Though state-of-the-art models can generate high-quality images of common entities, they often have difficulty generating images of uncommon entities, such as `Chortai (dog)' or `Picarones (food)'. To tackle this issue, we present the Retrieval-Augmented Text-to-Image Generator (Re-Imagen), a generative model that uses retrieved information to produce high-fidelity and faithful images, even for rare or unseen entities. Given a text prompt, Re-Imagen accesses an external multi-modal knowledge base to retrieve relevant (image, text) pairs and uses them as references to generate the image. With this retrieval step, Re-Imagen is augmented with the knowledge of high-level semantics and low-level visual details of the mentioned entities, and thus improves its accuracy in generating the entities' visual appearances. We train Re-Imagen on a constructed dataset containing (image, text, retrieval) triples to teach the model to ground on both text prompt and retrieval. Furthermore, we develop a new sampling strategy to interleave the classifier-free guidance for text and retrieval conditions to balance the text and retrieval alignment. Re-Imagen achieves significant gain on FID score over COCO and WikiImage. To further evaluate the capabilities of the model, we introduce EntityDrawBench, a new benchmark that evaluates image generation for diverse entities, from frequent to rare, across multiple object categories including dogs, foods, landmarks, birds, and characters. Human evaluation on EntityDrawBench shows that Re-Imagen can significantly improve the fidelity of generated images, especially on less frequent entities.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MV-RAG: Retrieval Augmented Multiview Diffusion

    cs.CV 2025-08 conditional novelty 7.0 of 10

    A retrieval-augmented multiview diffusion model conditions on web images to generate 3D-consistent views of rare concepts, trained with a hybrid 3D/2D objective and evaluated on a new OOD benchmark.

  2. ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    ToolArtist trains a unified multimodal model to reason, search the web, and generate images as one policy, improving scores on WISE and WorldGenBench-Humanities.

  3. USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learning

    cs.CV 2025-08 conditional novelty 6.0 of 10

    USO trains one DiT model for subject-driven, style-driven, and joint generation by disentangling content and style from triplet data and adding a style-reward objective, claiming SOTA on USO-Bench.

  4. FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A method for multi-subject image personalization that fuses independently trained LoRA modules at inference time on visual autoregressive models.

  5. BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A dual-stream diffusion model trained with Blender-render conditioning, source masking, and object jittering performs 3D-grounded multi-object editing and compositing better than existing baselines on three video datasets.

  6. IA-T2I: Internet-Augmented Text-to-Image Generation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    IA-T2I uses active retrieval, hierarchical image selection, and self-reflection to supply internet reference images to T2I models, improving generation accuracy on uncertain-knowledge prompts.

  7. Retrieval Augmented Comic Image Generation

    cs.CV 2025-06 reject novelty 5.0 of 10

    RaCig combines retrieval-based character assignment with regional IP-Adapter and ControlNet injection to generate comic panels with consistent characters and diverse gestures.

  8. Dimension-Reduction Attack! Video Generative Models are Experts on Controllable Image Synthesis

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A video diffusion model, HunyuanVideo-I2V, is adapted with mixup transitions, frame-skip position embeddings, and attention masking to outperform image-only models on several controllable image generation benchmarks.

Pith tools