Pith. sign in

REVIEW 5 cited by

Diffusion Lens: Interpreting Text Encoders in Text-to-Image Pipelines

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.05846 v2 pith:VWAN3P6K submitted 2024-03-09 cs.CV cs.CL

Michael Toker , Hadas Orgad , Mor Ventura , Dana Arad , Yonatan Belinkov This is my paper · ORCID
classification cs.CVcs.CL
keywords textdiffusionencoderlensmodelsrepresentationcomparedconcepts
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Text-to-image diffusion models (T2I) use a latent representation of a text prompt to guide the image generation process. However, the process by which the encoder produces the text representation is unknown. We propose the Diffusion Lens, a method for analyzing the text encoder of T2I models by generating images from its intermediate representations. Using the Diffusion Lens, we perform an extensive analysis of two recent T2I models. Exploring compound prompts, we find that complex scenes describing multiple objects are composed progressively and more slowly compared to simple scenes; Exploring knowledge retrieval, we find that representation of uncommon concepts requires further computation compared to common concepts, and that knowledge retrieval is gradual across layers. Overall, our findings provide valuable insights into the text encoder component in T2I pipelines.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Verbalizable Representations Form a Global Workspace in Language Models

    cs.CL 2026-07 conditional novelty 7.0 of 10

    Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.

  2. Interpreting Language Model Hidden States at Scale

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A single low-rank, sparsely trained lens family decodes residual, attention, and MLP states in models up to 70B parameters, revealing that visible and causally effective locations for a behavior can differ.

  3. SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoders

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A supervised sparse autoencoder binds each concept to a single neuron, letting Stable Diffusion erase a concept by steering one latent.

  4. VSC: Visual Search Compositional Text-to-Image Diffusion Model

    cs.CV 2025-05 conditional novelty 5.0 of 10

    VSC improves attribute binding by fusing visual prototypes of per-pair reference images into text embeddings via a lightweight MLP, with a segmentation-based localization loss.

  5. $\textit{Revelio}$: Interpreting and leveraging semantic information in diffusion models

    cs.CV 2024-11 conditional novelty 5.0 of 10

    Diffusion model features, when decoded with k-sparse autoencoders, reveal interpretable visual concepts, and a lightweight classifier on the best layer (up_ft1 at t=25) beats prior diffusion-based classifiers on fine-...

Pith tools