REVIEW 6 cited by
ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Do the rich representations of multi-modal diffusion transformers (DiTs) exhibit unique properties that enhance their interpretability? We introduce ConceptAttention, a novel method that leverages the expressive power of DiT attention layers to generate high-quality saliency maps that precisely locate textual concepts within images. Without requiring additional training, ConceptAttention repurposes the parameters of DiT attention layers to produce highly contextualized concept embeddings, contributing the major discovery that performing linear projections in the output space of DiT attention layers yields significantly sharper saliency maps compared to commonly used cross-attention maps. ConceptAttention even achieves state-of-the-art performance on zero-shot image segmentation benchmarks, outperforming 15 other zero-shot interpretability methods on the ImageNet-Segmentation dataset. ConceptAttention works for popular image models and even seamlessly generalizes to video generation. Our work contributes the first evidence that the representations of multi-modal DiTs are highly transferable to vision tasks like segmentation.
Forward citations
Cited by 6 Pith papers
-
Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers
Chat-template tokens in LLM-conditioned DiTs act as implicit semantic registers: they absorb object identity from image latents and maintain it, while direct prompt-reading heads are causally inert.
-
S3OD: Towards Generalizable Salient Object Detection with Synthetic Data
A 139k-image synthetic dataset with diffusion- and DINO-derived masks, trained with a multi-mask decoder, improves cross-dataset salient-object detection and reaches state-of-the-art after fine-tuning.
-
LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformers
LoRAShop localizes each LoRA's effect to attention-derived spatial masks inside a Flux transformer, enabling training-free multi-concept image generation and editing.
-
From Generation to Generalization: Emergent Few-Shot Learning in Video Diffusion Models
Frozen CogVideoX1.5, adapted with LoRA on 3 to 30 input-output videos, performs segmentation, pose estimation, and abstract reasoning (ARC-AGI 16.75%) with modest but real generalization.
-
FAIR_XAI: Improving Multimodal Foundation Model Fairness via Explainability for Wellbeing Assessment
Zero-shot vision-language models are unreliable and vary widely for depression screening, and explainability-based fairness interventions often trade away accuracy without reliable fairness gains.
-
Attention of a Kiss: Exploring Attention Maps in Video Diffusion for XAIxArts
A method and case study for visualizing cross-attention maps in Wan video diffusion transformers, showing token-region alignment over time and their use as artistic material.
Discussion (0). Sign in to comment.