REVIEW 13 cited by
T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Despite the impressive advances in text-to-image models, they often struggle to effectively compose complex scenes with multiple objects, displaying various attributes and relationships. To address this challenge, we present T2I-CompBench++, an enhanced benchmark for compositional text-to-image generation. T2I-CompBench++ comprises 8,000 compositional text prompts categorized into four primary groups: attribute binding, object relationships, generative numeracy, and complex compositions. These are further divided into eight sub-categories, including newly introduced ones like 3D-spatial relationships and numeracy. In addition to the benchmark, we propose enhanced evaluation metrics designed to assess these diverse compositional challenges. These include a detection-based metric tailored for evaluating 3D-spatial relationships and numeracy, and an analysis leveraging Multimodal Large Language Models (MLLMs), i.e. GPT-4V, ShareGPT4v as evaluation metrics. Our experiments benchmark 11 text-to-image models, including state-of-the-art models, such as FLUX.1, SD3, DALLE-3, Pixart-${\alpha}$, and SD-XL on T2I-CompBench++. We also conduct comprehensive evaluations to validate the effectiveness of our metrics and explore the potential and limitations of MLLMs.
Forward citations
Cited by 13 Pith papers
-
Moving Alphabet: A Controlled Study of Training Data for Text-to-Video Generation
Using a synthetic alphabet video testbed, balanced data mixing and caption precision are shown to dominate T2V model quality, while CFG and fine-tuning only partially compensate for corrupted captions.
-
Simile Understanding in Text-to-Image Models: An Evaluation Framework
A YOLO-based evaluation framework shows that text-to-image models consistently render the literal vehicle of a simile, a failure that CLIPScore and PickScore miss.
-
Poplar: A Scalable Pipeline for Human-Centric Image Dataset Synthesis
Poplar-9K is a curated human-centric image dataset generated through a reproducible attribute sampling, diffusion rendering, and vision-language inspection pipeline, with 9,401 accepted pairs from 11,765 candidates.
-
LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Model
LLaSO releases a 3.8B speech-language model, 25.5M training instances, and an evaluation benchmark, claiming a normalized score of 0.72.
-
Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation
A curated GPT-4o synthetic image dataset improves open-source generation models on instruction-following, surreal scenes, and multi-reference synthesis, plus two new benchmarks to measure those skills.
-
Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformers
TACA scales cross-modal attention logits by a timestep-dependent temperature to rebalance text and visual tokens, improving T2I-CompBench alignment on FLUX and SD3.5.
-
RICO: Improving Accuracy and Completeness in Image Recaptioning via Visual Reconstruction
RICO refines image captions by reconstructing them into images with a text-to-image model and asking GPT-4o to fix discrepancies against the original, iteratively, with a DPO-distilled fast variant.
-
MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding
MultiCompose combines embedding regularization, cross-attention suppression, and mask-guided denoising to compose independently personalized subjects into one image while keeping each subject's attributes exclusive.
-
Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models
ATLAS adds a Think–Plan–Paint loop with shared positional tokens to unified MLLMs, plus RL-based layout alignment, achieving large reported gains over prior layout-based unified models on compositional image generatio...
-
Sealing The Backdoor: Unlearning Adversarial Text Triggers In Diffusion Models Using Knowledge Distillation
SKD-CAG erases adversarial text triggers from diffusion models by distilling the model's own clean outputs through cross-attention guidance, claiming 100% and 93% removal for pixel and style backdoors.
-
TextPixs: Glyph-Conditioned Diffusion with Character-Aware Attention and OCR-Guided Supervision
The GCDA framework claims state-of-the-art text rendering in diffusion images via dual-stream encoding, attention segregation, and OCR supervision, but the paper lacks verifiable artifacts and contains internal incons...
-
Towards Evaluating Robustness of Prompt Adherence in Text to Image Models
New benchmark results show that Stable Diffusion 3.x and Janus Pro models struggle to place simple geometric shapes in the correct image quadrant, with best F1 scores around 0.41 to 0.5.
-
Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models
Current image-text alignment metrics, including CLIPScore and DSGScore, produce unstable model rankings under random seeds and are highly sensitive to tiny image perturbations.
Discussion (0). Continue with ORCID to comment.