Pith. sign in

REVIEW 2 cited by

DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2210.10606 v1 pith:7OIJ3CR2 submitted 2022-10-19 cs.CL cs.LG

Royi Rassin , Shauli Ravfogel , Yoav Goldberg This is my paper · ORCID
classification cs.CLcs.LG
keywords dalle-2entitiespropertieshumanimagelanguagemodelsmodify
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We study the way DALLE-2 maps symbols (words) in the prompt to their references (entities or properties of entities in the generated image). We show that in stark contrast to the way human process language, DALLE-2 does not follow the constraint that each word has a single role in the interpretation, and sometimes re-use the same symbol for different purposes. We collect a set of stimuli that reflect the phenomenon: we show that DALLE-2 depicts both senses of nouns with multiple senses at once; and that a given word can modify the properties of two distinct entities in the image, or can be depicted as one object and also modify the properties of another object, creating a semantic leakage of properties between entities. Taken together, our study highlights the differences between DALLE-2 and human language processing and opens an avenue for future study on the inductive biases of text-to-image models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EvalGIM: A Library for Evaluating Generative Image Models

    cs.CV 2024-12 conditional novelty 6.0 of 10

    EvalGIM packages text-to-image evaluation into a single extensible library with four 'Evaluation Exercises', two of which introduce new analysis methods for ranking robustness and balanced prompt-style comparisons.

  2. Scaling Down Semantic Leakage: Investigating Associative Bias in Smaller Language Models

    cs.CL 2025-01 conditional novelty 5.0 of 10

    Tests four Qwen2.5 models and finds the 0.5B model shows the least semantic leakage, but the size trend is non-monotonic.

Pith tools