Follow the Flow: On Information Flow Across Textual Tokens in Text-to-Image Models

Guy Kaplan; Michael Toker; Roy Schwartz; Yonatan Belinkov; Yuval Reif

arxiv: 2504.01137 · v3 · submitted 2025-04-01 · 💻 cs.CL

Follow the Flow: On Information Flow Across Textual Tokens in Text-to-Image Models

Guy Kaplan , Michael Toker , Yuval Reif , Yonatan Belinkov , Roy Schwartz This is my paper

Pith reviewed 2026-05-22 21:30 UTC · model grok-4.3

classification 💻 cs.CL

keywords text-to-image modelsinformation flowtoken representationsprompt alignmenttext encoderpatching techniquessemantic concentration

0 comments

The pith

Semantic information in text prompts for image generation concentrates in one or two tokens per item, and editing those tokens early fixes many alignment errors.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tracks how meaning moves through the tokens of a text prompt inside text-to-image models. It shows that a lexical item usually stores its full visual meaning in just one or two of its tokens, so the rest can be dropped. Tokens from separate items mostly stay isolated, but when they do mix they sometimes create wrong concepts that hurt the final image. The authors test simple changes to the token representations right after encoding and find these changes raise how closely the output image matches the prompt. This shifts attention from the diffusion steps to the text encoder as a practical place to correct generation problems.

Core claim

Patching experiments on the text encoder reveal that information about each lexical item is typically concentrated in one or two of its tokens, as in the case where the token 'Gate' alone suffices for the full expression 'San Francisco's Golden Gate Bridge'. Lexical items generally remain separate from one another, yet contextual mixing occasionally produces misrepresentations such as 'pool' becoming 'pool table' in the prompt 'a pool by a table'. Direct interventions that alter token representations at the encoding stage measurably improve prompt-image alignment and overall generation quality.

What carries the argument

Patching techniques that isolate and replace individual token representations in the text encoder to measure their downstream effect on image generation.

If this is right

Most tokens in a typical prompt can be ignored or heavily down-weighted with little loss to the generated image.
Cross-item mixing explains some common misalignments and can be corrected by targeted token edits.
Alignment quality can be raised by changes made before the diffusion process begins rather than only during it.
Token-level encoding patterns determine whether objects and relations in the prompt appear correctly in the output.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same concentration pattern may appear in other text-conditioned generation systems and could be exploited for shorter prompts.
Automated detection of harmful cross-item mixing could allow real-time correction of prompts before diffusion starts.
Pruning low-information tokens early might reduce compute while preserving output quality.

Load-bearing premise

Patching techniques accurately isolate the contribution of each token without introducing unintended changes to the original information flow.

What would settle it

Run the model on 'San Francisco's Golden Gate Bridge' and check whether zeroing out all tokens except 'Gate' still produces an image containing the bridge while zeroing out 'Gate' alone destroys the bridge; the opposite outcome would falsify the concentration claim.

read the original abstract

Text-to-image generation models suffer from alignment problems, where generated images fail to accurately capture the objects and relations in the text prompt. Prior work has focused on improving alignment by refining the diffusion process, ignoring the role of the text encoder, which guides the diffusion. In this work, we investigate how semantic information is distributed across token representations in text-to-image prompts, analyzing it at two levels: (1) in-item representation-whether individual tokens represent their lexical item (i.e., a word or expression conveying a single concept), and (2) cross-item interaction-whether information flows between tokens of different lexical items. We use patching techniques to uncover encoding patterns, and find that information is usually concentrated in only one or two of the item's tokens; for example, in the item ``San Francisco's Golden Gate Bridge'', the token ``Gate'' sufficiently captures the entire expression while the other tokens could effectively be discarded. Lexical items also tend to remain isolated; for instance, in the prompt ``a green dog'', the token ``dog'' encodes no visual information about ``green''. However, in some cases, items do influence each other's representation, often leading to misinterpretations-e.g., in the prompt ``a pool by a table'', the token ``pool'' represents a ``pool table'' after contextualization. Our findings highlight the critical role of token-level encoding in image generation, and demonstrate that simple interventions at the encoding stage can substantially improve alignment and generation quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Patching shows token-level concentration in T2I text encoders but the intervention likely perturbs attention and needs controls.

read the letter

The main thing to know is that this paper uses patching on the text encoder to argue semantic info for a lexical item usually sits in one or two tokens, with items mostly isolated from each other except in cases that produce misalignments like pool becoming pool table. They claim simple encoding-stage changes can fix some generation problems. The examples are concrete and the isolation pattern is presented clearly enough to be useful for thinking about prompt handling. What is new is the direct application of patching to map token contributions inside the text encoder of diffusion models, an angle earlier alignment work mostly left alone while focusing on the diffusion steps themselves. The paper earns credit for highlighting that the encoder is not a neutral conduit and for showing specific misinterpretation cases that trace back to contextualization. The soft spot is the patching method. Replacing one token's hidden state necessarily changes the attention computations for every other token in the same and later layers, so the observed concentration and the reported isolation could be created by the patch rather than reflecting the original flow. The stress-test concern lands here because the abstract gives no indication of controls that would separate the intervention effect from the model's natural behavior. The work stays qualitative on the patterns and does not appear to include quantitative metrics or comparisons to unmodified runs, which leaves the central claims only moderately supported. This is for people working on interpretability or alignment fixes in text-to-image systems who want token-level diagnostics. A reader already familiar with activation patching will see the extension but will also see the methodological gap. It deserves peer review so referees can ask for those controls and metrics.

Referee Report

2 major / 1 minor

Summary. The manuscript investigates semantic information flow in the text encoders of text-to-image models at the in-item and cross-item levels. Using patching experiments on prompts such as 'San Francisco's Golden Gate Bridge', 'a green dog', and 'a pool by a table', it claims that information is typically concentrated in one or two tokens per lexical item, that items are generally isolated from one another, and that targeted interventions at the encoding stage can improve alignment and generation quality.

Significance. If the patching results hold without intervention artifacts, the work would usefully shift focus from diffusion-process fixes to text-encoder token representations, offering both diagnostic insight and simple practical interventions. The illustrative examples and the reported qualitative improvements constitute a strength.

major comments (2)

[Methods] Methods / patching experiments: the central claim that patching isolates the true contribution of individual tokens (and thereby reveals concentration and isolation) is load-bearing, yet the manuscript provides no controls or analysis for the fact that replacing one hidden state necessarily perturbs attention keys/queries for all other tokens in the same and subsequent layers of the text encoder.
[Results] Results section: all reported patterns are qualitative with no quantitative metrics, ablation studies on patch count, or statistical controls, so the generality of the 'one or two tokens' concentration claim and the cross-item isolation claim cannot be assessed from the presented evidence.

minor comments (1)

[Abstract] The abstract states that 'simple interventions at the encoding stage can substantially improve alignment' but does not name or locate the specific interventions; a dedicated subsection describing them with before/after examples would improve clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and indicate planned revisions to strengthen the manuscript.

read point-by-point responses

Referee: [Methods] Methods / patching experiments: the central claim that patching isolates the true contribution of individual tokens (and thereby reveals concentration and isolation) is load-bearing, yet the manuscript provides no controls or analysis for the fact that replacing one hidden state necessarily perturbs attention keys/queries for all other tokens in the same and subsequent layers of the text encoder.

Authors: We acknowledge this methodological point. Patching necessarily alters attention computations for other tokens, and the original manuscript did not include explicit controls or analysis of these secondary effects. Our approach follows standard patching protocols from interpretability literature, measuring net impact on final representations and generated images. To address the concern directly, we will add an analysis of attention weight changes before and after patching in the revised manuscript, which will help quantify any artifacts and support the validity of the concentration and isolation observations. revision: yes
Referee: [Results] Results section: all reported patterns are qualitative with no quantitative metrics, ablation studies on patch count, or statistical controls, so the generality of the 'one or two tokens' concentration claim and the cross-item isolation claim cannot be assessed from the presented evidence.

Authors: The referee is correct that the presented results rely on qualitative examples. These were selected to clearly illustrate consistent patterns observed during our experiments. To improve assessment of generality, the revised manuscript will incorporate quantitative metrics, such as the fraction of lexical items where one or two tokens capture full semantics across a larger prompt set, ablations varying the number of patches, and basic statistical summaries of the observed patterns. revision: yes

Circularity Check

0 steps flagged

No circularity: claims rest on empirical patching observations

full rationale

The paper presents an empirical investigation using patching techniques to analyze token representations in text encoders of text-to-image models. Its central claims about information concentration in one or two tokens per lexical item and limited cross-item interactions are stated as direct observations from these interventions (e.g., replacing hidden states and measuring effects on generation). No mathematical derivation chain, equations, fitted parameters renamed as predictions, or self-citation load-bearing premises appear in the abstract or described methodology. The work does not invoke uniqueness theorems, smuggle ansatzes via citations, or rename known results; it reports experimental patterns without reducing them to inputs by construction. The analysis is therefore self-contained against external benchmarks of patching-based interpretability.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The analysis depends on the domain assumption that activation patching reveals causal information flow in the text encoder without substantial side effects from the intervention.

axioms (1)

domain assumption Patching interventions can isolate the contribution of individual token representations to the final image output.
Invoked when using patching to uncover encoding patterns and information flow.

pith-pipeline@v0.9.0 · 5809 in / 1205 out tokens · 32905 ms · 2026-05-22T21:30:13.138591+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We use patching techniques to uncover encoding patterns, and find that information is usually concentrated in only one or two of the item’s tokens
IndisputableMonolith/Foundation/AbsoluteFloorClosure.lean absolute_floor_iff_bare_distinguishability unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Lexical items also tend to remain isolated; for instance, in the prompt “a green dog”, the token “dog” encodes no visual information about “green”

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Vision-Language Binding in In-Context Image Generation
cs.CV 2026-05 unverdicted novelty 7.0

Text tokens in FLUX.2 absorb reference image properties like color and style to influence outputs while pixel-exact details bypass them, localized to padding tokens via causal interventions.