Pith. sign in

REVIEW 4 major objections 9 minor 76 references

Improving Multi-Subject Consistency in Open-Domain Image Generation with Isolation and Reposition Attention

T0 review · 4 major / 9 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training-free image generators can keep multiple subjects from merging into one character by masking their mutual attention, a new method claims.

desk verdict Simple, effective training-free attention modifications with real gains, but the mechanistic explanation is correlational and the evaluation needs error bars and a clearer retry policy. read the letter →

arxiv 2411.19261 v2 pith:CGAWRI5O submitted 2024-11-28 cs.CV

Huiguo He , Qiuyue Wang , Yuan Zhou , Yuxuan Cai , Hongyang Chao , Jian Yin , Huan Yang This is my paper · ORCID
classification cs.CV
keywords multi-subjectconsistencytraining-freediffusionself-attentionmaskingsubjectconvergencerepositionattentionopen-domainimagegenerationanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Training-free multi-subject image generators fuse reference characters into one scene through attention layers, but they still mix subjects together or copy features from the wrong location. This paper claims that two attention behaviors cause the failures: subjects inside the target image attract each other and collapse into a hybrid character, and tokens prefer nearby information, so a reference subject placed far from its target position is poorly reused. The proposed fix, IR-Diffusion, is a plug-in that masks out the attention edges between different subjects in the target image and rescales and shifts each reference subject's features to the target subject's location before computing attention. On the DS-500 benchmark this raises the multi-subject consistency score by about 18.6% for two-subject scenes and 75.4% for three-subject scenes relative to DreamStory, without finetuning or retraining.

What carries the argument

Isolation Attention and Reposition Attention. Isolation Attention is an attention-mask operation that, for each subject's query, zeroes out responses from the keys and values of the other subjects in the target image, leaving only the subject's own tokens, the background, and its own reference features; it is computed as a product of pairwise mask complements and applied to the self-attention logits. Reposition Attention rescales and shifts the reference subject's key/value features so they occupy the same spatial position as the target subject, then fills the outside region with zeros, so that nearby-token bias works with the reference instead of against it. Together they are inserted into the self-attention layers of a U-Net diffusion backbone with a masking mechanism.

What would settle it

Run the same IR-Diffusion pipeline but replace Isolation Attention with masks over the same number of randomly chosen cross-subject attention entries; if consistency stays at the improved level, the isolation explanation is not doing the work. Similarly, apply Reposition Attention with reference features shifted to random positions rather than the target subject's; if consistency remains high, the spatial-alignment explanation is unsupported.

Watch

Extended reading notes

Core claim

The central claim is that the residual inconsistency of training-free multi-subject generators is caused by two measurable, intrinsic properties of self-attention: an internal attraction among the subjects of the target image, shown by higher mean attention responses between subjects than toward background (Table 1), and a distance-decay bias in which tokens attend more strongly to nearby tokens (Figure 4). Removing the attention edges between subjects, the Isolation Attention mask, prevents the convergence of multiple subjects into one composite entity, and spatially aligning the reference features to the target positions, Reposition Attention, restores the effective use of reference information. The paper argues that these two operations, applied as masks on top of the DreamStory pipeline, are sufficient to reach state-of-the-art multi-subject consistency in open-domain generation.

Load-bearing premise

The whole repair rests on treating the measured attention patterns as the cause: that the higher response between subjects and the fall-off of attention with distance are why subjects merge and misalign, so deleting exactly those attention edges is the correct fix.

Editorial extensions

If this is right

  • If correct, multi-subject consistency improves without finetuning, datasets, or additional training: the whole system remains training-free and plugs into existing pipelines.
  • The gains grow with subject count: three-subject scenes improve more than two-subject scenes, consistent with the claim that internal attraction and misalignment accumulate as more subjects compete for attention.
  • The two operations are backbone-agnostic: applying the same masks to SDXL, Playground, and Kolors improves consistency across all three (supplementary Table 6).
  • The diagnostic claims (subject response exceeds background; attention decays with distance) give practitioners a direct measurement to detect when a generator will suffer subject fusion or misalignment.
  • The approach transfers to other attention-based generators: the authors point to DiT architectures such as FLUX and PixArt-Σ, and to applications in attribute binding and video generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the correlation shown in Table 1 and Figure 4 does not by itself prove causality; the gains could partly come from a generic sparsification of the attention matrix. A control that masks the same number of randomly chosen cross-subject edges would separate the two explanations.
  • Editorial inference: the repositioning idea is a special case of a broader principle — reference conditioning is stronger when reference and target feature layouts are geometrically aligned — so other warpings (affine, optical-flow based, or learned) might generalize the same benefit beyond translate-and-scale.
  • Editorial inference: because the method uses rehearsal-generated masks that downscale with the VAE, very small subject masks are a stated failure mode; upsampling masks to keep small subjects at token-level resolution is a natural extension suggested by the paper's own limitation discussion.
  • Editorial inference: the same two mechanisms might apply to single-subject attribute binding, where an object's features bleed into a second object in the same image; masking cross-object attention could be tested as a direct transfer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 9 minor

Summary. This paper proposes IR-Diffusion for open-domain multi-subject consistent image generation, building on the training-free DreamStory pipeline. It diagnoses two failure modes in self-attention: internal attraction between different subjects of the target image, claimed to cause subject convergence, and a preference for spatially proximate tokens, claimed to reduce the usefulness of reference features when reference and target subject positions differ. The proposed fixes are Isolation Attention (IA), which masks attention from each subject's queries to other subjects' keys and values (Eq. 7), and Reposition Attention (RA), which rescales and shifts reference keys and values to align with target subject positions (Eq. 8). On the DS-500 benchmark, IR-Diffusion reports D&C-DS of 0.6458 (2-subject) and 0.4095 (3-subject), versus 0.5444 and 0.2335 for DreamStory, with ablations (Tab. 3), a user study (Tabs. 4 and 5), cross-backbone experiments (Tab. 6), and runtime analysis (Tab. 7). The supplementary material acknowledges failure cases with small subject masks (Sec. C.3) and reports an overhead of roughly 45--57% over the baseline.

Significance. The engineering result is valuable if reproducible: the method is training-free and plug-and-play, the two modules show consistent though unequal gains, the user study points in the same direction as the objective metrics (e.g., 66.6% vs 4.9% consistency preference over DreamStory, Tab. 4), and the gains generalize across three backbones. The paper is honest about failure cases and overhead. However, the scientific claim that the paper reveals internal attraction and positional bias as the causal mechanisms is not established: Table 1 and Fig. 4 are correlational with known confounds, and the interventions in Eqs. (7)--(8) bundle several attention changes at once. The stress-test concern that the causal story is underdetermined lands, on my reading. In addition, no code or data release is indicated, objective metrics lack variance and significance tests, and the DS-500 benchmark and DreamStory baseline both come from the authors' own prior work. The method is credible; the diagnosis needs stronger controls or a softened claim. No circularity is involved: the gains are measured against an external benchmark and published baselines.

major comments (4)
  1. [Sec. 3.2.1 / Tab. 1] Section 3.2.1 presents Table 1 as strong evidence of internal attraction between subjects, but the aggregate per-token response comparison of 'other subjects' versus 'background' does not control for spatial distance or region geometry. Given the distance-response decay the paper itself reports in Fig. 4, subject regions placed near the query subject will show elevated responses from proximity alone; the gap (0.1168 vs 0.1018 in the 2-subject case) is also reported without any null model, variance, or per-case breakdown. Please add a matched-distance control (e.g., other-subject versus background tokens at the same distance bin) and per-case distributions. As it stands, the evidence supports only a correlation, and the abstract's claim that IA removes a discovered 'internal attraction' mechanism is not established.
  2. [Sec. 3.2.2 / Eq. (7)] The IA operation in Eq. (7) simultaneously zeros cross-subject K/V edges, reallocates softmax mass to the remaining reference, self, and background tokens, and removes any legitimate semantic interaction between subjects (mutual gaze, spatial relations such as 'beside' or 'watching'). The ablation gain (Tab. 3, 3-subject D&C-DS 0.2378 to 0.3636) may therefore partly come from reweighting of the reference branch rather than from eliminating a harmful attraction. Please add a control intervention, such as masking an equal number of randomly chosen tokens or attenuating (rather than zeroing) cross-subject attention, to show that the specific structure of the mask drives the improvement. Without such a control, the mechanism attributed to IA remains undetermined.
  3. [Sec. 3.3.1 / Fig. 4 and Tab. 3 (RA row)] The distance-response decay in Fig. 4 does not by itself establish a learned positional prior, because nearby tokens in natural images usually belong to the same object or texture; the decay is equally consistent with semantic grouping. A control matching same-object versus different-object token pairs at equal distances would be needed. The improvement data also give only weak support to the claimed positional mechanism: the RA-only ablation changes 3-subject D&C-DS from 0.2378 to 0.2405 (+1.1%) in Tab. 3, and the 2-subject RA-only gain (0.5592 to 0.5978) is reported without variance. The main benefit of RA appears as an interaction with IA; please report variance or confidence intervals for these small differences and state whether the RA-only effect is statistically meaningful.
  4. [Sec. 4.3 / Tabs. 2 and 3] Objective metrics in Tables 2, 3, and 6 are single numbers with no variance, no number of independent runs, and no significance tests, although the text repeatedly concludes that IR-Diffusion significantly enhances consistency. The headline 3-subject improvements are large enough that the qualitative conclusion is likely robust, but the smaller claims, notably the RA-only gain and several 2-subject differences, cannot be evaluated as reported. Please report run-to-run variance (e.g., over seeds) or equivalent statistical evidence for all reported differences, and reserve 'significantly' for comparisons where such evidence exists.
minor comments (9)
  1. [Sec. 3.2.1 / Tab. 1] The 2-subject background response is 0.1018 in Table 1 but is stated as 0.1037 in the text of Sec. 3.2.1; please reconcile the numbers.
  2. [Tab. 2 / Sec. 1 / Supp. B.2 / author block] Typos: 'RI-Diffusion' in the Table 2 header, 'IR-Diffusuion' in the contribution list, 'attration' in Supplementary Sec. B.2, and 'Sun Yat-Sun University' in the affiliation block.
  3. [Sec. B.4] The maximum-20-attempt retry used to obtain accurate subject masks is an engineering choice that could affect fairness across methods; please state in the main paper whether the same retry was applied to all baselines on DS-500 and what the retry is conditioned on (presumably the number of subjects stated in the prompt).
  4. [Abstract / Sec. 4.4] No code or data release is mentioned; since the method is training-free and DS-500 is public, a code release statement would substantially support reproducibility of the reported gains.
  5. [Fig. 4] The two curves labeled Scale 1 and Scale 2 are not defined in terms of U-Net resolution, layer range, or aggregation across timesteps; please specify which layers produce each curve and whether the distance trend holds at the per-layer level rather than only on average.
  6. [Sec. C.3 / Fig. 7] The acknowledged small-mask limitation is reasonable and honestly presented; because it qualifies the open-domain claim in the abstract, I suggest moving this limitation (with Fig. 7) from the supplementary material into the main paper.
  7. [Sec. 3.2.2 / Eq. (7)] Equation (7) and the surrounding prose omit the mask term that actually implements isolation; the mask construction appears only in Supplementary Eq. (14). A compact version of the mask formula should appear in the main text so the method is self-contained.
  8. [Sec. 4.3] The text says the authors 'evaluate generated results based on three main criteria' but then lists four metrics (AES, CLIP-T, DS, D&C-DS); please state the mapping of the four metrics to the three criteria.
  9. [Sec. 4.3 / Tabs. 4 and 5] The user-study protocol includes a 'comparable' option, but Tables 4 and 5 report only Win and Lose percentages, which do not sum to 100; please report the third category or state explicitly why it is omitted.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation found: Isolation and Reposition Attention are explicit attention-mask modifications validated against external benchmarks; the causal-mechanism narrative is interpretive, not circular.

full rationale

The paper's claimed derivation chain is not circular. Isolation Attention (Eq. 7) and Reposition Attention (Eq. 8) are explicit modifications of the attention operator: IA replaces the all-ones target mask in the DreamStory formulation (Eq. 12) with the isolation mask M_TGT (Eq. 14), and RA resamples reference K/V features to target-subject positions. These are algorithms, not fitted parameters, and their reported effect (D&C-DS 0.2378 -> 0.4095 on the 3-subject DS-500 benchmark, Tab. 3) is evaluated with external metrics (DreamSim-based DS and D&C-DS) against published baselines. The evidence for "internal attraction" (Tab. 1) and positional decay (Fig. 4) is observational and could be confounded, but that is an interpretive weakness in the causal narrative, not a case where a prediction is equal to its input by construction. The paper repeatedly builds on the authors' own DreamStory [16] as baseline and backbone; this is a self-citation, but it is not load-bearing because the new masks are ablated against DreamStory and measured on an external benchmark rather than established by citation. The acknowledged limitation (small masks, Sec. C.3) is a boundary condition, not a circular step. Overall: no significant circularity; score reflects a minor, non-load-bearing reliance on the authors' prior work.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method contributes no fitted constants, but it relies on several unverified premises: the accuracy of GroundedSAM masks, the causal interpretation of attention statistics, and the assumption that affine repositioning of reference features is harmless in pretrained attention. The retry limit and dropout rate are hand-chosen engineering values.

free parameters (4)
  • maximum retry attempts = 20
    Supplementary B.4: retries to get accurate subject masks; average 4.97 attempts on the 3-subject benchmark, tuned to balance mask accuracy and runtime.
  • token dropout rate = 0.5
    Supplementary B.4: increases diversity of subject poses; inherited from prior work [16,61,73].
  • guidance scale = 7.0
    Supplementary B.4: standard classifier-free guidance strength.
  • inference steps = 50
    Supplementary B.4: default scheduler steps.
assumptions (3)
  • domain assumption GroundedSAM segmentation provides correct subject masks.
    Used to measure internal attraction (Sec 3.2.1) and define isolation and reposition masks; errors would mis-target the attention modifications.
  • domain assumption Attention response values quantify subject information flow and subject fusion.
    Tab 1 and Fig 4 are interpreted as causal evidence; the paper provides no control to show that higher response causes blending.
  • domain assumption The pretrained model's positional bias can be exploited by affine repositioning of reference features without retraining.
    RA assumes that moving K and V features to the target position makes the model use them better, while zero-filled surrounding regions do not disturb generation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Multi-Subject Consistency in Open-Domain Image Generation with Isolation and Reposition Attention." pith.science (2026). https://pith.science/paper/CGAWRI5O

@misc{pith2026241119261,
  author       = {Pith},
  title        = {Pith review of: Improving Multi-Subject Consistency in Open-Domain Image Generation with Isolation and Reposition Attention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CGAWRI5O}},
  note         = {Machine review of arXiv:2411.19261}
}
read the original abstract

Training-free diffusion models have achieved remarkable progress in generating multi-subject consistent images within open-domain scenarios. The key idea of these methods is to incorporate reference subject information within the attention layer. However, existing methods still obtain suboptimal performance when handling numerous subjects. This paper reveals two primary issues contributing to this deficiency. Firstly, the undesired internal attraction between different subjects within the target image can lead to the convergence of multiple subjects into a single entity. Secondly, tokens tend to reference nearby tokens, which reduces the effectiveness of the attention mechanism when there is a significant positional difference between subjects in reference and target images. To address these issues, we propose a training-free diffusion model with Isolation and Reposition Attention, named IR-Diffusion. Specifically, Isolation Attention ensures that multiple subjects in the target image do not reference each other, effectively eliminating the subject convergence. On the other hand, Reposition Attention involves scaling and repositioning subjects in both reference and target images to the same position within the images. This ensures that subjects in the target image can better reference those in the reference image, thereby maintaining better consistency. Extensive experiments demonstrate that IR-Diffusion significantly enhances multi-subject consistency, outperforming all existing methods in open-domain scenarios.

Figures

Figures reproduced from arXiv: 2411.19261 by the authors.

Figure 1
Figure 1. Illustration of our idea. (a) Internal attraction among [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of our IR-Diffusion: (a) Isolation Attention (IA): IA isolates internal attraction between different subjects by ensuring [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of the overall Self-Attention mechanism [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Average response values between tokens at varying [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Comparisons of multi-subject consistency generation between our IR-Diffusion and other SOTA methods. The superior [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Ablation studies of different generation results. All [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Failure cases in scenarios with small subject masks. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Comparisons of multi-subject consistency generation between our IR-Diffusion and other SOTA methods. The superior [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Comparisons of multi-subject consistency generation between our IR-Diffusion and other SOTA methods. The superior [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Comparisons of anime style generation between our IR-Diffusion and MS-Diffusion [ [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Comparisons of anime style generation between our IR-Diffusion, MS-Diffusion [ [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Generated images from our IR-Diffusion. Real-style images are displayed on the left, and anime-style images on the right. Each [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Ablation studies of different generation results. All methods use the top portraits as a reference. Different subjects are indicated [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Mutual attention heatmap visualization for the ablation study. With the integration of our IR-Diffusion, reference image features [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

76 extracted references · 58 canonical work pages

  1. [1]

    Moab Arar, Rinon Gal, Yuval Atzmon, Gal Chechik, Daniel Cohen-Or, Ariel Shamir, and Amit H. Bermano. Domain- agnostic tuning-encoder for fast personalization of text-to- image models. In SIGGRAPH Asia, pages 1–10, 2023. 3

    2023

  2. [2]

    Wasser- stein generative adversarial networks

    Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasser- stein generative adversarial networks. In ICML, pages 214– 223, 2017. 2

    work page 2017

  3. [3]

    The chosen one: Consistent characters in text-to-image diffusion models

    Omri Avrahami, Amir Hertz, Yael Vinker, Moab Arar, Shlomi Fruchter, Ohad Fried, Daniel Cohen-Or, and Dani Lischin- ski. The chosen one: Consistent characters in text-to-image diffusion models. In SIGGRAPH, pages 1–12, 2024. 2

    work page 2024

  4. [4]

    MasaCtrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. MasaCtrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In ICCV, pages 22560–22570, 2023. 1, 3, 12

    work page 2023

  5. [5]

    Character-centric story visualization via visual planning and token alignment

    Hong Chen, Rujun Han, Te-Lin Wu, Hideki Nakayama, and Nanyun Peng. Character-centric story visualization via visual planning and token alignment. In EMNLP, pages 8259–8272,

    work page

  6. [6]

    PixArt- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. PixArt- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023. 9

    arXiv

  7. [7]

    AnyDoor: Zero-shot object-level image customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. AnyDoor: Zero-shot object-level image customization. In CVPR, pages 6593–6602, 2024. 3

    work page 2024

  8. [8]

    Au- toStudio: Crafting consistent subjects in multi-turn interactive image generation

    Junhao Cheng, Xi Lu, Hanhui Li, Khun Loun Zai, Baiqiao Yin, Yuhao Cheng, Yiqiang Yan, and Xiaodan Liang. Au- toStudio: Crafting consistent subjects in multi-turn interactive image generation. arXiv preprint arXiv:2406.01388, 2024. 3

    arXiv 2024

Show all 76 references
  1. [9]

    TheaterGen: Character management with llm for consistent multi-turn image generation

    Junhao Cheng, Baiqiao Yin, Kaixin Cai, Minbin Huang, Hanhui Li, Yuxin He, Xi Lu, Yue Li, Yifei Li, Yuhao Cheng, et al. TheaterGen: Character management with llm for consistent multi-turn image generation. arXiv preprint arXiv:2404.18919, 2024. 3

    2024 arXiv

  2. [10]

    IDAdapter: Learning mixed features for tuning-free personalization of text-to-image mod- els

    Siying Cui, Jia Guo, Xiang An, Jiankang Deng, Yongle Zhao, Xinyu Wei, and Ziyong Feng. IDAdapter: Learning mixed features for tuning-free personalization of text-to-image mod- els. In CVPR Workshops, pages 950–959, 2024. 3

    2024

  3. [11]

    DreamSim: Learning new dimensions of human visual similarity using synthetic data

    Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. DreamSim: Learning new dimensions of human visual similarity using synthetic data. NeurIPS, 36, 2024. 7

    2024

  4. [12]

    TeViS: Translating text synopses to video storyboards

    Xu Gu, Yuchong Sun, Feiyue Ni, Shizhe Chen, Xihua Wang, Ruihua Song, Boyuan Li, and Xiang Cao. TeViS: Translating text synopses to video storyboards. In ACM MM, pages 4968–4979, 2023. 12

    2023

  5. [13]

    Improved training of Wasserstein GANs

    Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of Wasserstein GANs. NeurIPS, 30, 2017. 2

    2017

  6. [14]

    Imagine this! scripts to composi- tions to videos

    Tanmay Gupta, Dustin Schwenk, Ali Farhadi, Derek Hoiem, and Aniruddha Kembhavi. Imagine this! scripts to composi- tions to videos. In ECCV, pages 598–613, 2018. 12

    2018

  7. [15]

    Learning profitable NFT image diffusions via multiple visual- policy guided reinforcement learning

    Huiguo He, Tianfu Wang, Huan Yang, Jianlong Fu, Nicholas Jing Yuan, Jian Yin, Hongyang Chao, and Qi Zhang. Learning profitable NFT image diffusions via multiple visual- policy guided reinforcement learning. In ACM MM, pages 6831–6840, 2023. 2

    2023

  8. [16]

    DreamStory: Open-domain story visualization by LLM-guided multi-subject consistent diffusion, 2024

    Huiguo He, Huan Yang, Zixi Tuo, Yuan Zhou, Qiuyue Wang, Yuhang Zhang, Zeyu Liu, Wenhao Huang, Hongyang Chao, and Jian Yin. DreamStory: Open-domain story visualization by LLM-guided multi-subject consistent diffusion, 2024. 1, 2, 3, 4, 5, 6, 7, 8, 12, 13, 14

    2024

  9. [17]

    Rotary position embedding for vision transformer

    Byeongho Heo, Song Park, Dongyoon Han, and Sangdoo Yun. Rotary position embedding for vision transformer. In European Conference on Computer Vision, pages 289–305. Springer, 2024. 6

    2024

  10. [18]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS Workshop, 2021. 13

    2021

  11. [19]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. NeurIPS, 33:6840–6851, 2020. 2

    2020

  12. [20]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. LoRA: Low-rank adaptation of large language models. In ICLR, 2021. 2

    2021

  13. [21]

    How much po- sition information do convolutional neural networks encode? In ICLR, 2020

    Md Amirul Islam, Sen Jia, and Neil DB Bruce. How much po- sition information do convolutional neural networks encode? In ICLR, 2020. 5

    2020

  14. [22]

    Position, padding and predic- tions: A deeper look at position information in cnns

    Md Amirul Islam, Matthew Kowal, Sen Jia, Konstantinos G Derpanis, and Neil DB Bruce. Position, padding and predic- tions: A deeper look at position information in cnns. IJCV, pages 1–22, 2024. 5

    2024

  15. [23]

    Identity decoupling for multi-subject personalization of text- to-image models, 2024

    Sangwon Jang, Jaehyeong Jo, Kimin Lee, and Sung Ju Hwang. Identity decoupling for multi-subject personalization of text- to-image models, 2024. 2, 5, 7, 8, 14

    2024

  16. [24]

    InstantFamily: Masked attention for zero-shot multi-id image generation

    Chanran Kim, Jeongin Lee, Shichang Joung, Bongmo Kim, and Yeul-Min Baek. InstantFamily: Masked attention for zero-shot multi-id image generation. arXiv preprint arXiv:2404.19427, 2024. 3

    2024 arXiv

  17. [25]

    Auto-encoding varia- tional bayes

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. arXiv preprint arXiv:1312.6114, 2013. 2, 14

    2013 arXiv

  18. [26]

    OMG: Occlusion-friendly personalized multi-concept generation in diffusion models

    Zhe Kong, Yong Zhang, Tianyu Yang, Tao Wang, Kaihao Zhang, Bizhu Wu, Guanying Chen, Wei Liu, and Wenhan Luo. OMG: Occlusion-friendly personalized multi-concept generation in diffusion models. In ECCV, pages 253–270. Springer, Springer, 2024. 2

    2024

  19. [27]

    Black Forest Labs. FLUX. https://github.com/ black-forest-labs/flux, 2023. 9

    2023

  20. [28]

    Direct consistency optimization for compositional text- to-image personalization

    Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn, and Jinwoo Shin. Direct consistency optimization for compositional text- to-image personalization. arXiv preprint arXiv:2402.12004,

    arXiv

  21. [29]

    Playground v2.5: Three insights to- wards enhancing aesthetic quality in text-to-image generation,

    Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2.5: Three insights to- wards enhancing aesthetic quality in text-to-image generation,

  22. [30]

    BLIP-diffusion: Pre- trained subject representation for controllable text-to-image generation and editing

    Dongxu Li, Junnan Li, and Steven Hoi. BLIP-diffusion: Pre- trained subject representation for controllable text-to-image generation and editing. NeurIPS, 36, 2024. 3

    2024

  23. [31]

    BLIP: bootstrapping language-image pre-training for unified vision- language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: bootstrapping language-image pre-training for unified vision- language understanding and generation. In ICML, pages 12888–12900. PMLR, 2022. 12

    2022

  24. [32]

    BLIP- 2: bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP- 2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, pages 19730–19742. PMLR, 2023. 12

    2023

  25. [33]

    StoryGAN: A sequential conditional gan for story visu- alization

    Yitong Li, Zhe Gan, Yelong Shen, Jingjing Liu, Yu Cheng, Yuexin Wu, Lawrence Carin, David Carlson, and Jianfeng Gao. StoryGAN: A sequential conditional gan for story visu- alization. In CVPR, 2019. 2, 12

    2019

  26. [34]

    PhotoMaker: Customizing realistic human photos via stacked id embedding

    Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming- Ming Cheng, and Ying Shan. PhotoMaker: Customizing realistic human photos via stacked id embedding. In CVPR, pages 8640–8650, 2024. 3

    2024

  27. [35]

    Unveiling the mask of position-information pattern through the mist of im- age features

    Chieh Hubert Lin, Hung-Yu Tseng, Hsin-Ying Lee, Ma- neesh Kumar Singh, and Ming-Hsuan Yang. Unveiling the mask of position-information pattern through the mist of im- age features. In ICML. JMLR.org, 2023. 5

    2023

  28. [36]

    Intelligent grimm-open-ended visual storytelling via latent diffusion models

    Chang Liu, Haoning Wu, Yujie Zhong, Xiaoyun Zhang, Yan- feng Wang, and Weidi Xie. Intelligent grimm-open-ended visual storytelling via latent diffusion models. In CVPR, pages 6190–6200, 2024. 12

    2024

  29. [37]

    One-Prompt-One-Story: Free-lunch consistent text-to-image generation using a single prompt

    Tao Liu, Kai Wang, Senmao Li, Joost van de Weijer, Fa- had Shahbaz Khan, Shiqi Yang, Yaxing Wang, Jian Yang, and Ming-Ming Cheng. One-Prompt-One-Story: Free-lunch consistent text-to-image generation using a single prompt. In ICLR, 2025. 3

    2025

  30. [38]

    DPM-solver++: Fast solver for guided sampling of diffusion probabilistic models

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022. 13

    2022 arXiv

  31. [39]

    Subject- Diffusion: Open domain personalized text-to-image gener- ation without test-time fine-tuning

    Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject- Diffusion: Open domain personalized text-to-image gener- ation without test-time fine-tuning. In SIGGRAPH, pages 1–12, 2024. 3

    2024

  32. [40]

    AI illustrator: Translating raw descriptions into images by prompt-based cross-modal generation

    Yiyang Ma, Huan Yang, Bei Liu, Jianlong Fu, and Jiaying Liu. AI illustrator: Translating raw descriptions into images by prompt-based cross-modal generation. In ACM MM, pages 4282–4290, 2022. 2

    2022

  33. [41]

    Integrating visuospa- tial, linguistic, and commonsense structure into story visual- ization

    Adyasha Maharana and Mohit Bansal. Integrating visuospa- tial, linguistic, and commonsense structure into story visual- ization. In EMNLP, pages 6772–6786, 2021. 12

    2021

  34. [42]

    Im- proving generation and evaluation of visual stories via seman- tic consistency

    Adyasha Maharana, Darryl Hannan, and Mohit Bansal. Im- proving generation and evaluation of visual stories via seman- tic consistency. In NAACL HLT, pages 2427–2442, 2021

    2021

  35. [43]

    StoryDALL-E: Adapting pretrained text-to-image transform- ers for story continuation

    Adyasha Maharana, Darryl Hannan, and Mohit Bansal. StoryDALL-E: Adapting pretrained text-to-image transform- ers for story continuation. In ECCV, pages 70–87. Springer,

  36. [44]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, pages 8162–8171, 2021. 2

    2021

  37. [45]

    Synthesizing coherent story with auto-regressive latent diffusion models

    Xichen Pan, Pengda Qin, Yuhong Li, Hui Xue, and Wenhu Chen. Synthesizing coherent story with auto-regressive latent diffusion models. In W ACV, pages 2920–2930, 2024. 12

    2024

  38. [46]

    PortraitBooth: A versatile portrait model for fast identity-preserved personalization

    Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, and Rongrong Ji. PortraitBooth: A versatile portrait model for fast identity-preserved personalization. In CVPR, pages 27080–27090, 2024. 2

    2024

  39. [47]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 2, 4, 5, 12, 13

    2023 arXiv

  40. [48]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pages 8748–8763, 2021. 12

    2021

  41. [49]

    Make-a-Story: Visual memory conditioned consistent story generation

    Tanzila Rahman, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Shweta Mahajan, and Leonid Sigal. Make-a-Story: Visual memory conditioned consistent story generation. In CVPR, pages 2493–2502, 2023. 12

    2023

  42. [50]

    Grounded SAM: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded SAM: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159, 2024. 4

    2024 arXiv

  43. [51]

    Image-based video game asset generation and evaluation using deep learning: a systematic review of meth- ods and applications

    Rafael Ribeiro, Alexandre Valle de Carvalho, and Nelson Bil- ber Rodrigues. Image-based video game asset generation and evaluation using deep learning: a systematic review of meth- ods and applications. IEEE Transactions on Games, 2024. 1

    2024

  44. [52]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2, 4, 12

    2022

  45. [53]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, pages 234–241. Springer, 2015. 14

    2015

  46. [54]

    DreamBooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DreamBooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration. In CVPR, pages 22500–22510, 2023. 2

    2023

  47. [55]

    LAION-5B: An open large-scale dataset for training next gen- eration image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. LAION-5B: An open large-scale dataset for training next gen- eration image-text models. arXiv preprint arXiv:2210.08402,

    arXiv

  48. [56]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2020. 2

    2020

  49. [57]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2020. 2

    2020

  50. [58]

    Character-preserving coherent story visualization

    Yun-Zhu Song, Zhi Rui Tam, Hung-Jen Chen, Huiao-Han Lu, and Hong-Han Shuai. Character-preserving coherent story visualization. In ECCV, pages 18–33. Springer, 2020. 2

    2020

  51. [59]

    Create your world: Lifelong text-to- image diffusion

    Gan Sun, Wenqi Liang, Jiahua Dong, Jun Li, Zhengming Ding, and Yang Cong. Create your world: Lifelong text-to- image diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 2

    2024

  52. [60]

    Kolors: Effective training of diffusion model for photorealistic text-to-image synthesis

    Kolors Team. Kolors: Effective training of diffusion model for photorealistic text-to-image synthesis. arXiv preprint, 2024. 13

    2024

  53. [61]

    Training-free consistent text-to-image generation

    Yoad Tewel, Omri Kaduri, Rinon Gal, Yoni Kasten, Lior Wolf, Gal Chechik, and Yuval Atzmon. Training-free consistent text-to-image generation. TOG, 43(4):1–18, 2024. 1, 2, 3, 4, 5, 7, 8, 12, 13, 14

    2024

  54. [62]

    Storytelling and visualization: An extended survey

    Chao Tong, Richard Roberts, Rita Borgo, Sean Walton, Robert S Laramee, Kodzo Wegba, Aidong Lu, Yun Wang, Huamin Qu, Qiong Luo, et al. Storytelling and visualization: An extended survey. Information, 9(3):65, 2018. 1

    2018

  55. [63]

    OneActor: Consistent subject generation via cluster- conditioned guidance

    Jiahao Wang, Caixia Yan, Haonan Lin, Weizhan Zhang, Mengmeng Wang, Tieliang Gong, Guang Dai, and Hao Sun. OneActor: Consistent subject generation via cluster- conditioned guidance. NeurIPS, 37:21502–21536, 2025. 2

    2025

  56. [64]

    Videofactory: Swap atten- tion in spatiotemporal diffusions for text-to-video generation

    Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu. Videofactory: Swap atten- tion in spatiotemporal diffusions for text-to-video generation. arXiv preprint arXiv:2305.10874, 2023. 2

    2023 arXiv

  57. [65]

    MS-Diffusion: Multi-subject zero-shot image personalization with layout guidance

    Xierui Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. MS-Diffusion: Multi-subject zero-shot image personalization with layout guidance. In ICLR, 2025. 3, 5, 7, 8, 13, 14, 17, 18

    2025

  58. [66]

    High-fidelity person-centric subject-to-image synthesis

    Yibin Wang, Weizhong Zhang, Jianwei Zheng, and Cheng Jin. High-fidelity person-centric subject-to-image synthesis. In CVPR, pages 7675–7684, 2024. 3

    2024

  59. [67]

    IP- Adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. IP- Adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  60. [68]

    LaPE: Layer- adaptive position embedding for vision transformers with independent layer normalization

    Runyi Yu, Zhennan Wang, Yinhuai Wang, Kehan Li, Chang Liu, Haoyi Duan, Xiangyang Ji, and Jie Chen. LaPE: Layer- adaptive position embedding for vision transformers with independent layer normalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision,...

    2023

  61. [69]

    Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation

    Yu Zeng, Vishal M Patel, Haochen Wang, Xun Huang, Ting- Chun Wang, Ming-Yu Liu, and Yogesh Balaji. Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation. In CVPR, pages 6786–6795, 2024. 3

    2024

  62. [70]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, pages 3836–3847, 2023. 3

    2023

  63. [71]

    SSR-Encoder: Encoding selective subject representation for subject-driven generation

    Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. SSR-Encoder: Encoding selective subject representation for subject-driven generation. In CVPR, pages 8069–8078, 2024. 3

    2024

  64. [72]

    Pia: Your personalized image animator via plug-and-play modules in text-to-image models

    Yiming Zhang, Zhening Xing, Yanhong Zeng, Youqing Fang, and Kai Chen. Pia: Your personalized image animator via plug-and-play modules in text-to-image models. In CVPR, pages 7747–7756, 2024. 3

    2024

  65. [73]

    StoryDiffusion: Consistent self-attention for long-range image and video generation

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. StoryDiffusion: Consistent self-attention for long-range image and video generation. NeurIPS, 37: 110315–110340, 2025. 1, 3, 4, 5, 7, 8, 12, 13, 14

    2025

  66. [74]

    StoryMaker: Towards holistic consistent characters in text-to-image generation

    Zhengguang Zhou, Jing Li, Huaxia Li, Nemo Chen, and Xu Tang. StoryMaker: Towards holistic consistent characters in text-to-image generation. arXiv preprint arXiv:2409.12576,

    arXiv

  67. [75]

    MultiBooth: Towards generating all your concepts in an im- age from text

    Chenyang Zhu, Kai Li, Yue Ma, Chunming He, and Li Xiu. MultiBooth: Towards generating all your concepts in an im- age from text. arXiv preprint arXiv:2404.14239, 2024. 2

    2024 arXiv

  68. [76]

    Moviefactory: Automatic movie creation from text using large generative models for language and images

    Junchen Zhu, Huan Yang, Huiguo He, Wenjing Wang, Zixi Tuo, Wen-Huang Cheng, Lianli Gao, Jingkuan Song, and Jianlong Fu. Moviefactory: Automatic movie creation from text using large generative models for language and images. In ACM MM, pages 9313–9319, 2023. 2 Supplementary Mat...

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.