Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Object-level Visual Prompts for Compositional Image Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Two encoders, one attention trick: object identity survives while scenes vary.

desk verdict KV-mixed cross-attention is a solid, well-ablated idea, but the headline DINOcomp gain is compromised by the compositional guidance optimizing the same metric. read the letter →

arxiv 2501.01424 v1 pith:XBM6R44V submitted 2025-01-02 cs.CV cs.AIcs.GR

Gaurav Parmar , Or Patashnik , Kuan-Chieh Wang , Daniil Ostashev , Srinivasa Narasimhan , Jun-Yan Zhu , Daniel Cohen-Or , Kfir Aberman This is my paper · ORCID
classification cs.CVcs.AIcs.GR
keywords compositionalimagegenerationvisualpromptsKV-mixedcross-attentionidentitypreservationtext-to-imagediffusionlayoutdiversityobject-levelguidancemulti-objectcomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VisualComposer asks whether a text-to-image diffusion model can treat several input images as composable visual prompts—each object and the background as a separate conditioning signal—and generate new scenes that keep each object recognizable while arranging them freely. The paper's central claim is that it can, and that the key step is a KV-mixed cross-attention mechanism: attention keys are computed from a coarse, strongly compressed visual encoder to control layout, while values come from a fine-grained encoder to carry appearance details. This split directly targets the identity-diversity tradeoff that limits prior image-prompt adapters, where a narrow bottleneck loses identity and a wide bottleneck overfits to the input pose. At inference, object-level compositional guidance uses segmentation masks and DINO similarity to sharpen identity and layout adherence. The paper reports that the method beats image-prompt, multimodal, and optimization-based baselines on DINOcomp and CLIPcomp compositional identity metrics while keeping LPIPS diversity high.

What carries the argument

The load-bearing mechanism is the KV-mixed cross-attention layer. It takes layout tokens from a coarse encoder's global CLIP embedding through a small layout adapter (a linear layer with layer normalization) and uses them as keys, while appearance tokens from a fine-grained CLIP grid-feature encoder pass through a Perceiver-based appearance adapter and are used as values; keys and values are therefore drawn from two different visual representations. A bounded cross-attention loss during training forces each prompt's attention map to align with its binary object mask. At inference, Compositional Guidance first segments a draft image, assigns segments to prompts by Hungarian matching on DINOv2 similarity, then masks out attention outside each matched region and optimizes the appearance tokens to maximize DINO similarity to each prompt.

What would settle it

Run the method on a held-out set of object prompts and compare KV-mixed against coarse-only and fine-only variants with identical seeds; if the identity-versus-diversity curve for KV-mixed lies below the Pareto frontier set by the two pure variants, or if coarse keys fail to place attention inside object masks for unseen objects, the central claim is refuted.

Watch

Extended reading notes

Core claim

The paper introduces VisualComposer, a feed-forward method for composing N object-level visual prompts (N−1 objects plus a background) in a frozen text-to-image diffusion model. Its central discovery is that the identity-diversity tradeoff in image prompting can be broken by separating the two roles of cross-attention: use a coarse encoder for keys, which determine where each prompt exerts influence, and a fine-grained encoder for values, which determine what each prompt looks like. On top of this, Compositional Guidance at inference zeroes attention outside each object's detected segment and backpropagates a DINO-based identity loss through the appearance tokens, further aligning generation with the input prompts. In the paper's evaluations the method preserves input identity substantially better than IP-Adapter, IP-Adapter Plus, BLIP-Diffusion, KOSMOS-G, λ-ECLIPSE, and Break-a-Scene, while matching or exceeding their layout diversity.

Load-bearing premise

The whole method rests on the assumption that attention keys govern layout and values govern appearance, and that splitting them across a coarse and a fine encoder keeps that separation clean enough for arbitrary unseen objects.

Editorial extensions

If this is right

  • Users can compose a background and several object crops into a text-to-image model without hand-drawn layouts, and get multiple varied scenes that still look like the input objects.
  • Because conditioning is per-object, a user can move or resize a single object by editing its attention map, and the rest of the scene, including poses and reflections, adapts around it.
  • The method is feed-forward: no per-subject or per-scene optimization is needed, so the same trained model handles new objects and new background prompts at inference.
  • The KV-split gives a direct handle on the adapter tradeoff: identity comes from value granularity and diversity from key granularity, so each can be tuned separately.
  • Compositional guidance improves identity, reduces attribute leakage between objects, and suppresses duplicate objects, addressing common failure modes of multi-object generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the key-value role split holds generally, the same coarse-key/fine-value recipe could be transplanted to other conditional generators, such as video, audio, or 3D synthesis, wherever one modality supplies layout and another supplies texture.
  • The paper's bottleneck argument suggests a continuous design axis: by choosing the degree of compression for keys and values, future systems could dial identity versus diversity for a given object class instead of rearchitecting the adapter.
  • The current guidance pipeline depends on open-set segmentation and DINO matching at inference; a natural test is whether a segmentation-free variant can recover the same gains, which would reveal how much of the improvement is architectural versus inference-time.
  • Because the method operates on object-level prompts, it could be combined with text-only editing of one object in a multi-object scene, enabling localized style or identity swaps without regenerating the whole composition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes VisualComposer, a method for composing multiple object-level visual prompts in a text-to-image diffusion model. The core architectural contribution is a KV-mixed cross-attention layer in which keys are computed from a coarse encoder with a small bottleneck, for layout control, and values are computed from a fine-grained encoder with a large bottleneck, for appearance identity. Training uses a diffusion reconstruction loss plus a bounded cross-attention loss that aligns attention maps with object masks. At inference, the method adds compositional guidance: it first generates an image, segments it with an open-set segmentation model, matches segments to input prompts using DINOv2 similarity and Hungarian assignment, and then backpropagates a DINOv2-similarity loss into the appearance tokens while also zeroing attention outside matched segments. Experiments compare against IP-Adapter, IP-Adapter Plus, BLIP-Diffusion, KOSMOS-G, λ-ECLIPSE, and Break-A-Scene on identity preservation (DINOcomp, CLIPcomp) and diversity (LPIPSavg), plus a user preference study.

Significance. If the identity-diversity tradeoff claim holds, the KV-mixed cross-attention design is a simple and plausible feed-forward solution to a well-known limitation of image-prompt adapters. The paper is clearly written, the architecture is easy to reproduce from the description, and the user study is a useful addition. However, the primary quantitative identity evidence is compromised by evaluation circularity: the headline DINOcomp metric is the same DINOv2 similarity that the inference-time guidance explicitly optimizes. The paper therefore does not currently establish its central claim that KV-mixed attention, rather than the test-time optimization, preserves identity. The missing error bars and uneven baseline adaptations further weaken the quantitative comparisons. The contribution is potentially significant, but the evaluation must be reworked before the claim is convincing.

major comments (4)
  1. [Section 3.4, Eq. (3); Section 4.1, Table 1] The DINOcomp metric is not independent of the method being evaluated. Equation (3) defines an inference-time loss that maximizes DINO(P_v^n, S_{sigma(n)}) by backpropagating into the appearance tokens, and Section 4.1 defines DINOcomp as the same DINOv2 similarity with the same Hungarian matching protocol. Consequently, the DINOcomp advantage (e.g., 0.518 vs. 0.363 for Break-A-Scene) can be inflated by the compositional guidance itself and does not by itself establish that KV-mixed cross-attention preserves identity. I recommend reporting DINOcomp for the model without compositional guidance, or using a different feature extractor that is not involved in the optimization, and treating the guided variant as a separate inference-time method.
  2. [Table 1 and Figure 7] No error bars, confidence intervals, or significance tests are reported for any quantitative result. The differences in CLIPcomp (0.676 vs. 0.669 for λ-ECLIPSE) and LPIPSavg (0.688 vs. 0.687 for KOSMOS-G) are small enough to fall within sampling noise, so the claim of outperforming all baselines on both axes is not supported without a repeated-runs analysis or at least per-method standard deviations.
  3. [Section 4.2, baseline adaptations] The baselines are adapted with different, ad hoc procedures: summing decoupled cross-attention outputs for IP-Adapter, averaging tokens for BLIP-Diffusion, and pasting object segments at random positions for Break-A-Scene. These uneven adaptations can either favor or disadvantage a baseline relative to a native multi-object method. Please justify each adaptation and, where possible, use the multi-image variants recommended by the original authors or provide a sensitivity analysis showing that the conclusions are robust to the adaptation choice.
  4. [Section 3.2 and Appendix B, Figure 13] The central architectural assumption that keys control layout and values control identity is imported from text-prompt cross-attention analysis and is not directly verified for the mixed-encoder setting. The ablation in Figure 7 is suggestive but relies on the same DINOcomp metric discussed above. A direct analysis of attention maps in controlled layout-change experiments, or an evaluation with a held-out metric unrelated to DINOv2, would strengthen the causal claim that the KV mix, rather than the guidance, is responsible for the identity-diversity tradeoff improvement.
minor comments (4)
  1. [Section 4.3] The text says 'whereas using a fine-grained encoder has pood diversity' but 'pood' should be 'poor'.
  2. [Table 2] The user study protocol is under-specified: the number of users, the exact display format, whether the input prompt was shown alongside both outputs, and the variance across users are not reported, and the statement that each comparison comprises 13,500 questions needs clarification given the dataset size and number of baselines.
  3. [Section 4.1] The evaluation protocol refers to 'open-set object detection' at the start but later uses 'segmentation' for the matching in Eq. (2); please make the terminology consistent and clarify whether detected segments or bounding-box crops are used for feature extraction.
  4. [Appendix D] The hyperparameter α in Eq. (1) and the settings for the compositional guidance optimization (e.g., number of gradient steps, learning rate, whether the loss is applied at every denoising step) are not reported, which prevents reproduction of the inference-time method.

Circularity Check

1 steps flagged · score 6.0 of 10

DINOcomp is the same DINOv2 similarity that the inference-time compositional guidance (Eq. 3) directly optimizes, so the main quantitative identity claim is partially self-scored.

  1. fitted input called prediction [Section 3.4, Eqs. (2)-(3) and Section 4.1, DINOcomp definition]
    "Sim(n, j) =DINO(P n v , Sj). (2) ... Lid = X n (1 − Sim(n, σ(n))), (3) ... We backpropagate this loss through the model to update the appearance tokens ... Following previous works that measured identity preservation for the personalization task, we use both DINOv2 [38] and CLIP [48] as our feature extractors and denote the corresponding scores as DINO comp and CLIPcomp, respectively."

    The identity metric DINOcomp is computed as DINOv2 similarity between input visual prompts and detected segments, matched via Hungarian assignment. At inference, the method's Compositional Guidance (Eq. 3) directly backpropagates to maximize the same DINOv2 similarity (Eq. 2) between each prompt and its matched segment, using the same open-set segmentation and Hungarian matching. Therefore the reported DINOcomp advantage is not an independent measure of identity preservation; it is the exact objective optimized at test time. The DINOcomp column in Table 1 thus partly reflects the success of per-instance optimization rather than the KV-mixed representation.

full rationale

The central circularity is in the evaluation protocol: the DINOcomp metric (Section 4.1) uses DINOv2 similarity with Hungarian matching between input prompts and detected segments, and the inference-time Compositional Guidance (Section 3.4, Eq. 3) explicitly maximizes that same DINOv2 similarity by backpropagating into the appearance tokens. Consequently, the paper's strongest quantitative evidence for identity preservation (Table 1, DINOcomp 0.518 vs. 0.363 for Break-A-Scene) is at least in part a direct measure of how well the per-instance optimization succeeded, not an independent test of the KV-mixed cross-attention design. This is a partial circularity because CLIPcomp and the user preference study are not directly optimized and provide some independent support, though the user study lacks error bars and measures only adherence, not diversity. The architectural claim that keys control layout and values control appearance is supported by prior work that includes non-overlapping authors (e.g., [59]) and by the paper's own ablation (Figure 7), so it is not a self-citation chain. No other load-bearing circular steps were found.

Assumptions & free parameters 5 free parameters · 3 assumptions · 2 invented entities

The method relies on a domain assumption about key and value roles, a DINOv2 similarity as identity measure, and a training distribution assumption. The free parameters are mostly standard hyperparameters, but the compositional guidance optimization settings are unspecified, which limits reproducibility.

free parameters (5)
  • alpha (α) in bounded cross-attention loss = not reported in main text
    Eq. (1) hyperparameter controlling penalty on attention outside object masks; hand-chosen and affects layout adherence.
  • Classifier-free guidance scale = 7.5
    Inference hyperparameter used in all comparisons; hand-chosen standard value.
  • DDIM inference steps = 25
    Inference schedule hyperparameter; hand-chosen.
  • Dataset aesthetic filter threshold = CLIP-Aesthetic score 5.0
    Training and evaluation data filtering threshold in Appendix D; hand-chosen and could bias the evaluation distribution.
  • Compositional guidance optimization settings
    Section 3.4 does not report the step size or number of iterations for backpropagating Lid into appearance tokens; this free choice affects identity improvement.
assumptions (3)
  • domain assumption Keys in cross-attention control layout and values control appearance.
    Used in Section 3.2 to justify KV-mixed design; supported by prior text-prompt studies but assumed to transfer to image-prompt conditioning.
  • domain assumption DINOv2 similarity is a valid measure of object identity preservation.
    Used both as the guidance objective (Eq. 3) and as the evaluation metric (DINOcomp in Table 1); assumes the similarity captures identity robust to pose and layout changes.
  • domain assumption Training data (COYO plus synthetic Flux images with SAM2 masks) is representative of user-supplied visual prompts.
    The model learns composition from this dataset; unusual combinations like dog plus single shoe plus forest are acknowledged as failure cases in Appendix E.
invented entities (2)
  • KV-Mixed Cross-Attention Layer
    purpose: Mixes keys from a coarse encoder with values from a fine-grained encoder to balance layout diversity and identity preservation.
    New architectural module introduced in Section 3.2; evidence comes only from the paper's own experiments, no external validation yet.
  • Object-level Compositional Guidance
    purpose: Inference-time refinement that masks attention to matched segments and backpropagates a DINO similarity loss into appearance tokens.
    New inference procedure in Section 3.4; its reported benefit is entangled with the DINO-based evaluation metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Object-level Visual Prompts for Compositional Image Generation." pith.science (2026). https://pith.science/paper/XBM6R44V

@misc{pith2026250101424,
  author       = {Pith},
  title        = {Pith review of: Object-level Visual Prompts for Compositional Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XBM6R44V}},
  note         = {Machine review of arXiv:2501.01424}
}
read the original abstract

We introduce a method for composing object-level visual prompts within a text-to-image diffusion model. Our approach addresses the task of generating semantically coherent compositions across diverse scenes and styles, similar to the versatility and expressiveness offered by text prompts. A key challenge in this task is to preserve the identity of the objects depicted in the input visual prompts, while also generating diverse compositions across different images. To address this challenge, we introduce a new KV-mixed cross-attention mechanism, in which keys and values are learned from distinct visual representations. The keys are derived from an encoder with a small bottleneck for layout control, whereas the values come from a larger bottleneck encoder that captures fine-grained appearance details. By mixing keys and values from these complementary sources, our model preserves the identity of the visual prompts while supporting flexible variations in object arrangement, pose, and composition. During inference, we further propose object-level compositional guidance to improve the method's identity preservation and layout correctness. Results show that our technique produces diverse scene compositions that preserve the unique characteristics of each visual prompt, expanding the creative potential of text-to-image generation.

Figures

Figures reproduced from arXiv: 2501.01424 by the authors.

Figure 1
Figure 1. We introduce a method for composing object-level visual prompts (shown above each column), where prompts consist of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. KV-Mixing. Image Prompt Adapters capture visual in￾formation from images to guide the generation process. The fea￾ture extractor’s bottleneck size (top row) determines the level of detail in the extracted Key-Value (KV) features. Using only coarse KVs (left) sacrifices identity preservation, while using only fine￾grained KVs (middle) limits scene variation. In contrast, combin￾ing mixed-granularity KVs (right) achie… view at source ↗
Figure 3
Figure 3. VisualComposer architecture. Our method begins by encoding all input visual prompts through two separate branches: an appearance branch (top row, shown in orange) that uses a Fine-Grained encoder followed by an Appearance adapter to encode per-prompt appearance tokens, and a layout branch (bottom row, shown in blue) that uses a Coarse encoder followed by a Layout adapter to encode per-prompt layout tokens. Once the … view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Gallery. Compositional images generated by VisualComposer. Four outputs (right) for each set of input visual prompts (left). To address this, we propose a new compositional iden￾tity metric that employs a feature extractor F. We first ap￾ply an open-set object detectio…
Figure 5
Figure 5. Figure 5: Comparisons to prior methods. We show a set of input visual prompts on the left. For each set, we show results generated by different methods. Our method achieves the best balance between identity preservation of the input prompts and image diversity. Our method is the…
Figure 6
Figure 6. Figure 6: Ablating Compositional Guidance. Our inference￾time compositional guidance improves identity preservation, re￾duces leakage between objects, and removes duplicates. Without guidance, the duck’s features leak into the bear (top row) and two vases get generated (bottom r…
Figure 7
Figure 7. Figure 7: Ablations. We ablate the importance of KV-Mixture and Compositional guidance. If Fine-Grained encoder is used for both keys and values, the method overfits and does not generate adequate variations. Conversely, using Coarse encoder results in poor identity preservation…
Figure 8
Figure 8. Figure 8: Compositional generation results. We show additional image composition results here. The input visual prompts are shown on the left and the generated compositional images are shown on the right. Input Image Reshuffled Outputs Input Image Reshuffled Outputs [PITH_FULL_…
Figure 9
Figure 9. Figure 9: Gallery of reshuffling results. The input images is shown on the left and three reshuffled results are shown on the right. E. Limitations and Societal Impacts. We show the limitations of our model in [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Additional comparisons to prior methods. We show a set of input visual prompts on the left. For each set, we show results generated by different methods. Our method outperforms each of the prior methods in terms adherence to the input visual prompt and diversity. Meth…
Figure 11
Figure 11. Figure 11: Additional comparisons to prior methods with a painting background prompt. We show a set of input visual prompts on the left. Notably, the background prompt for both examples is a painting. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Translation control. Our object-level image prompts provide fine-grained control over each object. For example, we move the orange ball by manipulating its attention map, and the dog’s pose changes in response to the ball’s location. Fine￾Grained Encoder Coarse Encode…
Figure 13
Figure 13. Figure 13: Visual ablation of KV Mixture. Visual Prompts Generated Compositional Images [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: Limitations. We show the results of an example that illustrates the limitations of our method. Our method tends to per￾form worse for unusual combinations of input visual prompts. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Muses: Designing, Composing, Generating Nonexistent Fantasy 3D Creatures without Training

    cs.CV 2026-01 conditional novelty 6.0 of 10

    Muses creates new fantasy 3D animals by designing a combined skeleton, fusing voxel parts from separate 3D models along that skeleton, then restyling textures via image editing — with no training.

  2. Doc-CoB: Enhancing Document Understanding with Visual Chain-of-Boxes Reasoning

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A two-stage layout-focused visual reasoning method, Doc-CoB, improves document question answering by having the model select key layout boxes and then answer from those boxes.

Reference graph

Works this paper leans on

74 extracted references · 48 canonical work pages · cited by 2 Pith papers

  1. [1]

    A neural space-time representation for text- to-image personalization

    Yuval Alaluf, Elad Richardson, Gal Metzer, and Daniel Cohen-Or. A neural space-time representation for text- to-image personalization. ACM Transactions on Graphics (TOG), 42(6):1–10, 2023. 3

    work page 2023

  2. [2]

    Cross-image attention for zero- shot appearance transfer

    Yuval Alaluf, Daniel Garibi, Or Patashnik, Hadar Averbuch- Elor, and Daniel Cohen-Or. Cross-image attention for zero- shot appearance transfer. In ACM SIGGRAPH 2024 Confer- ence Papers, pages 1–12, 2024. 4

    2024

  3. [3]

    Moab Arar, Rinon Gal, Yuval Atzmon, Gal Chechik, Daniel Cohen-Or, Ariel Shamir, and Amit H. Bermano. Domain- agnostic tuning-encoder for fast personalization of text-to- image models. In SIGGRAPH Asia 2023 Conference Papers, pages 1–10, 2023. 2, 4

    work page 2023

  4. [4]

    Break-a-scene: Extracting multi- ple concepts from a single image

    Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen- Or, and Dani Lischinski. Break-a-scene: Extracting multi- ple concepts from a single image. In SIGGRAPH Asia 2023 Conference Papers, pages 1–12, 2023. 2, 7, 8

    work page 2023

  5. [5]

    Spatext: Spatio-textual representation for con- trollable image generation

    Omri Avrahami, Thomas Hayes, Oran Gafni, Sonal Gupta, Yaniv Taigman, Devi Parikh, Dani Lischinski, Ohad Fried, and Xi Yin. Spatext: Spatio-textual representation for con- trollable image generation. In CVPR, 2023. 3

    work page 2023

  6. [6]

    Layer normalization

    Jimmy Lei Ba. Layer normalization. arXiv preprint arXiv:1607.06450, 2016. 4

    arXiv 2016

  7. [7]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015. 3

    work page 2015

  8. [8]

    Separate-and-enhance: Composi- tional finetuning for text-to-image diffusion models

    Zhipeng Bao, Yijun Li, Krishna Kumar Singh, Yu-Xiong Wang, and Martial Hebert. Separate-and-enhance: Composi- tional finetuning for text-to-image diffusion models. InACM SIGGRAPH 2024 Conference Papers, New York, NY , USA,

    work page 2024

Show all 74 references
  1. [9]

    Coyo-700m: Image-text pair dataset

    Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim. Coyo-700m: Image-text pair dataset. https : / / github . com / kakaobrain/coyo-dataset, 2022. 4

    2022

  2. [10]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV) , pages 22...

  3. [11]

    Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4):1–10, 2023. 2, 5

    2023

  4. [12]

    Training-free layout control with cross-attention guidance

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024. 3

    2024

  5. [13]

    Wenhu Chen, Hexiang Hu, Y ANDONG LI, Nataniel Ruiz, Xuhui Jia, Ming-Wei Chang, and William W. Cohen. Subject-driven text-to-image generation via apprenticeship learning. In Thirty-seventh Conference on Neural Informa- tion Processing Systems, 2023. 2

    2023

  6. [14]

    Anydoor: Zero-shot object-level image customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , page 6593–6602. IEEE, 2024. 3

    2024

  7. [15]

    Be yourself: Bounded attention for multi-subject text-to-image generation

    Omer Dahary, Or Patashnik, Kfir Aberman, and Daniel Cohen-Or. Be yourself: Bounded attention for multi-subject text-to-image generation. In European Conference on Com- puter Vision, pages 432–448. Springer, 2025. 2, 3, 5

    2025

  8. [16]

    Diffusion models beat GANs on image synthesis

    Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neu- ral Information Processing Systems, 2021. 5

    2021

  9. [17]

    An image is worth one word: Personalizing text-to-image gener- ation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image gener- ation using textual inversion. In The Eleventh International Conference on Learning Representations, 2022. 2

    2022

  10. [18]

    Encoder-based domain tuning for fast personalization of text-to-image models.ACM Transactions on Graphics (TOG), 42(4):1–13, 2023

    Rinon Gal, Moab Arar, Yuval Atzmon, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Encoder-based domain tuning for fast personalization of text-to-image models.ACM Transactions on Graphics (TOG), 42(4):1–13, 2023. 2, 4

    2023

  11. [19]

    Mix-of-show: Decentralized low- rank adaptation for multi-concept customization of diffusion models

    Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yun- peng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, et al. Mix-of-show: Decentralized low- rank adaptation for multi-concept customization of diffusion models. Advances in Neural Information Processing ...

    2024

  12. [20]

    Lo- calized text-to-image generation for free via cross attention control

    Yutong He, Ruslan Salakhutdinov, and J Zico Kolter. Lo- calized text-to-image generation for free via cross attention control. arXiv preprint arXiv:2306.14636, 2023. 3

    2023 arXiv

  13. [21]

    Prompt-to-prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. In ICLR, 2023. 2, 4 9

    2023

  14. [22]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 5

    2021

  15. [23]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 3

    2020

  16. [24]

    Diffusers

    HuggingFace. Diffusers. https://huggingface. co/docs/diffusers/en/using-diffusers/ip_ adapter, 2024. 7

    2024

  17. [25]

    Perceiver: General perception with iterative attention

    Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. In International confer- ence on machine learning, pages 4651–4664. PMLR, 2021. 4

    2021

  18. [26]

    Taming encoder for zero fine-tuning image customization with text-to-image diffusion models

    Xuhui Jia, Yang Zhao, Kelvin CK Chan, Yandong Li, Han Zhang, Boqing Gong, Tingbo Hou, Huisheng Wang, and Yu-Chuan Su. Taming encoder for zero fine-tuning image customization with text-to-image diffusion models. arXiv preprint arXiv:2304.02642, 2023. 2

    2023 arXiv

  19. [27]

    Scaling up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2023. 1, 2

    2023

  20. [28]

    Instantfamily: Masked attention for zero-shot multi-id image generation

    Chanran Kim, Jeongin Lee, Shichang Joung, Bongmo Kim, and Yeul-Min Baek. Instantfamily: Masked attention for zero-shot multi-id image generation. arXiv preprint arXiv:2404.19427, 2024. 3

    2024 arXiv

  21. [29]

    Dense text-to-image generation with attention modulation

    Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, and Jun-Yan Zhu. Dense text-to-image generation with attention modulation. In ICCV, 2023. 3

    2023

  22. [30]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. In ICLR, 2015. 12

    2015

  23. [31]

    The hungarian method for the assignment problem

    Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly , 2(1-2):83–97,

  24. [32]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In CVPR, 2023. 2

    2023

  25. [33]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2024. 4

    2024

  26. [34]

    Blip-diffusion: Pre- trained subject representation for controllable text-to-image generation and editing

    Dongxu Li, Junnan Li, and Steven Hoi. Blip-diffusion: Pre- trained subject representation for controllable text-to-image generation and editing. Advances in Neural Information Pro- cessing Systems, 36, 2024. 2, 7

    2024

  27. [35]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22511–22521, 2023. 2, 3

    2023

  28. [36]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 12

    2024

  29. [37]

    Divco: Diverse conditional image synthesis via contrastive generative adversarial network

    Rui Liu, Yixiao Ge, Ching Lam Choi, Xiaogang Wang, and Hongsheng Li. Divco: Diverse conditional image synthesis via contrastive generative adversarial network. InIEEE Con- ference on Computer Vision and Pattern Recognition, 2021. 6

    2021

  30. [38]

    Dinov2: Learning robust visual features without super- vision

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without super- vision. Transactions on Machine Learning Research,...

    2023

  31. [39]

    Kosmos-g: Generating images in context with multimodal large language models

    Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, and Furu Wei. Kosmos-g: Generating images in context with multimodal large language models. In The Twelfth International Conference on Learning Representa- tions, 2024. 7, 8

    2024

  32. [40]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceed- ings, pages 1–11, 2023. 2, 4

    2023

  33. [41]

    One-step image translation with text-to-image models

    Gaurav Parmar, Taesung Park, Srinivasa Narasimhan, and Jun-Yan Zhu. One-step image translation with text-to-image models. arXiv preprint arXiv:2403.12036, 2024. 3

    2024 arXiv

  34. [42]

    λ-eclipse: Multi-concept personalized text-to-image diffusion models by leveraging clip latent space

    Maitreya Patel, Sangmin Jung, Chitta Baral, and Yezhou Yang. λ-eclipse: Multi-concept personalized text-to-image diffusion models by leveraging clip latent space. arXiv preprint arXiv:2402.05195, 2024. 7, 8

    2024 arXiv

  35. [43]

    Grounded text-to-image synthesis with attention refocusing

    Quynh Phung, Songwei Ge, and Jia-Bin Huang. Grounded text-to-image synthesis with attention refocusing. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 7932–7942. IEEE, 2024. 2

    2024

  36. [44]

    Grounded text-to-image synthesis with attention refocusing

    Quynh Phung, Songwei Ge, and Jia-Bin Huang. Grounded text-to-image synthesis with attention refocusing. In CVPR,

  37. [45]

    Orthogonal adaptation for modular customization of diffusion models

    Ryan Po, Guandao Yang, Kfir Aberman, and Gordon Wet- zstein. Orthogonal adaptation for modular customization of diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 7964–7973, 2024. 2

    2024

  38. [46]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth Interna- tional Conference on Learning Representations, 2023. 5

    2023

  39. [47]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth Interna- tional Conference on Learning Representations, 2024. 2, 3, 4

    2024

  40. [48]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  41. [49]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2 10

    2022 arXiv

  42. [50]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junt- ing Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Doll´ar, and Christoph Feic...

    2024 arXiv

  43. [51]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 1, 2, 3, 4, 5

    2022

  44. [52]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...

    2023

  45. [53]

    Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat...

    2024

  46. [54]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  47. [55]

    In- stantbooth: Personalized text-to-image generation without test-time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. In- stantbooth: Personalized text-to-image generation without test-time finetuning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 8543–8552, 2024. 2, 4

    2024

  48. [56]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 3

    2015

  49. [57]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021. 5

    2021

  50. [58]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In International Conference on Learning Represen- tations. 3

  51. [59]

    Key-locked rank one editing for text-to-image personaliza- tion

    Yoad Tewel, Rinon Gal, Gal Chechik, and Yuval Atzmon. Key-locked rank one editing for text-to-image personaliza- tion. In ACM SIGGRAPH 2023 Conference Proceedings ,

    2023

  52. [60]

    Moa: Mixture-of-attention for subject-context disentanglement in personalized image gen- eration

    Kuan-Chieh Wang, Daniil Ostashev, Yuwei Fang, Sergey Tulyakov, and Kfir Aberman. Moa: Mixture-of-attention for subject-context disentanglement in personalized image gen- eration. In SIGGRAPH Asia 2024 Conference Papers, pages 1–12, 2024. 2, 3

    2024

  53. [61]

    Cnn-generated images are surprisingly easy to spot

    Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8695–8704, 2020. 14

    2020

  54. [62]

    Data attribution for text-to-image models by unlearning synthesized images

    Sheng-Yu Wang, Aaron Hertzmann, Alexei A Efros, Jun-Yan Zhu, and Richard Zhang. Data attribution for text-to-image models by unlearning synthesized images. InNeurIPS, 2024. 14

    2024

  55. [63]

    Instancediffusion: Instance-level control for image generation

    Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, and Ishan Misra. Instancediffusion: Instance-level control for image generation. In CVPR, 2024. 3

    2024

  56. [64]

    Elite: Encoding visual concepts into textual embeddings for customized text-to-image gener- ation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text-to-image gener- ation. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision (ICCV), pages 15943–15...

    2023

  57. [65]

    Florence-2: Advancing a unified representation for a variety of vision tasks

    Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. Florence-2: Advancing a unified representation for a variety of vision tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 481...

    2024

  58. [66]

    Fastcomposer: Tuning-free multi- subject image generation with localized attention

    Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention. Interna- tional Journal of Computer Vision, pages 1–20, 2024. 3, 5

    2024

  59. [67]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  60. [68]

    Florence: A new foundation model for computer vision

    Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, et al. Florence: A new foundation model for computer vision. arXiv preprint arXiv:2111.11432, 2021. 5, 6

    2021 arXiv

  61. [69]

    Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation

    Yu Zeng, Vishal M Patel, Haochen Wang, Xun Huang, Ting- Chun Wang, Ming-Yu Liu, and Yogesh Balaji. Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogni...

    2024

  62. [70]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 3

    2023

  63. [71]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 6

    2018

  64. [72]

    Large scale image comple- tion via co-modulated generative adversarial networks

    Shengyu Zhao, Jonathan Cui, Yilun Sheng, Yue Dong, Xiao Liang, Eric I Chang, and Yan Xu. Large scale image comple- tion via co-modulated generative adversarial networks. InIn- ternational Conference on Learning Representations (ICLR),

  65. [73]

    To- ward multimodal image-to-image translation

    Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Dar- rell, Alexei A Efros, Oliver Wang, and Eli Shechtman. To- ward multimodal image-to-image translation. InAdvances in Neural Information Processing Systems 30, pages 465–476. Curran Associates, Inc., 2017. 6 11 Appendix A pr...

    2017

  66. [2024]

    Association for Computing Machinery. 2, 5

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.