Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RelationAdapter claims that a decoupled attention module can pull editing intent from one before/after image pair and re-apply it to new images, across 218 editing tasks and unseen edits.

desk verdict A large, potentially useful dataset and a plausible adapter design, but the headline comparison is confounded and the same-backbone ablation shows a much smaller effect than the abstract implies. read the letter →

arxiv 2506.02528 v1 pith:QLW4HHYC submitted 2025-06-03 cs.CV

Yan Gong , Yiren Song , Yicheng Li , Chenglin Li , Yin Zhang This is my paper
classification cs.CV
keywords visualin-contextlearningimageeditingdiffusiontransformerpairpromptedittransferLoRARelation252K
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RelationAdapter claims that one before-and-after image pair is enough to tell a diffusion model what edit to perform, and that the edit can then be applied to a new photo the model has never seen. The paper introduces a lightweight module placed inside each Diffusion Transformer block that reads the pair, encodes it with a shared vision encoder, and re-injects the extracted relation through a separate attention path rather than by concatenating long visual tokens into the text stream. To train and test this, the authors built Relation252K, a dataset of 218 editing tasks derived from 33K image pairs, and an In-Context Editor that uses LoRA fine-tuning, noise-free source conditioning, and positional encoding cloning to keep edits aligned. On the benchmark, RelationAdapter reports lower MSE and higher CLIP-I, GPT-based consistency, and GPT-based accuracy than Edit Transfer and VisualCloze, and the same-backbone ablation shows the adapter itself contributes. If the claims hold, example pairs could replace written instructions for edits that are hard to describe in language.

What carries the argument

The load-bearing object is RelationAdapter's decoupled attention processor, a per-block module that encodes the visual prompt pair with a shared SigLIP encoder, projects the pair features through separate key/value weights $W'_k$, $W'_v$, and fuses the result into the backbone attention with $Z_{\mathrm{new}} = Z_B + \alpha Z_V$. Its job is to make the transformation in the example pair visible to the frozen DiT without letting visual tokens crowd out text conditioning. The supporting mechanism is the In-Context Editor, which concatenates noisy target latents, clean source tokens, and text tokens, clones positional encodings from source to target, and is fine-tuned with high-rank LoRA so the whole system can be trained on 251,580 editing samples.

What would settle it

Retrain Edit Transfer and VisualCloze on Relation252K with the same FLUX.1-dev backbone, LoRA rank, and training budget as RelationAdapter; if their MSE and CLIP-I scores approach or beat the reported ones, the claimed advantage of the adapter is unsupported. A second check keeps the In-Context Editor fixed and feeds the prompt pair through simple token concatenation instead of the decoupled KV path; if scores stay high, the decoupling is not the active mechanism.

Watch

Extended reading notes

Core claim

The central claim is that editing intent can be explicitly decoupled from generation and injected into a frozen Diffusion Transformer. Given a prompt pair $(I_{\mathrm{prm}}, I_{\mathrm{ref}})$ — the same image before and after an edit — a shared vision encoder produces feature sequences that the RelationAdapter fuses into every DiT block. The fusion uses a decoupled key-value attention processor: visual features get their own key and value projections $W'_k$, $W'_v$, while the query $Q$ comes from the backbone, and the visual attention output $Z_V$ is added to the backbone attention output $Z_B$ with scalar weight $\alpha$. This keeps long visual-prompt sequences from diluting text guidance and lets the frozen backbone apply the learned relation to a new source image. The paper further claims that an In-Context Editor with positional encoding cloning and a noise-free source branch preserves structure and extends the framework to tasks not seen during training.

Load-bearing premise

The headline comparison assumes the performance gap over Edit Transfer and VisualCloze comes from RelationAdapter, but those baselines run on their own base generators, so a stronger base model could explain some or all of the lead.

Editorial extensions

If this is right

  • A user can specify an edit by showing one before-and-after pair, covering low-level processing, style transfer, semantic edits, and customized generation in a single framework.
  • Because prompt-pair features travel through their own key/value projections while sharing the backbone query, text prompts keep their guiding role even when visual embeddings are long.
  • The adapter is modular and frozen-backbone friendly, so the same relation-extraction path can be attached to other DiT generators with LoRA-scale tuning.
  • The Relation252K benchmark, with 218 tasks including held-out ones, gives edit-transfer research a common test bed for measuring generalization.
  • Removing the adapter lowers reported scores on both seen and unseen tasks, so the gain is not attributable to the base generator or LoRA tuning alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's experiments, if decoupled KV injection is the active ingredient, the same module should transfer to other DiT-based generators (for example, video or multi-frame diffusion) and to relation tasks such as relighting or pose transfer without architectural changes.
  • The headline comparison runs Edit Transfer and VisualCloze on their own official backbones; a natural follow-up is retraining both on Relation252K with the same FLUX.1-dev backbone, LoRA rank, and compute budget, since the paper's own ablation hints the gap narrows when the backbone is held fixed.
  • The dataset is machine-annotated with a multimodal language model, so a clean human-labeled subset would test whether the reported GPT-based editing scores reflect human-perceived quality rather than the annotator model's preferences.
  • At roughly 1.57B trainable parameters and 40 GB inference memory, the method is not lightweight in deployment terms; a practical next step is to measure how much performance drops when LoRA rank or adapter width is reduced.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RelationAdapter, a DiT-based adapter that injects visual pair features into a diffusion transformer via decoupled key-value attention, together with an In-Context Editor that uses position encoding cloning and a noise-free conditioning scheme for image editing. The authors also introduce Relation252K, a dataset of 218 editing tasks and 251,580 (or 2,515,800, see major comments) training instances, and report quantitative and qualitative results against Edit Transfer and VisualCloze, including a same-backbone ablation study. The central claim is that RelationAdapter enables efficient capture and transfer of visual transformations from minimal example pairs, yielding notable gains in generation quality and editing performance.

Significance. If the empirical claims withstand scrutiny, the decoupled attention-injection design is a clean and potentially scalable way to condition DiT-based editors on exemplar pairs, and the Relation252K dataset would be a useful community resource. The paper explicitly ships a dataset, a user study, and implementation details, and the ablation in Table 2 does suggest a real effect for the adapter module. However, the headline comparison in Table 1 is confounded by different base models and training data, the automated GPT-4o judge is the same model family that generated the training annotations, and the unseen-task evaluation set is small and unspecified. These issues currently leave the central quantitative claim under-supported, although they are addressable with additional experiments and reporting.

major comments (4)
  1. [§4.3, Table 1] The quantitative comparison in Table 1 does not control for the base generative model or training distribution. Section 4.3 states that Edit Transfer and VisualCloze are evaluated with their official implementations and recommended hyperparameters, but it does not state that either baseline was retrained on Relation252K or adapted to the FLUX.1-dev backbone used for RelationAdapter. The reported gaps (e.g., MSE 0.020 vs 0.043 and CLIP-I 0.905 vs 0.827 against Edit Transfer) could therefore be driven by the base model or by differences in training data rather than by the RelationAdapter design. The same-backbone ablation in Table 2 shows a much smaller advantage (seen MSE 0.044 vs 0.055; unseen MSE 0.053 vs 0.061), and its "w/o RA" baseline is a concatenation-based variant rather than a competing method. Please add same-backbone, same-data baseline re-trainings and report repeated-seed standard deviations or confidence intervals.
  2. [§3.3, §3.4, and contributions list] The dataset size is stated inconsistently: Section 3.3 says the in-context editing dataset comprises 2,515,800 samples, while Section 3.4 and the contributions list state 251,580 editing samples produced from 33,274 images via permutation. This discrepancy is load-bearing for the dataset contribution and must be resolved. Please clarify the exact number of training instances, the permutation procedure, and why the two sections disagree.
  3. [§4.4, Fig. 10 and §3.4, Fig. 4] The GPT-C and GPT-A evaluation scores are produced by GPT-4o, the same model family that generated the edit instructions and image captions used to build Relation252K (Figure 4 and the prompt in Figure 9). Because the judge's notion of a correct edit is derived from the same model that labeled the training pairs, the automated scores may systematically favor outputs that align with GPT-4o's interpretation, creating a circularity risk. The paper does not report any validation of GPT-C/GPT-A against human ratings; the user study in Appendix D measures preference against baselines but does not calibrate the automated judge. Please provide a human-correlation study for the automated metrics or, at minimum, report results with the judge's identity and prompt held fixed across all methods and ablations.
  4. [§4.2, Table 3] The generalization claim to unseen tasks rests on a 300-sample subset whose composition is not described. Section 4.2 states that 6,540 benchmark samples include 6,240 seen-task samples and 300 unseen-task samples, but the paper does not list which unseen task types are included, how many tasks they span, or the per-task sample counts. Table 3 aggregates results into four broad categories without separating seen from unseen, so the reader cannot assess whether the 300 samples are representative of the claimed 218-task diversity. Please provide a task-level breakdown of the unseen set and per-task metrics for both seen and unseen splits.
minor comments (5)
  1. [§4.1] The paper reports 1,569.76 million total trainable parameters, which sits oddly with the repeated claim that RelationAdapter is "lightweight." Please clarify how many parameters belong to the RelationAdapter module itself versus the rank-128 LoRA, and discuss whether the total fine-tuning cost is consistent with the efficiency narrative.
  2. [§3.4] The sentence "Directional editing instruction (Isrc → Itar) are provided solely as text prompt" has a subject-verb agreement error and should read "instructions are."
  3. [Fig. 8] The caption and legend text of Figure 8 contain garbled character sequences (e.g., "/uni0000001c/uni0000011a/...") that appear to be rendering artifacts; these need to be cleaned before publication.
  4. [Table 3 and §5] Table 3 is described as covering "four image generation tasks" while the surrounding text calls them "four classification tasks"; please align the terminology and clarify whether the n values are per-category sample counts or per-task counts.
  5. [References] References [32] and [33] appear to be the same CLIP paper; please merge or disambiguate them to avoid duplicate citations.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity; the main caveat is a self-referential GPT-4o evaluation loop, not an equation-level reduction.

  1. other [Section 3.4 (data annotation) and Section 4.4 (evaluation metrics)]
    "To improve annotation efficiency and scalability, we leverage the multimodal capabilities of GPT-4o to automatically generate image captions and editing instructions. ... To further assess editing quality from a human-centered perspective, we leverage GPT-4o to interpret the intended transformation from the prompt image Iprm to the reference image Iref, and evaluate the predictions based on two dimensions: Editing Consistency (GPT-C) ... and Editing Accuracy (GPT-A)."

    The 'intended transformation' used to score outputs is the same construct that GPT-4o produced when generating the training edit instructions. A model whose outputs align with GPT-4o's interpretation is therefore rewarded by the same generator that defined the label, so GPT-C/GPT-A are partially measuring agreement with the label source rather than an external ground truth. This does not force the MSE/CLIP-I results or the Table 2 ablation, which are independent, but it means the automated GPT-based half of the headline evaluation is not an independent test of editing intent. It is an evaluation loop rather than an equation-level reduction.

full rationale

RelationAdapter's architecture equations (Eq. 1 and Eq. 6) state a composition D(R(...), Isrc, Tprm) that does not presuppose the outcome; the ablation in Table 2 compares against a same-backbone in-context baseline trained for the same number of steps, giving independent evidence for a real effect. No fitted parameter is renamed as a prediction, no uniqueness theorem or load-bearing self-citation is invoked, and the external low-level comparisons in Tables 4-7 are self-contained benchmarks. The only circularity-adjacent element is the GPT-4o loop: the same model that writes the edit instructions used as training supervision also scores GPT-C/GPT-A, so the automated metrics are not independent of the data-generation process. This is an evaluation-bias and validity concern rather than a derivation-level circularity; the central empirical claim still rests on the ablations and user study. Score 2 reflects one minor self-referential evaluation loop, not a circular derivation.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its assumptions are domain-level: the pretrained backbone's capability, the vision encoder's representational adequacy, the validity of the synthetic dataset, and the reliability of the automated judge. These are typical for an empirical deep learning paper, but they are not verified within the paper.

free parameters (1)
  • attention fusion weight alpha = 1.0 fixed
    In Eq. 6, Znew = ZB + alpha*ZV. The scalar is set to 1.0 without sensitivity analysis; it is a hand-chosen weight controlling the RelationAdapter contribution.
assumptions (4)
  • domain assumption The FLUX.1-dev DiT backbone can represent editing transformations if conditioned appropriately
    The method relies on the pretrained FLUX.1-dev as the generative base; if the backbone cannot perform a task, the adapter cannot produce the edit.
  • domain assumption SigLIP features encode semantically aligned pair information
    The dual-branch SigLIP encoder extracts features from the image pair; this assumes SigLIP captures the visual correspondence needed to transfer edits.
  • domain assumption MidJourney and GPT-4o generated images are valid ground-truth edits
    Relation252K is built semi-automatically via MidJourney and GPT-4o without human verification of all pairs; training and evaluating against these targets assumes they represent correct edits.
  • domain assumption GPT-4o scores correlate with human judgment
    GPT-C and GPT-A metrics are used as primary quality measures; if the judge is biased, reported scores may not reflect actual editing quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers." pith.science (2026). https://pith.science/paper/QLW4HHYC

@misc{pith2026250602528,
  author       = {Pith},
  title        = {Pith review of: RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QLW4HHYC}},
  note         = {Machine review of arXiv:2506.02528}
}
read the original abstract

Inspired by the in-context learning mechanism of large language models (LLMs), a new paradigm of generalizable visual prompt-based image editing is emerging. Existing single-reference methods typically focus on style or appearance adjustments and struggle with non-rigid transformations. To address these limitations, we propose leveraging source-target image pairs to extract and transfer content-aware editing intent to novel query images. To this end, we introduce RelationAdapter, a lightweight module that enables Diffusion Transformer (DiT) based models to effectively capture and apply visual transformations from minimal examples. We also introduce Relation252K, a comprehensive dataset comprising 218 diverse editing tasks, to evaluate model generalization and adaptability in visual prompt-driven scenarios. Experiments on Relation252K show that RelationAdapter significantly improves the model's ability to understand and transfer editing intent, leading to notable gains in generation quality and overall editing performance.

Figures

Figures reproduced from arXiv: 2506.02528 by the authors.

Figure 1
Figure 1. Our framework, RelationAdapter, can effectively perform a variety of image editing tasks [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall architecture and training paradigm of RelationAdapter. We employ the RelationAdapter to decouple inputs by injecting visual prompt features into the MMAttention module to control the generation process. Meanwhile, a high-rank LoRA is used to train the In-Context Editor on a large-scale dataset. During inference, the In-Context Editor encodes the source image into conditional tokens, concatenates them wit… view at source ↗
Figure 3
Figure 3. Overview of the four main task cate￾gories in our dataset. Each block lists represen￾tative sub-tasks (with ellipses indicating more), along with image-pair examples. Source Cap￾on: A person wearing a white shirt with rolled￾up sleeves, posing casually. Target Cap￾on: A person wearing a white shirt with bu￾oned sleeves against a red background. Edit Instruc￾on: Apply an ID photo transforma￾on to the portrait, front-… view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Compared to baselines, RelationAdapter demonstrates outstanding instruction-following [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Ablation study results. Our strategy shows better editorial consistency. 5 Discussion As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The generated results of RelationAdapter. RelationAdapter can understand the transfor￾mations in example image editing pairs and apply them to the original image to achieve high-quality image editing. It demonstrates a certain level of generalization capability on unse…
Figure 8
Figure 8. Figure 8: User study results comparing our method with baselines (in-context learning, VisualCloze [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Structured prompt used for labeling image pairs and extracting transformation instructions. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Evaluation prompt used to assess edit consistency and accuracy between two generated [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Failure cases on gesture editing, background pedestrian removal, document rectification, [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Additional experimental results of RelationAdapter. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Additional experimental results of RelationAdapter. [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Additional experimental results of RelationAdapter. [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TryOnCrafter: Unleashing Camera Trajectories for Realistic Video Virtual Try-on via a Renderable 4D Try-on Proxy

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    TryOnCrafter is the first DiT-based framework for camera-controllable video virtual try-on via a renderable 4D try-on proxy distilled from 2D priors into 3DGS avatar animated with SMPL-X.

  2. WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generation

    cs.CV 2025-11 conditional novelty 5.0 of 10

    A bidirectional egocentric-to-exocentric video translation framework trained with in-context attention on a new synthetic+real dataset, with evaluation flaws around reference leakage and missing direct baselines.

Reference graph

Works this paper leans on

69 extracted references · 29 canonical work pages · cited by 2 Pith papers

  1. [1]

    Flux: Official inference repository for flux.1 models

    Black Forest Labs. Flux: Official inference repository for flux.1 models. https://github.com/ black-forest-labs/flux , 2024. Accessed: 2025-05-14

    work page 2024

  2. [2]

    Flux.1-dev: A 12b parameter rectified flow transformer for text-to-image generation

    Black Forest Labs. Flux.1-dev: A 12b parameter rectified flow transformer for text-to-image generation. https://huggingface.co/black-forest-labs/FLUX.1-dev , 2024. Accessed: 2025-05-14

    work page 2024

  3. [3]

    Flux.1 redux-dev

    Black Forest Labs. Flux.1 redux-dev. https://huggingface.co/black-forest-labs/FLUX. 1-Redux-dev, 2024. Accessed: 2025-05-14

    work page 2024

  4. [4]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 18392–18402, 2023

    2023

  5. [5]

    Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF international conference on computer vision, pages 22560–22570, 2023

    2023

  6. [6]

    Edit transfer: Learning image editing via vision in-context relations

    Lan Chen, Qi Mao, Yuchao Gu, and Mike Zheng Shou. Edit transfer: Learning image editing via vision in-context relations. arXiv preprint arXiv:2503.13327, 2025

    arXiv 2025

  7. [7]

    Ringid: Rethinking tree-ring watermarking for enhanced multi-key identification

    Hai Ci, Pei Yang, Yiren Song, and Mike Zheng Shou. Ringid: Rethinking tree-ring watermarking for enhanced multi-key identification. In European Conference on Computer Vision, pages 338–354. Springer, 2024

    work page 2024

  8. [8]

    Video demoireing with relation-based temporal consistency

    Peng Dai, Xin Yu, Lan Ma, Baoheng Zhang, Jia Li, Wenbo Li, Jiajun Shen, and Xiaojuan Qi. Video demoireing with relation-based temporal consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    work page 2022

Show all 69 references
  1. [9]

    The cube++ illumination estimation dataset

    Egor Ershov, Alexey Savchik, Illya Semenkov, Nikola Bani´c, Alexander Belokopytov, Daria Senshina, Karlo Košˇcevi´c, Marko Subaši´c, and Sven Lonˇcari´c. The cube++ illumination estimation dataset. IEEE Access, 8:227511–227527, 2020

    2020

  2. [10]

    Geometric representation learning for document image rectification

    Hao Feng, Wengang Zhou, Jiajun Deng, Yuechen Wang, and Houqiang Li. Geometric representation learning for document image rectification. In Proceedings of the European Conference on Computer Vision (ECCV), 2022

    2022

  3. [11]

    Concept sliders: Lora adaptors for precise control in diffusion models

    Rohit Gandikota, Joanna Materzy ´nska, Tingrui Zhou, Antonio Torralba, and David Bau. Concept sliders: Lora adaptors for precise control in diffusion models. In European Conference on Computer Vision, pages 172–188. Springer, 2024

    2024

  4. [12]

    Any2anytryon: Leveraging adaptive position embeddings for versatile virtual clothing tasks

    Hailong Guo, Bohan Zeng, Yiren Song, Wentao Zhang, Chuang Zhang, and Jiaming Liu. Any2anytryon: Leveraging adaptive position embeddings for versatile virtual clothing tasks. arXiv preprint arXiv:2501.15891, 2025

    2025 arXiv

  5. [13]

    From sky to the ground: A large-scale benchmark and simple baseline towards real rain removal

    Yun Guo, Xueyao Xiao, Yi Chang, Shumin Deng, and Luxin Yan. From sky to the ground: A large-scale benchmark and simple baseline towards real rain removal. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12097–12107, October 2023

    2023

  6. [14]

    Prompt-to- prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to- prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022

    2022 arXiv

  7. [15]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  8. [16]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  9. [17]

    Photodoodle: Learning artistic image editing from few-shot pairwise data.arXiv preprint arXiv:2502.14397, 2025

    Shijie Huang, Yiren Song, Yuxuan Zhang, Hailong Guo, Xueyin Wang, Mike Zheng Shou, and Jiaming Liu. Photodoodle: Learning artistic image editing from few-shot pairwise data.arXiv preprint arXiv:2502.14397, 2025

    2025 arXiv

  10. [18]

    Humansd: A na- tive skeleton-guided diffusion model for human image generation

    Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, and Qiang Xu. Humansd: A na- tive skeleton-guided diffusion model for human image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15988–15998, 2023

    2023

  11. [19]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6007–6017, 2023. 10

    2023

  12. [20]

    Auto-encoding variational bayes, 2013

    Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013

    2013

  13. [21]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22511–22521, 2023

    2023

  14. [22]

    Visualcloze: A universal image generation framework via visual in-context learning.arXiv preprint arXiv:2504.07960, 2025

    Zhong-Yu Li, Ruoyi Du, Juncheng Yan, Le Zhuo, Zhen Li, Peng Gao, Zhanyu Ma, and Ming-Ming Cheng. Visualcloze: A universal image generation framework via visual in-context learning.arXiv preprint arXiv:2504.07960, 2025

    2025

  15. [23]

    Exploring the limits of weakly supervised pretraining

    Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. Exploring the limits of weakly supervised pretraining. In Proceedings of the European conference on computer vision (ECCV), pages 181–196, 2018

    2018

  16. [24]

    Sdedit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073, 2021

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073, 2021

    2021 arXiv

  17. [25]

    T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 4...

    2024

  18. [26]

    Deep multi-scale convolutional neural network for dynamic scene deblurring

    Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3883–3891, 2017

    2017

  19. [27]

    Gpt-4o technical report

    OpenAI. Gpt-4o technical report. https://openai.com/index/gpt-4o, May 2024. Accessed: 2025- 05-14

    2024

  20. [28]

    Multi-modal attention for speech emotion recognition

    Zexu Pan, Zhaojie Luo, Jichen Yang, and Haizhou Li. Multi-modal attention for speech emotion recognition. arXiv preprint arXiv:2009.04107, 2020

    2009 arXiv

  21. [29]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  22. [30]

    Ntire 2021 challenge on high dynamic range imaging: Dataset, methods and results

    Eduardo Pérez-Pellitero, Sibi Catley-Chandar, Ales Leonardis, and Radu Timofte. Ntire 2021 challenge on high dynamic range imaging: Dataset, methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 691–700, 2021

    2021

  23. [31]

    Dense extreme inception network: Towards a robust cnn model for edge detection

    Xavier Soria Poma, Edgar Riba, and Angel Sappa. Dense extreme inception network: Towards a robust cnn model for edge detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1923–1932, 2020

    1923

  24. [32]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  25. [33]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. arXiv preprint arXiv...

    2021 arXiv

  26. [34]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  27. [35]

    Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 225...

    2023

  28. [36]

    Schönberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger

    Thomas Schöps, Johannes L. Schönberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi- camera videos. In Proceedings of the IEEE Conference on Computer Vision and Pat...

    2017

  29. [37]

    Fonts: Text rendering with typography and style controls

    Wenda Shi, Yiren Song, Dengming Zhang, Jiaming Liu, and Xingxing Zou. Fonts: Text rendering with typography and style controls. arXiv preprint arXiv:2412.00136, 2024. 11

    2024 arXiv

  30. [38]

    Insert anything: Image insertion via in-context editing in dit

    Wensong Song, Hong Jiang, Zongxing Yang, Ruijie Quan, and Yi Yang. Insert anything: Image insertion via in-context editing in dit. arXiv preprint arXiv:2504.15009, 2025

    2025 arXiv

  31. [39]

    Layertracer: Cognitive-aligned layered svg synthesis via diffusion transformer

    Yiren Song, Danze Chen, and Mike Zheng Shou. Layertracer: Cognitive-aligned layered svg synthesis via diffusion transformer. arXiv preprint arXiv:2502.01105, 2025

    2025 arXiv

  32. [40]

    Processpainter: Learn painting process from sequence data

    Yiren Song, Shijie Huang, Chen Yao, Xiaojun Ye, Hai Ci, Jiaming Liu, Yuxuan Zhang, and Mike Zheng Shou. Processpainter: Learn painting process from sequence data. arXiv preprint arXiv:2406.06062, 2024

    2024 arXiv

  33. [41]

    Makeanything: Harnessing diffusion transformers for multi-domain procedural sequence generation

    Yiren Song, Cheng Liu, and Mike Zheng Shou. Makeanything: Harnessing diffusion transformers for multi-domain procedural sequence generation. arXiv preprint arXiv:2502.01572, 2025

    2025 arXiv

  34. [42]

    Omniconsistency: Learning style-agnostic consistency from paired stylization data

    Yiren Song, Cheng Liu, and Mike Zheng Shou. Omniconsistency: Learning style-agnostic consistency from paired stylization data. arXiv preprint arXiv:2505.18445, 2025

    2025 arXiv

  35. [43]

    Diffsim: Taming diffusion models for evaluating visual similarity

    Yiren Song, Xiaokang Liu, and Mike Zheng Shou. Diffsim: Taming diffusion models for evaluating visual similarity. arXiv preprint arXiv:2412.14580, 2024

    2024 arXiv

  36. [44]

    Anti-reference: Universal and immediate defense against reference-based generation

    Yiren Song, Shengtao Lou, Xiaokang Liu, Hai Ci, Pei Yang, Jiaming Liu, and Mike Zheng Shou. Anti-reference: Universal and immediate defense against reference-based generation. arXiv preprint arXiv:2412.05980, 2024

    2024 arXiv

  37. [45]

    Grid: Visual layout generation

    Cong Wan, Xiangyang Luo, Zijian Cai, Yiren Song, Yunlong Zhao, Yifan Bai, Yuhang He, and Yihong Gong. Grid: Visual layout generation. arXiv preprint arXiv:2412.10718, 2024

    2024 arXiv

  38. [46]

    Images speak in images: A generalist painter for in-context visual learning

    Xinlong Wang, Wen Wang, Yue Cao, Chunhua Shen, and Tiejun Huang. Images speak in images: A generalist painter for in-context visual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6830–6839, 2023

    2023

  39. [47]

    Image quality assessment: From error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004

    2004

  40. [48]

    Investigating prompt engineering in diffusion models

    Sam Witteveen and Martin Andrews. Investigating prompt engineering in diffusion models. arXiv preprint arXiv:2211.15462, 2022

    2022 arXiv

  41. [49]

    Versatile diffusion: Text, images and variations all in one diffusion model

    Xingqian Xu, Zhangyang Wang, Gong Zhang, Kai Wang, and Humphrey Shi. Versatile diffusion: Text, images and variations all in one diffusion model. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7754–7765, 2023

    2023

  42. [50]

    Paint by example: Exemplar-based image editing with diffusion models

    Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-based image editing with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18381–18391, 2023

    2023

  43. [51]

    Imagebrush: Learning visual in-context instructions for exemplar-based image manipulation

    Yifan Yang, Houwen Peng, Yifei Shen, Yuqing Yang, Han Hu, Lili Qiu, Hideki Koike, et al. Imagebrush: Learning visual in-context instructions for exemplar-based image manipulation. Advances in Neural Information Processing Systems, 36:48723–48743, 2023

    2023

  44. [52]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023

    2023

  45. [53]

    Finestyle: Fine-grained controllable style personalization for text-to-image models

    Gong Zhang, Kihyuk Sohn, Meera Hahn, Humphrey Shi, and Irfan Essa. Finestyle: Fine-grained controllable style personalization for text-to-image models. Advances in Neural Information Processing Systems, 37:52937–52961, 2024

    2024

  46. [54]

    Transparent image layer diffusion using latent transparency

    Lvmin Zhang and Maneesh Agrawala. Transparent image layer diffusion using latent transparency. arXiv preprint arXiv:2402.17113, 2024

    2024 arXiv

  47. [55]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    2023

  48. [56]

    Ssr-encoder: Encoding selective subject representation for subject-driven generation

    Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. Ssr-encoder: Encoding selective subject representation for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  49. [57]

    Fast personalized text to image synthesis with attention injection

    Yuxuan Zhang, Yiren Song, Jinpeng Yu, Han Pan, and Zhongliang Jing. Fast personalized text to image synthesis with attention injection. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6195–6199. IEEE, 2024. 12

    2024

  50. [58]

    Stable-makeup: When real-world makeup transfer meets diffusion model

    Yuxuan Zhang, Lifu Wei, Qing Zhang, Yiren Song, Jiaming Liu, Huaxia Li, Xu Tang, Yao Hu, and Haibo Zhao. Stable-makeup: When real-world makeup transfer meets diffusion model. arXiv preprint arXiv:2403.07764, 2024

    2024 arXiv

  51. [59]

    Easycontrol: Adding efficient and flexible control for diffusion transformer

    Yuxuan Zhang, Yirui Yuan, Yiren Song, Haofan Wang, and Jiaming Liu. Easycontrol: Adding efficient and flexible control for diffusion transformer. arXiv preprint arXiv:2503.07027, 2025

    2025 arXiv

  52. [60]

    Stable-hair: Real-world hair transfer via diffusion model

    Yuxuan Zhang, Qing Zhang, Yiren Song, and Jiaming Liu. Stable-hair: Real-world hair transfer via diffusion model. arXiv preprint arXiv:2407.14078, 2024

    2024 arXiv

  53. [61]

    4 images are organized into a grid of 2 rows and 2 columns

    Le Zhuo, Ruoyi Du, Han Xiao, Yangguang Li, Dongyang Liu, Rongjie Huang, Wenze Liu, Lirui Zhao, Fu-Yun Wang, Zhanyu Ma, et al. Lumina-next: Making lumina-t2x stronger and faster with next-dit. arXiv preprint arXiv:2406.18583, 2024. 13 Appendices The Appendices provide a compreh...

    2024 arXiv

  54. [62]

    Edit Accuracy: Which image better aligns with the editing intent implied by the prompt pair?

  55. [63]

    Edit Consistency: Which image better preserves the structure and identity of the source image?

  56. [64]

    Overall Preference: Which image do you prefer overall? The aggregated results of the user study are summarized in Figure 8. When compared with an in-context learning-based method, our approach was preferred for tasks included in training in 73.19% of cases for Edit Accuracy, 8...

  57. [65]

    ’left_image_description’

  58. [66]

    ’right_image_description’

  59. [67]

    Figure 9: Structured prompt used for labeling image pairs and extracting transformation instructions

    ’edit_instruction’ Do not include any other text or explanation. Figure 9: Structured prompt used for labeling image pairs and extracting transformation instructions. Text Prompts You are given a composite image with two columns. The left column contains three images arranged ...

  60. [68]

    Edit Consistency (1–5): How visually consistent is the edited image (B1 or B2) with the original image B? Focus: Are key objects, colors, and structures consistent with the source?

  61. [69]

    B1": {"consistency

    Edit Accuracy (1–5): Assess how accurately the editing operation applied to B (to produce B1 or B2) mirrors the transformation seen from A → A1. Focus: Did the editor apply similar changes, in the correct location, with the same degree of modification? Avoid giving tied scores...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.