Pith. sign in

REVIEW 4 major objections 4 minor 49 references

RefEdit: A Benchmark and Method for Improving Instruction-based Image Editing Model on Referring Expressions

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Instruction-based image editors fail when an instruction must pick one of several similar objects; the paper shows 20,000 targeted training examples fix that failure.

desk verdict Useful new benchmark for referring-expression editing, but the '20K triplets beat millions' headline is undercut by the MagicBrush mixing and pretrained initialization. read the letter →

arxiv 2506.03448 v1 pith:IPIJJVFV submitted 2025-06-03 cs.CV

Bimsara Pathiraja , Maitreya Patel , Shivam Singh , Yezhou Yang , Chitta Baral This is my paper
classification cs.CV
keywords referringexpressionimageeditinginstruction-basedsyntheticdatagenerationRefEdit-BenchCOCOlocalizedefficiencydiffusionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Instruction-based image editors follow commands easily when the image has one salient object, but they falter when the command must pick out one instance among several similar ones, such as "the chef cutting vegetables" or "the rightmost person." The paper argues that this failure is a data problem, not just a model-size problem, and that a small, carefully constructed set of training examples can fix it. It introduces RefEdit-Bench, a benchmark built from real-world RefCOCO images with easy and hard splits, and RefEdit, a model fine-tuned on only 20,000 synthetic editing triplets. The paper reports that RefEdit outperforms baselines trained on millions of examples on referring-expression editing, improves standard editing benchmarks, and is preferred by human raters. If correct, the result says targeted synthetic data can substitute for brute-force data scale in instruction-based image editing.

What carries the argument

The load-bearing mechanism is the synthetic data generation pipeline, whose key idea is to generate the textual editing case and the image together from one template. GPT-4o writes an image prompt containing two or more instances of the same object distinguished by color, item, or activity, and also writes the edited prompt, editing instruction, target object, a referring expression, and a more descriptive version of that expression. FLUX renders the image, Grounded SAM converts the descriptive expression into a mask over exactly the referred instance, and FlowChef (for color, object, texture, and add edits) or Inpaint Anything (for removals) produces the edited image from the masked region. Assembling pairs this way guarantees that every training instruction has a uniquely identifiable referent and a localized ground-truth edit, which is what the released RefEdit-Data provides.

What would settle it

Take a random sample of the released 20,000 training triplets and have independent annotators mark each as correct or wrong on three criteria: the referring expression identifies exactly one intended object, the Grounded SAM mask covers that object and nothing else, and the edited image changes only inside the mask. If wrong triplets are common, the claimed advantage is not evidence for referring-expression training.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the referring-expression capability of instruction-based editors is bottlenecked by training data, not by backbone capacity. Existing instruction-editing datasets almost always contain a single dominant object per image, so models learn to edit "the object" and have no practice resolving expressions that identify one instance among several similar ones. The authors' synthetic pipeline generates 20,000 training triplets in which images deliberately contain several similar instances, and each instruction's referring expression uniquely identifies the target; the mask isolates that target, and the edit changes only it. Fine-tuning InstructPix2Pix and an SD3-based editor on this data, mixed with existing MagicBrush data, yields a model that the paper shows beats million-example baselines on RefEdit-Bench and PIE-Bench and wins human A/B comparisons. The paper frames this as evidence that quality-focused synthetic data, not scale alone, is what unlocks complex real-world editing.

Load-bearing premise

The advantage rests on the synthetic pipeline producing correct training pairs: each instruction's referring expression truly picks out one intended object, the mask covers exactly that object, and the edited image changes nothing outside it.

Editorial extensions

If this is right

  • A model trained on 20,000 targeted referring-expression triplets can beat instruction-editing models trained on millions of examples, on both the new benchmark and PIE-Bench.
  • The easy/hard split of RefEdit-Bench provides a reusable probe: methods that score well on generic editing benchmarks drop sharply on referring-expression tasks, so the benchmark isolates a distinct capability.
  • Fine-tuning a larger SD3-based editor on the same synthetic data pushes performance further, suggesting the data helps across backbone scales rather than only small models.
  • Mixing MagicBrush data with RefEdit-Data preserves general editing ability, so the synthetic dataset works as a supplement rather than a replacement for existing instruction-editing data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If targeted data composition is the active ingredient, the same pipeline should transfer to other referring-expression benchmarks, and the performance gap between RefEdit and generic editors should widen as the number of visually similar entities per image grows.
  • Editorial inference: The reported results do not isolate which pipeline stage matters; ablating the LLM-expression generator, the masker, and the editing model on a fixed evaluation subset would reveal whether the gains come from instruction diversity, mask precision, or edit fidelity.
  • Editorial inference: Because RefEdit-Bench scores are produced by an MLLM, a natural stress test is to re-rank the same model outputs with different MLLM judges and prompt templates to confirm the benchmark measures referring-expression skill rather than evaluator preference.
  • Editorial inference: Separating edit leakage from semantic correctness would be a useful extension; the reported overall score combines instruction alignment and perceptual quality, so it does not by itself say whether edits stay inside the referred object.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces RefEdit-Bench, a 200-image benchmark for instruction-based image editing with referring expressions, split into Easy and Hard categories. It also proposes a synthetic data pipeline: GPT-4o generates the textual components (image prompts, editing instructions, referring expressions), FLUX renders images, Grounded SAM produces masks, and FlowChef/Inpaint Anything create the edited images, yielding roughly 20,000 synthetic editing triplets. The authors fine-tune InstructPix2Pix (SD1.5) and UltraEdit (SD3) on a combination of these synthetic triplets and MagicBrush data, producing RefEdit and RefEdit-SD3. They evaluate on RefEdit-Bench and PIE-Bench using GPT-4o-based VIEScore and human A/B tests, reporting improvements over prior instruction-based models and qualitative comparisons with closed-source methods.

Significance. If the central claim were established—that roughly 20,000 synthetic triplets suffice to beat million-scale training for referring-expression editing—this would be a valuable efficiency result. The benchmark itself addresses a real gap, since existing editing datasets and evaluations focus mostly on single prominent objects. The data-generation pipeline is automated and the authors release data and checkpoints, which supports reproducibility. However, the headline claim is currently undercut by the training protocol and evaluation choices; with proper ablations and independent metrics, the benchmark and pipeline could still be a solid contribution.

major comments (4)
  1. [Abstract / Section 3.3] The abstract claims RefEdit is 'trained on only 20,000 editing triplets', but Section 3.3 states that MagicBrush data is combined with the generated data, yielding 30K combined training samples, and that both variants start from pretrained checkpoints (InstructPix2Pix and UltraEdit). Therefore the 20K synthetic set is neither the sole training signal nor the origin of the model's capability. The comparison of RefEdit-SD3 with UltraEdit is especially confounded: RefEdit-SD3 is UltraEdit fine-tuned on additional data. An ablation that removes MagicBrush data, or that trains from a randomly initialized backbone, is required to support the 'small data beats millions' claim; without it, the paper's headline claim is not established.
  2. [Section 5.1 / Section 3.2] Section 5.1 uses VIEScore with GPT-4o as the judge, while Section 3.2 uses GPT-4o to generate the synthetic training instructions and descriptive referring expressions. This creates a potential bias: the trained model may be rewarded for matching GPT-4o's linguistic style rather than for the actual semantic edit. In addition, the 'Modified VIEScore' for RefEdit-Bench incorporates the ground-truth mask to extract the region of interest, which may remove the referring-expression localization component from the metric. The paper should report evaluations with an independent judge (e.g., a different MLLM or human ratings with inter-annotator agreement) and should report both full-image and masked-ROI scores.
  3. [Table 4 / Figure 6] Table 4 reports VIEScore margins of 5.21 vs 5.12 (RefEdit vs OmniEdit) and 5.70 vs 5.12 (RefEdit-SD3 vs OmniEdit) with no error bars, repeated runs, or significance tests. The human evaluation in Figure 6 reports only percentages, without the number of annotators, their recruitment, or agreement measures. Given the small margins and the self-constructed benchmark, the 'state-of-the-art' and 'significantly outperforms' claims are not statistically supported.
  4. [Section 3.2 / Figure 3] The data-generation pipeline has no described quality filtering: GPT-4o may generate ambiguous expressions, Grounded SAM may produce incorrect masks, and FlowChef/Inpaint Anything may fail to apply localized edits. If any component fails systematically, the 20K triplets could teach incorrect input-output mappings. The paper should provide a quality audit (e.g., manual inspection of a random subset, mask-edit overlap statistics, or filtering heuristics) and report how many generated triplets were discarded.
minor comments (4)
  1. [Section 3.3] The text says training follows Eq. (1), but the InstructPix2Pix objective is given in Eq. (2); please clarify which loss is actually used.
  2. [Section 5.1] The description of 'Modified VIEScore' is ambiguous: it should state explicitly whether both SC and PQ are computed on the masked region or on the full edited image with the mask used only for prompting.
  3. [References] References [5] and [6] are duplicates (the InstructPix2Pix paper), as are [45] and [46] (Inpaint Anything); please consolidate them.
  4. [Table 1] The meaning of the numeric entries in the 'Use of referring expressions?' column is unclear; please clarify what the values represent.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity by construction; the small-data claim is an experimental-support concern, not a derivation that reduces to its inputs.

full rationale

The paper's central derivation is the synthetic data pipeline (Section 3.2), followed by standard diffusion fine-tuning (Section 3.3, Eq. 2), with evaluation on a RefCOCO-derived benchmark whose instructions are manually crafted (Section 4), plus PIE-Bench and human A/B testing (Figure 6). I checked for steps where an output is defined in terms of an input or where a fitted parameter is relabeled as a prediction, and found none. The 20K synthetic triplets are training data, not fitted parameters; the benchmark edit instructions are human-written; and human preference results provide external evidence independent of the GPT-4o-based VIEScore. The use of the authors' own FlowChef as an editing tool is a methodological choice, not a self-justifying uniqueness theorem, and it does not force the measured outcomes. The discrepancy between the abstract's 'only 20,000 editing triplets' and Section 3.3's statement that MagicBrush data is combined with the generated data to yield 30K total samples is a real concern about whether the headline comparison isolates the contribution of the 20K set. Similarly, using GPT-4o both to generate training instructions and to score editing success via VIEScore raises a possible evaluation-alignment concern. However, both are issues of experimental support and reporting, not circularity: no equation, definition, or fitted quantity makes the reported scores equivalent to the training input by construction. Under the hard rule that circularity must be exhibited as a specific reduction, the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

No physics-style invented entities are introduced; the model and benchmark are engineering artifacts. The ledger therefore focuses on hand-chosen hyperparameters and the load-bearing domain assumptions about data quality, mask accuracy, edit faithfulness, metric validity, and synthetic-to-real transfer.

free parameters (3)
  • Combined training data size = 30K triplets (20K RefEdit-Data + 10K MagicBrush)
    The abstract claims 'only 20,000 editing triplets', but Section 3.3 says training uses 30K combined data with MagicBrush. The reported performance depends on this hand-chosen mix.
  • Training schedule for SD1.5 variant = 24 epochs
    Chosen to avoid overfitting; no ablation is shown for epoch count, so the reported result depends on this unexamined setting (Section 3.3).
  • Training iterations for SD3 variant = 6000 iterations
    Hand-chosen schedule with no sensitivity analysis; the SD3 variant's results depend on this setting (Section 3.3).
assumptions (6)
  • domain assumption Grounded SAM masks accurately localize objects described by GPT-4o-generated descriptive referring expressions.
    Used to define edit regions in the synthetic data pipeline; mask errors directly corrupt training triplets (Section 3.2, Figure 3).
  • domain assumption FlowChef and Inpaint Anything produce faithful edits localized to the mask.
    The edited images serve as training targets; if pseudo-edits leak or fail, the model learns incorrect input-output pairs (Section 3.2, Figure 3c).
  • domain assumption RefCOCO ground-truth masks are correct for benchmark ROI extraction.
    Modified VIEScore uses ground-truth masks to crop regions; incorrect masks would invalidate the metric (Section 5.1).
  • domain assumption GPT-4o-generated prompts, instructions, and expressions are semantically correct and diverse.
    The 20K training texts are synthesized without automatic verification; the pipeline assumes no systematic hallucinations (Section 3.2).
  • domain assumption VIEScore with GPT-4o is a valid proxy for human judgment of editing success.
    Primary quantitative results depend on this metric; human evaluation covers only 400 pairs with no agreement statistics (Section 5.1, Figure 6).
  • domain assumption FLUX-generated synthetic images are representative enough of real multi-entity scenes for transfer.
    Training images are synthetic while evaluation is on real RefCOCO images; the paper assumes transfer without domain-shift analysis (Sections 3.2 and 4).

how reviews work

0 comments
Cite this review

Pith. "Pith review of RefEdit: A Benchmark and Method for Improving Instruction-based Image Editing Model on Referring Expressions." pith.science (2026). https://pith.science/paper/IPIJJVFV

@misc{pith2026250603448,
  author       = {Pith},
  title        = {Pith review of: RefEdit: A Benchmark and Method for Improving Instruction-based Image Editing Model on Referring Expressions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IPIJJVFV}},
  note         = {Machine review of arXiv:2506.03448}
}
read the original abstract

Despite recent advances in inversion and instruction-based image editing, existing approaches primarily excel at editing single, prominent objects but significantly struggle when applied to complex scenes containing multiple entities. To quantify this gap, we first introduce RefEdit-Bench, a rigorous real-world benchmark rooted in RefCOCO, where even baselines trained on millions of samples perform poorly. To overcome this limitation, we introduce RefEdit -- an instruction-based editing model trained on our scalable synthetic data generation pipeline. Our RefEdit, trained on only 20,000 editing triplets, outperforms the Flux/SD3 model-based baselines trained on millions of data. Extensive evaluations across various benchmarks demonstrate that our model not only excels in referring expression tasks but also enhances performance on traditional benchmarks, achieving state-of-the-art results comparable to closed-source methods. We release data \& checkpoint for reproducibility.

Figures

Figures reproduced from arXiv: 2506.03448 by the authors.

Figure 1
Figure 1. RefEdit is a referring expression-based image editing benchmark and a finetuned model. Our proposed RefEdit model can [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Three samples from each of Easy and Hard categories and the overlayed mask of the interested object are shown. Datasets Automatic generated? #Edits #Editing types Multiple similar entities? Avg. #similar entities Use of referring expressions? InstructPix2Pix [6] 450K 4 1 MagicBrush [47] 10K 7 1 HQ-Edit [15] 200K 7 1 UtraEdit [49] 4M 9+ 1 InstructDiffusion [11] 425K 6 1 HIVE [48] 1.1M 5+ 1 OmniEdit [43] 1.2M 7 1 RefE… view at source ↗
Figure 4
Figure 4. Two samples from our training dataset. From left to [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: An overview of keywords in edit instructions. The inner [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Human preference analysis for image editing, [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results on image editing. The top 3 samples are from the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparisons with closed-source methods. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Additional training samples [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Qualitative results on image editing. The top 4 samples are from the [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: SC score prompt masked version. You are a professional digital artist. You will have to evaluate the effectiveness of the AI-edited image. Two images will be provided: The first being the original image selected from COCO dataset and the second being an AI edited vers…
Figure 12
Figure 12. Figure 12: PQ score version [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 24 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. 2

    arXiv 2023

  2. [2]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 18187–18197. IEEE, 2022. 2

    work page 2022

  3. [3]

    Blended latent diffusion.ACM Transactions on Graphics, 42(4):1–11,

    Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion.ACM Transactions on Graphics, 42(4):1–11,

    work page

  4. [4]

    Sega: Instructing text-to-image models using semantic guidance

    Manuel Brack, Felix Friedrich, Dominik Hintersdorf, Lukas Struppek, Patrick Schramowski, and Kristian Kersting. Sega: Instructing text-to-image models using semantic guidance. Advances in Neural Information Processing Systems, 36: 25365–25389, 2023. 2

    work page 2023

  5. [5]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 2

    work page 2023

  6. [6]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions,

  7. [7]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz L...

    2020

  8. [8]

    Diffedit: Diffusion-based seman- tic image editing with mask guidance.arXiv preprint arXiv:2210.11427, 2022

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion-based seman- tic image editing with mask guidance.arXiv preprint arXiv:2210.11427, 2022. 2

    arXiv 2022

Show all 49 references
  1. [9]

    Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021. 1

    2021

  2. [10]

    Scaling rectified flow trans- formers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim En- tezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. InForty-first International Conference on Machi...

    2024

  3. [11]

    Instructdiffusion: A generalist mod- eling interface for vision tasks, 2023

    Zigang Geng, Binxin Yang, Tiankai Hang, Chen Li, Shuyang Gu, Ting Zhang, Jianmin Bao, Zheng Zhang, Han Hu, Dong Chen, and Baining Guo. Instructdiffusion: A generalist mod- eling interface for vision tasks, 2023. 2, 3

    2023

  4. [12]

    Prompt-to-prompt im- age editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022. 1, 2, 6

    2022 arXiv

  5. [13]

    Prompt-to-prompt image editing with cross attention control, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control, 2022. 2

    2022

  6. [14]

    Denoising diffu- sion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  7. [15]

    Hq-edit: A high-quality dataset for instruction-based image editing

    Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang, Yuyin Zhou, and Cihang Xie. Hq-edit: A high-quality dataset for instruction-based image editing. arXiv preprint arXiv:2404.09990, 2024. 3

    2024 arXiv

  8. [16]

    Direct inversion: Boosting diffusion-based editing with 3 lines of code.arXiv preprint arXiv:2310.01506, 2023

    Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code.arXiv preprint arXiv:2310.01506, 2023. 2, 6

    2023 arXiv

  9. [17]

    Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything.arXiv:2304.02643, 2023. 5

    2023 arXiv

  10. [18]

    Viescore: Towards explainable metrics for conditional image synthesis evaluation, 2024

    Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explainable metrics for conditional image synthesis evaluation, 2024. 7

    2024

  11. [19]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2023. 4, 5

    2023

  12. [20]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxim- ilian Nickel, and Matthew Le. Flow matching for genera- tive modeling. InThe Eleventh International Conference on Learning Representations, 2023. 1

    2023

  13. [21]

    Referring image editing: Object-level image editing via referring expressions

    Chang Liu, Xiangtai Li, and Henghui Ding. Referring image editing: Object-level image editing via referring expressions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13128–13138, 2024. 3

    2024

  14. [22]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023. 5

    2023 arXiv

  15. [23]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. InThe Eleventh International Conference on Learning Representations, 2023. 1

    2023

  16. [24]

    On llms-driven synthetic data generation, curation, and evaluation: A survey.arXiv preprint arXiv:2406.15126, 2024

    Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. On llms-driven synthetic data generation, curation, and evaluation: A survey.arXiv preprint arXiv:2406.15126, 2024. 2

    2024 arXiv

  17. [25]

    Mag-edit: Localized image editing in complex scenar- ios via mask-based attention-adjusted guidance

    Qi Mao, Lan Chen, Yuchao Gu, Zhen Fang, and Mike Zheng Shou. Mag-edit: Localized image editing in complex scenar- ios via mask-based attention-adjusted guidance. InProceed- ings of the 32nd ACM International Conference on Multime- dia, pages 6842–6850, 2024. 2

    2024

  18. [26]

    Sdedit: Guided image synthesis and editing with stochastic differential equations

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073, 2021. 2

    2021 arXiv

  19. [27]

    Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models.arXiv preprint arXiv:2305.16807, 2023

    Daiki Miyake, Akihiro Iohara, Yu Saito, and Toshiyuki Tanaka. Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models.arXiv preprint arXiv:2305.16807, 2023. 2

    2023 arXiv

  20. [28]

    Null-text inversion for editing real im- ages using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real im- ages using guided diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 6038–6047, 2023. 1, 2

    2023

  21. [29]

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M ˛ adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex ...

    2024

  22. [30]

    λ-eclipse: Multi-concept personalized text-to-image diffusion models by leveraging clip latent space.ArXiv, abs/2402.05195, 2024

    Maitreya Patel, Sangmin Jung, Chitta Baral, and Yezhou Yang. λ-eclipse: Multi-concept personalized text-to-image diffusion models by leveraging clip latent space.ArXiv, abs/2402.05195, 2024. 1

    2024 arXiv

  23. [31]

    Eclipse: A resource-efficient text-to-image prior for image generations

    Maitreya Patel, Changhoon Kim, Sheng Cheng, Chitta Baral, and Yezhou Yang. Eclipse: A resource-efficient text-to-image prior for image generations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9069–9078, 2024. 1

    2024

  24. [32]

    Triplet- clip: Improving compositional reasoning of clip via synthetic vision-language negatives.Advances in neural information processing systems, 37:32731–32760, 2024

    Maitreya Patel, Naga Sai Abhiram Kusumba, Sheng Cheng, Changhoon Kim, Tejas Gokhale, Chitta Baral, et al. Triplet- clip: Improving compositional reasoning of clip via synthetic vision-language negatives.Advances in neural information processing systems, 37:32731–32760, 2024. 2

    2024

  25. [33]

    Metaxas, and Yezhou Yang

    Maitreya Patel, Song Wen, Dimitris N. Metaxas, and Yezhou Yang. Steering rectified flow models in the vector field for controlled image generation, 2024. 1, 2, 4, 5

    2024

  26. [34]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 2

    2023 arXiv

  27. [35]

    Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    arXiv

  28. [36]

    Grounded sam: Assembling open-world models for diverse visual tasks, 2024

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024. 2, 4, 5

    2024

  29. [37]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2

    2022

  30. [38]

    Pho- torealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Pho- torealistic text-to-image diffusion models with deep language understanding.Advances in neural informatio...

    2022

  31. [39]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. InThe Eleventh International Conference on Lear...

    2023

  32. [40]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502,

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502,

    2010 arXiv

  33. [41]

    Ledits: Real image editing with ddpm inversion and semantic guidance.arXiv preprint arXiv:2307.00522, 2023

    Linoy Tsaban and Apolinário Passos. Ledits: Real image editing with ddpm inversion and semantic guidance.arXiv preprint arXiv:2307.00522, 2023. 2

    2023 arXiv

  34. [42]

    Edict: Exact diffusion inversion via coupled transformations

    Bram Wallace, Akash Gokul, and Nikhil Naik. Edict: Exact diffusion inversion via coupled transformations. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22532–22541, 2023. 2

    2023

  35. [43]

    Omniedit: Building image edit- ing generalist models through specialist supervision.arXiv preprint arXiv:2411.07199, 2024

    Cong Wei, Zheyang Xiong, Weiming Ren, Xinrun Du, Ge Zhang, and Wenhu Chen. Omniedit: Building image edit- ing generalist models through specialist supervision.arXiv preprint arXiv:2411.07199, 2024. 2, 3, 7

    2024 arXiv

  36. [44]

    Berg, and Tamara L

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C. Berg, and Tamara L. Berg. Modeling context in referring expres- sions, 2016. 2, 6

    2016

  37. [45]

    Inpaint anything: Seg- ment anything meets image inpainting.arXiv preprint arXiv:2304.06790, 2023

    Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, and Zhibo Chen. Inpaint anything: Seg- ment anything meets image inpainting.arXiv preprint arXiv:2304.06790, 2023. 2

    2023 arXiv

  38. [46]

    Inpaint anything: Segment anything meets image inpainting, 2023

    Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, and Zhibo Chen. Inpaint anything: Segment anything meets image inpainting, 2023. 5

    2023

  39. [47]

    Magicbrush: A manually annotated dataset for instruction- guided image editing, 2024

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction- guided image editing, 2024. 2, 3, 5

    2024

  40. [48]

    Hive: Harnessing human feedback for instructional visual editing

    Shu Zhang, Xinyi Yang, Yihao Feng, Can Qin, Chia-Chih Chen, Ning Yu, Zeyuan Chen, Huan Wang, Silvio Savarese, Stefano Ermon, et al. Hive: Harnessing human feedback for instructional visual editing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2024

  41. [49]

    score" : [...],

    Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. Ultraedit: Instruction-based fine-grained image editing at scale.Advances in Neural Information Pro- cessing Systems, 37:3058–3093, 2025. 2, 3 Let t...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.