Towards Scalable Human-aligned Benchmark for Text-guided Image Editing

Dongsoo Shin; Eugene Baek; Joonseok Lee; Kihyun Kim; Suho Ryu

REVIEW 4 major objections 6 minor 36 references

Towards Scalable Human-aligned Benchmark for Text-guided Image Editing

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A fully automated benchmark for text-guided image editing, built on 49,840 queries and five weighted visual scores, reproduces human quality judgments with a total-score correlation of about 0.71 and per-criterion correlations typically…

desk verdict HATIE is a genuinely useful benchmark resource, but its central human-alignment claim rests on only six models, and the paper should either add more or soften the conclusion. read the letter →

arxiv 2505.00502 v1 pith:I7LC7EIG submitted 2025-05-01 cs.CV

Suho Ryu , Kihyun Kim , Eugene Baek , Dongsoo Shin , Joonseok Lee This is my paper · ORCID

classification cs.CV

keywords text-guidedimageeditingbenchmarkhuman-alignedevaluationautomateddiffusionmodelsuserstudycorrelationinstancesegmentationmetrics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that subjective evaluation of text-guided image editing can be automated without losing human alignment. It introduces HATIE, a benchmark of 18,226 images and 49,840 edit queries covering object addition, removal, replacement, attribute change, resizing, background change, and style change. Instead of relying on any single model-based score, it combines five criteria—object fidelity, background fidelity, object consistency, background consistency, and image quality—and fits the combination weights to pairwise human preference judgments. On held-out user study questions, the resulting total score tracks human winning rates with Pearson correlation about 0.71, and each individual criterion reaches 0.82–0.93. If correct, this gives the field a reproducible, labour-free way to compare editing models and tune their hyperparameters.

What carries the argument

The load-bearing object is the weighted five-criteria score: object fidelity, background fidelity, object consistency, background consistency, and image quality, aggregated as $\sigma_{\text{Total}}=\sum_{x\in X}w_x\sigma_x$ with weights fitted to user-study winning rates. Task-specific workflows decide which criteria apply; for object-centric edits an instance segmentation model crops the target object, so fidelity is measured on the object alone and consistency compares the object or background across original and edited images. The component metrics include CLIP alignment, detection confidence, LPIPS, DINO similarity, L2 distance, and rule-based position and size scores. The argument that the whole is human-aligned rests on fitting the weights to pairwise human comparisons and then verifying correlation on set-aside questions.

What would settle it

Compute the detection-failure rate separately for each editing model and correlate it with human preference scores on the same images; if models that produce unusual or degraded object appearances fail detection more often, the extreme-score replacement will depress their HATIE scores without depressing human ratings. A direct check is to rerun the benchmark after excluding all images where the segmenter fails and see whether model rankings change materially.

Watch

Extended reading notes

Core claim

The paper's central claim is that a structured combination of off-the-shelf metrics, organized into five categories and weighted by human preference data, can serve as a proxy for human evaluation of text-guided image editing. The benchmark defines fidelity as whether the instructed change happened, consistency as whether unintended content was preserved, and image quality as the distributional realism of outputs; instance segmentation separates both fidelity and consistency into object and background components. Each component score is a convex combination of simpler signals such as CLIP text-image alignment, detection confidence, LPIPS and DINO similarities, L2 distance, and rule-based position and size scores. The weights are selected by grid search to maximize Pearson correlation between model winning rates computed from user studies and from the automated scores. On a held-out user study test set, the paper reports correlations of 0.82–0.93 for the four component criteria and about 0.71 for the total score, and shows that each single conventional metric correlates much worse with human judgment.

Load-bearing premise

The load-bearing assumption is that when the instance segmentation model fails to detect a target object in 16.01% of evaluation images, the failure is caused by the object being genuinely absent or by detector limitations that are uncorrelated with editing quality; on that basis the pipeline replaces missing detections with extreme fidelity scores (0 for most tasks, 1 for removal).

Editorial extensions

If this is right

Editing models can be compared on a common 49,840-query benchmark without running new user studies, because the automated total score reproduces human preferences on held-out questions.
Model rankings can be decomposed by edit type and object class, exposing task-specific strengths such as Imagic on attribute and background changes or InstDiff on resizing.
The benchmark is sensitive enough to guide hyperparameter choice: varying edit strength produces monotone fidelity and consistency trends with a clear optimum around the fidelity-consistency trade-off.
Individual conventional metrics are insufficient: none of CLIP alignment, detection rate, LPIPS, DINO, L2, position or size scores, or FID alone matches human judgment as well as the combined score.
Because the pipeline is fully automated and does not depend on external API services, evaluations can be reproduced and extended to new models and datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If segmenter failures are not neutral—models that produce less canonical object appearances may be missed more often—the 16.01% failure rate with extreme-score replacement could bias rankings; this is testable by comparing HATIE rankings with and without failure cases.
The weight-fitting procedure uses only six description-based models and a small participant pool, so weights optimized on that model set may not be optimal for future models with different failure modes; periodic re-fitting or uncertainty reporting would strengthen the human-alignment claim.
The same five-criteria structure could transfer to open-vocabulary editors by replacing the fixed-class segmenter with an open-vocabulary detector, extending the benchmark beyond the 76 COCO classes.
Because the user study asks binary pairwise choices by majority vote, the fitted weights inherit the granularity of those comparisons; fine-grained preference strengths, such as ratings, might change the optimal weights.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a solid, well-documented benchmark paper that the editing subfield will likely want to use. The dataset scale and the fully automated pipeline are real contributions beyond EditVal and TEdBench. The careful task-specific workflows, the filtering, and the release of code/data all look responsible. Credit where due: the held-out correlations for the component criteria (r 0.82–0.93) are real evidence that the weighted combination tracks human preference better than the individual off-the-shelf metrics, which Table 3 shows to be weak. That is the paper's strongest empirical point.

The soft spots are mostly about the breadth of the alignment claim. The alignment is computed on winning rates over only six description-based models. With n=6, the total-score r=0.714 is not significant (p≈0.11), and the 'held-out' split holds out questions, not models: the weights were fit to these same six models, so stable model-level differences can inflate the test correlation. I don't think this sinks the benchmark—the resource is still useful—but the sentence claiming r>0.8 for every criterion is inaccurate (total is 0.714), and the Object Fidelity outlier exclusion needs the inclusive value reported. Also, the 16% detector failure with extreme-score substitution is a plausible bias channel; the appendix's example images are reassuring that failures are often the object truly being absent, but that's not a formal bias test.

Bottom line: as a benchmark and evaluation pipeline, it deserves a serious referee and probably publication after revision. The authors should either add more editing models to the alignment study (even a couple of instruction-based ones) or explicitly limit the human-alignment claim to the tested model family. Report the total-score r accurately, show the detector-failure robustness analysis, and this becomes a citation-worthy resource. I'd bring it to reading group as an example of a well-executed benchmark effort.

Referee Report

4 major / 6 minor

Summary. The paper introduces HATIE, a large-scale benchmark and automated evaluation pipeline for text-guided image editing. The benchmark is built on GQA, yielding 18,226 images and 49,840 editing queries across object-centric and non-object-centric tasks, and it evaluates edited images along five criteria: Image Quality, Object Fidelity, Background Fidelity, Object Consistency, and Background Consistency. The criteria are computed from CLIP, LPIPS, DINO, L2, detection confidence, and rule-based size/position scores, and are aggregated by convex combination weights fitted to a user study conducted on six description-based models. The paper reports Pearson correlations between model winning rates from user study and HATIE on a held-out split of user-study questions, compares against conventional metrics, and provides benchmark tables for nine editing models. The main claim is that HATIE provides a fully automated, human-aligned evaluation that can be used to assess existing and new text-guided image editing models.

Significance. If the human-alignment claim were established for unseen editing models, HATIE would be a valuable contribution to the community: it provides a much larger and more diverse query set than existing benchmarks, ships code, uses a structured multi-metric aggregation, and includes extensive appendix documentation. The comparison with conventional metrics and the bootstrapped benchmark scores are strengths. However, the current evidence for human alignment is weaker than claimed: the correlations are computed on only six models, the held-out split does not hold out models, the total-score correlation is below the reported 0.8 threshold, and the object-fidelity correlation excludes an outlier. The significance of the work therefore depends on additional validation of transfer to novel models and on corrected, uncertainty-aware reporting of the alignment results.

major comments (4)

[Section 4.2, Table 2, Fig. 8] The text states that 'Pearson's coefficient r over 0.8 throughout every criterion', but Table 2 reports Total Score r = 0.7143, which is below 0.8. In addition, Fig. 8 shows that the Object Fidelity correlation of 0.8208 is computed after excluding a marked outlier, so that coefficient is based on five rather than six model points. With only six models, the total-score correlation is not statistically significant at the 0.05 level, and even the component correlations have wide confidence intervals. Please report the exact sample size after outlier exclusion, provide confidence intervals or p-values for all reported correlations, and revise the abstract and Section 6 claims so they accurately reflect the observed total-score correlation.
[Sections 3.4 and 4.2] The held-out split in Section 4.2 holds out user-study questions, not editing models. The weights in Eqs. (2)-(5) are fitted to winning rates from the same six description-based models that are then used to compute the test-set correlations, so the experiment demonstrates transfer across questions for those six models but does not demonstrate transfer to a new editing model. Since the introduction and abstract position HATIE as a scalable evaluation method for 'existing and new image editing models', the paper should provide leave-one-model-out cross-validation or a separate alignment study with a held-out model. Without such evidence, the human-alignment claim must be restricted to the six models studied.
[Appendix D.2 and Appendix C.1] The evaluation relies on instance segmentation, but Appendix D.2 reports a 16.01% detection failure rate, and the task-specific workflows in Appendix C.1 assign extreme scores when the target object is not detected (e.g., Object Fidelity 0 for addition/replacement/resizing/attribute-change tasks, and 1 for removal tasks). Appendix D.2 argues qualitatively that most failures are due to absence of the target object, but no quantitative evidence is provided that detection failures are uncorrelated with editing quality. If models that produce less canonical or softer object appearances are more likely to trigger detection failures, the imputed extreme scores can bias model rankings independently of human judgment. Please report per-model detection failure rates and perform a sensitivity analysis that recomputes the alignment correlations and benchmark rankings with detection-failure cases excluded or with alternative imputation.
[Section 3.4 and Appendix C.3] The weight fitting procedure uses grid search to maximize Pearson correlation on model winning rates from only six models, and the training-set correlations in Table I reach 1.0 for Total Score and Background Consistency. This does not establish that the fitted weights are stable or that the aggregation generalizes. Please report the fitted weight values actually used, the number of grid points evaluated, and the stability of the weights under bootstrap resampling (of both questions and models). Without this information, the Total Score weights in Eq. (5) may be overfit to the six-model sample.

minor comments (6)

[Section 3.3, Eq. (2)] In the paragraph following Eq. (2), the text says 'The overall Object Fidelity score σOC' but the symbol being defined is σOF; please correct this typo.
[Section 3.4] The relation between 4,050 sampled images, 2,025 queries, 2,700 training images from 1,350 queries, and 1,350 test images from 675 queries is not transparent given that six models generate one output per query; please clarify how many model outputs per query are sampled and how the pairwise comparisons are constructed.
[Table 2] The symbols ρ and τ are used without definition; please define them as Spearman's rank and Kendall's tau, respectively, in the caption or in the text before first use.
[Fig. 8] For Object Fidelity, please report the correlation both with and without the marked outlier, and state which model and query produced the outlier, so the reader can judge whether its exclusion is justified.
[Appendix C.2] The thresholds r1, r2, and r3 in the size score are described as empirically set, but their values are not given; please report the actual values so that the rule-based size score is reproducible.
[Appendix D.4 and Fig. X] The claim that alignment is tested on an 'unseen new dataset' refers to new images from ImageNet but still uses the same six editing models; please rephrase to avoid implying generalization to novel models, and report the number of queries and models used in Fig. X.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the alignment weights are fitted on a training split of the user study and the human-alignment claim is tested on set-aside queries, so the result is not forced by construction.

full rationale

The central claim is that HATIE's aggregated metric scores correlate with human perception. The weights in Eqs. (2)-(5) are fitted by maximizing Pearson correlation between model winning rates from a user study and winning rates computed from the metrics, but only on 1,350 training queries; alignment is then evaluated on 1,350 held-out images / 675 queries not used for fitting (Sec. 3.4 and Sec. 4.2). This is a genuine held-out evaluation, not a restatement of the fitting target. The paper does not invoke a self-citation chain or a uniqueness theorem; its metric components (CLIP, LPIPS, DINO, detection, position/size) are standard external tools, and the benchmark is self-contained. The detector-failure substitution in Appendix D.2 is a validity risk rather than a circular step, because the dataset filtering and extreme-score rules are stated explicitly and the detector behavior is not defined in terms of the final human-alignment score. Statistical concerns about n=6 models and question-level versus model-level hold-out are generalizability and power issues, not circularity by construction. Therefore no circular step is exhibited, and the correct score is 0.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The central claim rests on proxy metrics (CLIP, DINO, LPIPS, FID, detection) and on a small user study. The free parameters are the weights in the score combinations, fitted by grid search to the user study, plus several hand-chosen constants. The evaluation is only as valid as these proxies and as the generality of the user study.

free parameters (9)

Object Fidelity weights (Eq. 2): w_OF_clip, w_OF_det, w_OF_size = fitted by grid search (step 0.01) on user study training set
Convex combination of CLIP alignment, detector confidence, and size score; fitted to maximize Pearson correlation with human winning rates.
Object Consistency weights (Eq. 3): w_OC_lpips, w_OC_dino, w_OC_l2, w_OC_pos, w_OC_size = fitted by grid search on user study
Combines LPIPS, DINO, L2, position and size consistency; weights fitted to human preferences.
Background Consistency weights (Eq. 4): w_BC_lpips, w_BC_dino, w_BC_l2 = fitted by grid search on user study
Combines LPIPS, DINO, L2 for background; weights fitted to human preferences.
Total Score weights (Eq. 5): w_IQ, w_OF, w_BF, w_OC, w_BC = fitted by grid search on user study
Aggregates the five criteria; weights fitted to maximize correlation with overall quality human ratings.
Task-specific Object Fidelity weights for addition/replacement/removal (Eq. 6-8): w_OF_clip,c, w_OF_clip,r, w_OF_det = fitted by grid search on user study
Used in Object Addition, Replacement, and Removal evaluation workflows.
Attribute Change and Style Change weights (Eq. 10-16): w'_OC_*, w''_OC_pos, w'_BC_* = fitted independently for these tasks
Additional weights for degraded and edge consistency scoring in attribute and style edits.
FID rescaling factor 25 in Eq. (1) = 25
Chosen so that the image quality score is close to 1 for state-of-the-art models; not fitted to human data.
Size score thresholds r1, r2, r3 (Appendix C.2) = not reported numerically
Empirically set boundaries for noticeable size change and maximum possible change; affect Object Resizing and Consistency scores.
Data filtering thresholds: min object area 0.5% of image, IoU > 0.5, VQA approval 4/6 = 0.5%, 0.5, 4/6
Hand-chosen criteria deciding which objects are editable and detectable; shape the benchmark composition.

assumptions (6)

domain assumption GQA scene graph annotations (object names, attributes, relations) are sufficiently accurate for generating feasible edit queries.
Invoked in Sec. 3.1; the whole query generation pipeline builds on these annotations without manual verification.
domain assumption The pre-trained instance segmentation model (Detectron2) reliably detects target objects in edited images.
Invoked in Sec. 3.3 and relied on for all object fidelity and consistency scores; Appendix D.2 admits a 16.01% failure rate.
domain assumption CLIP, DINO, LPIPS, and L2 similarities capture human-perceived fidelity and consistency in edited images.
Invoked in Sec. 3.3; these proxy metrics are combined without evidence beyond the small user study.
domain assumption FID between the input and output image sets measures the image quality of individual edits.
Invoked in Sec. 3.3 Eq. (1); FID is a distribution-level metric, used here to score the quality of edited outputs.
domain assumption A user study with 24 participants and majority votes of 3 per question provides a stable ground truth for human alignment.
Invoked in Sec. 3.4; the weight fitting and validation rely on this small study.
domain assumption Model-level winning rates are an appropriate target for calibrating metric weights.
Invoked in Sec. 3.4; correlations are computed over M=6 models, so the fitted weights optimize agreement on a very small sample.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Scalable Human-aligned Benchmark for Text-guided Image Editing." pith.science (2026). https://pith.science/paper/I7LC7EIG

@misc{pith2026250500502,
  author       = {Pith},
  title        = {Pith review of: Towards Scalable Human-aligned Benchmark for Text-guided Image Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I7LC7EIG}},
  note         = {Machine review of arXiv:2505.00502}
}

read the original abstract

A variety of text-guided image editing models have been proposed recently. However, there is no widely-accepted standard evaluation method mainly due to the subjective nature of the task, letting researchers rely on manual user study. To address this, we introduce a novel Human-Aligned benchmark for Text-guided Image Editing (HATIE). Providing a large-scale benchmark set covering a wide range of editing tasks, it allows reliable evaluation, not limited to specific easy-to-evaluate cases. Also, HATIE provides a fully-automated and omnidirectional evaluation pipeline. Particularly, we combine multiple scores measuring various aspects of editing so as to align with human perception. We empirically verify that the evaluation of HATIE is indeed human-aligned in various aspects, and provide benchmark results on several state-of-the-art models to provide deeper insights on their performance.

Figures

Figures reproduced from arXiv: 2505.00502 by the authors.

**Figure 1.** An example highlighting the importance of consistency in image editing. (a) Original image (b) Edited images for a prompt “Make her smile”. The left result is more consistent with the input than the right one, better preserving her identity. Method # Images # Queries # Obj. class Auto-eval TEdBench [15] 100 100 N/A No EditBench [30] 240 720 N/A N/A GIER [28] 6,179 30,895 N/A N/A EditVal [2] 92 648 19 Partial HATIE … view at source ↗

**Figure 2.** Overview of our HATIE Benchmark. HATIE consists of an image and query dataset for editing, along with an automated evaluation pipeline for assessing editing performance. We curate a large-scale comprehensive dataset with images and corresponding editing queries, on which a model would perform text-guided image editing. Then, HATIE evaluates the edited images from 5 different aspects: Object Fidelity, Background Fide… view at source ↗

**Figure 3.** Object Class Distribution in Our Dataset. HATIE evaluates fairly by providing evenly distributed dataset. Data Filtering and Augmentation. Not every object in a scene is suitable for editing. We filter out indistinguishable objects that are too small, cropped, or occluded. Also, for automated evaluation, it is necessary for an object to be detected by object detector, which is typically trained on a dataset labeled… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Query Set Distribution. (a) Distribution of edit types in our query set, (b) Distribution of the object classes designated as the target in object-centric queries. elements remain unaltered. Therefore, we design the evaluation based on three criteria. First, we assess…

**Figure 5.** Figure 5: Evaluation Workflow Specific to Each Editing Task. See Appendix C.1 for more details. is appropriately adjusted through a rule-based formula described in Appendix C.2. The overall Object Fidelity score σ OC is a convex combination of the three metrics: \sigma ^{OF} = …

**Figure 6.** Figure 6: Aggregated HATIE scores with Varied Editing Intensity for Prompt-to-Prompt. Larger τ means a weaker edit. The overall Object and Background Consistency scores, σ OC and σ BC , are a convex combination of the corresponding metrics: \sigma ^{OC} &=&& w^{OC}_\text {lpip…

**Figure 8.** Figure 8 [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

**Figure 7.** Figure 7: Demonstration of Our Evaluation Metrics for sample images for each criteria. Tested model is InstructPix2Pix with sT ∈ {2.5, 5.0, 7.5, 10.0, 12.5}. Higher sT means stronger edit. (the intended edit intensity decreases), two fidelity scores decrease while two consistenc…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 31 canonical work pages

[1]

Blended diffusion for text-driven editing of natural im- ages

Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural im- ages. In CVPR, 2022. 2

work page 2022
[2]

EditVal: Benchmarking diffusion based text-guided image edit- ing methods

Samyadeep Basu, Mehrdad Saberi, Shweta Bhardwaj, Atoosa Malemir Chegini, Daniela Massiceti, Maziar Sanjabi, Shell Xu Hu, and Soheil Feizi. EditVal: Benchmarking diffusion based text-guided image edit- ing methods. arXiv:2310.02426, 2023. 2, 8

arXiv
[3]

InstructPix2Pix: Learning to follow image edit- ing instructions

Tim Brooks, Aleksander Holynski, and Alexei A Efros. InstructPix2Pix: Learning to follow image edit- ing instructions. In CVPR, 2023. 2, 6

work page 2023
[4]

A computational approach to edge de- tection

John Canny. A computational approach to edge de- tection. IEEE Transactions on pattern analysis and machine intelligence, 8(6):679–698, 1986. vi

work page 1986
[5]

MasaCtrl: Tuning-free mutual self-attention control for consis- tent image synthesis and editing

Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. MasaCtrl: Tuning-free mutual self-attention control for consis- tent image synthesis and editing. In ICCV, 2023. 2, 6

work page 2023
[6]

DiffEdit: Diffusion- based semantic image editing with mask guidance

Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. DiffEdit: Diffusion- based semantic image editing with mask guidance. In ICLR, 2023. 2, 8

work page 2023
[7]

Imagenet: A large-scale hierarchi- cal image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchi- cal image database. In CVPR, 2009. xi

work page 2009
[8]

The llama 3 herd of models

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv:2407.21783,

arXiv

Show all 36 references

[9]

InstructDiffusion: A gen- eralist modeling interface for vision tasks

Zigang Geng, Binxin Yang, Tiankai Hang, Chen Li, Shuyang Gu, Ting Zhang, Jianmin Bao, Zheng Zhang, Houqiang Li, Han Hu, et al. InstructDiffusion: A gen- eralist modeling interface for vision tasks. In CVPR,
[10]

Prompt-to- prompt image editing with cross attention control

Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aber- man, Yael Pritch, and Daniel Cohen-Or. Prompt-to- prompt image editing with cross attention control. In ICLR, 2023. 1, 2, 8

2023
[11]

CLIPScore: A reference-free evaluation metric for image captioning”

Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning”. In EMNLP,
[12]

GANs trained by a two time-scale update rule con- verge to a local nash equilibrium

Martin Heusel, Hubert Ramsauer, Thomas Un- terthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule con- verge to a local nash equilibrium. NIPS, 2017. 4

2017
[13]

SmartE- dit: Exploring complex instruction-based image edit- ing with multimodal large language models

Yuzhou Huang, Liangbin Xie, Xintao Wang, Ziyang Yuan, Xiaodong Cun, Yixiao Ge, Jiantao Zhou, Chao Dong, Rui Huang, Ruimao Zhang, et al. SmartE- dit: Exploring complex instruction-based image edit- ing with multimodal large language models. InCVPR,
[14]

GQA: A new dataset for real-world visual reasoning and com- positional question answering

Drew A Hudson and Christopher D Manning. GQA: A new dataset for real-world visual reasoning and com- positional question answering. In CVPR, 2019. 2

2019
[15]

Imagic: Text-based real image editing with dif- fusion models

Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with dif- fusion models. In CVPR, 2023. 1, 2, 6, 8

2023
[16]

Diffusion-based im- age translation using disentangled style and content representation

Gihyun Kwon and Jong Chul Ye. Diffusion-based im- age translation using disentangled style and content representation. In ICLR, 2023. 2, 8

2023
[17]

mPLUG: Effective and effi- cient vision-language learning by cross-modal skip- connections

Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, Hehong Chen, Guohai Xu, Zheng Cao, et al. mPLUG: Effective and effi- cient vision-language learning by cross-modal skip- connections. In EMNLP, 2022. i

2022
[18]

Microsoft COCO: Common ob- jects in context

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft COCO: Common ob- jects in context. In ECCV, 2014. 3

2014
[19]

Text-driven image editing via learnable regions

Yuanze Lin, Yi-Wen Chen, Yi-Hsuan Tsai, Lu Jiang, and Ming-Hsuan Yang. Text-driven image editing via learnable regions. In CVPR, 2024. 2

2024
[20]

SDEdit: Guided image synthesis and editing with stochastic differential equations

Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In ICLR, 2022. 1

2022
[21]

Null-text inversion for editing real images using guided diffusion models

Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In CVPR,
[22]

GLIDE: Towards photore- alistic image generation and editing with text-guided diffusion models

Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photore- alistic image generation and editing with text-guided diffusion models. In ICML, 2022. 2

2022
[23]

DINOv2: Learning robust visual fea- tures without supervision

Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fer- nandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2: Learning robust visual fea- tures without supervision. In TMLR, 2023. 5

2023
[24]

Learning transferable visual models from nat- ural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In ICML, 2021. 1

2021
[25]

Zero-shot text-to-image generation

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, 2021. 1

2021
[26]

High- resolution image synthesis with latent diffusion mod- els

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High- resolution image synthesis with latent diffusion mod- els. In CVPR, 2022

2022
[27]

Photorealistic text- to-image diffusion models with deep language under- standing

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kam- yar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text- to-image diffusion models with deep language under- standing. NeurIPS, 2022. 1

2022
[28]

A benchmark and baseline for language-driven image editing

Jing Shi, Ning Xu, Trung Bui, Franck Dernoncourt, Zheng Wen, and Chenliang Xu. A benchmark and baseline for language-driven image editing. In ACCV,
[29]

InstructEdit: Improving automatic masks for diffusion-based image editing with user instructions

Qian Wang, Biao Zhang, Michael Birsak, and Peter Wonka. InstructEdit: Improving automatic masks for diffusion-based image editing with user instructions. arXiv:2305.18047, 2023. 2

2023 arXiv
[30]

Imagen editor and EditBench: Advancing and evalu- ating text-guided image inpainting

Su Wang, Chitwan Saharia, Ceslee Montgomery, Jordi Pont-Tuset, Shai Noy, Stefano Pellegrini, Yasumasa Onoe, Sarah Laszlo, David J Fleet, Radu Soricut, et al. Imagen editor and EditBench: Advancing and evalu- ating text-guided image inpainting. In CVPR, 2023. 1, 2

2023
[31]

FreeDiff: Progressive frequency truncation for image editing with diffusion models

Wei Wu, Qingnan Fan, Shuai Qin, Hong Gu, Ruoyu Zhao, and Antoni B Chan. FreeDiff: Progressive frequency truncation for image editing with diffusion models. In ECCV, 2024. 2, 6

2024
[32]

Detectron2

Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detectron2. https : / / github . com / facebookresearch / detectron2 , 2019. 3, 4

2019
[33]

GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding

Yiqi Wu, Xiaodan Hu, Ziming Fu, Siling Zhou, and Jiangong Li. GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding. arXiv:2406.09781, 2024. 4, ii

2024 arXiv
[34]

MagicBrush: A manually annotated dataset for instruction-guided image editing

Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. MagicBrush: A manually annotated dataset for instruction-guided image editing. NeurIPS, 2024. 1, 2, 8

2024
[35]

a photo of a crimson cat

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable ef- fectiveness of deep features as a perceptual metric. In CVPR, 2018. 1, 5 Appendix A. More Details on Dataset Filtering In order to decide the candidate objects to be potentially ...

2018
[36]

cream motorcycle

shape and 3) position, 4) without affecting the background. The first point is quantified by σOF size , which solely makes up σOF =σOF size . The second point is quantified byσOC lpips,σOC dino, andσOC ℓ2 , and the third point measured byσOC pos . These make up σOC as follows:...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Blended diffusion for text-driven editing of natural im- ages

Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural im- ages. In CVPR, 2022. 2

work page 2022

[2] [2]

EditVal: Benchmarking diffusion based text-guided image edit- ing methods

Samyadeep Basu, Mehrdad Saberi, Shweta Bhardwaj, Atoosa Malemir Chegini, Daniela Massiceti, Maziar Sanjabi, Shell Xu Hu, and Soheil Feizi. EditVal: Benchmarking diffusion based text-guided image edit- ing methods. arXiv:2310.02426, 2023. 2, 8

arXiv

[3] [3]

InstructPix2Pix: Learning to follow image edit- ing instructions

Tim Brooks, Aleksander Holynski, and Alexei A Efros. InstructPix2Pix: Learning to follow image edit- ing instructions. In CVPR, 2023. 2, 6

work page 2023

[4] [4]

A computational approach to edge de- tection

John Canny. A computational approach to edge de- tection. IEEE Transactions on pattern analysis and machine intelligence, 8(6):679–698, 1986. vi

work page 1986

[5] [5]

MasaCtrl: Tuning-free mutual self-attention control for consis- tent image synthesis and editing

Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. MasaCtrl: Tuning-free mutual self-attention control for consis- tent image synthesis and editing. In ICCV, 2023. 2, 6

work page 2023

[6] [6]

DiffEdit: Diffusion- based semantic image editing with mask guidance

Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. DiffEdit: Diffusion- based semantic image editing with mask guidance. In ICLR, 2023. 2, 8

work page 2023

[7] [7]

Imagenet: A large-scale hierarchi- cal image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchi- cal image database. In CVPR, 2009. xi

work page 2009

[8] [8]

The llama 3 herd of models

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv:2407.21783,

arXiv