Pith. sign in

REVIEW 4 major objections 9 minor 103 references

LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs

T0 review · 4 major / 9 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that one fine-tuned multimodal language model can replace separate specialist metrics for text-guided image editing by scoring perceptual quality, editing alignment, and attribute preservation in a single pass.

desk verdict A genuinely useful benchmark and a plausible metric, but the undefined train/test split undermines the headline numbers until clarified. read the letter →

arxiv 2507.16193 v2 pith:377A7M57 submitted 2025-07-22 cs.CV cs.MM

Zitong Xu , Huiyu Duan , Bingnan Liu , Guangji Ma , Jiarui Wang , Liu Yang , Shiqi Gao , Xiaoyu Wang
show 4 more authors
Jia Wang Xiongkuo Min Guangtao Zhai Weisi Lin
This is my paper
classification cs.CVcs.MM
keywords text-guidedimageeditingbenchmarkdatasetlargemultimodalmodelsqualityassessmenthumanpreferencealignmentinstructiontuningzero-shotgeneralizationmeanopinionscore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that one fine-tuned multimodal language model can judge text-guided image edits the way pooled human raters do, across three dimensions at once: perceptual quality, editing alignment, and attribute preservation. To train and test such a model, the authors built EBench-18K, a benchmark of 18,360 edited images produced by 17 editing models on 21 tasks, with 55,080 mean opinion scores and 18,360 yes/no question-answer pairs distilled from over a million human annotations. Their model, LMM4Edit, is claimed to reach rank correlations with human scores of 0.9136 for quality, 0.8830 for alignment, and 0.9048 for preservation on this benchmark, and to generalize zero-shot to another editing benchmark where it achieves the best question-answering accuracy among tested multimodal models. If these results hold, a single automatic judge could replace most per-dimension specialist metrics and much of human evaluation when comparing editing models.

What carries the argument

The carrying mechanism is a two-stage instruction-tuned LMM that always sees the source image, the edited image, and a prompt quoting the editing instruction and asking four questions: three score questions and one yes/no question. Stage one trains the model with cross-entropy loss to emit the four-answer text format, including approximate scores; stage two attaches a small quality-score decoder to the hidden state just before each score token and regresses the exact MOS with mean-squared error. AdaLoRA, adaptive low-rank adaptation that redistributes trainable parameters toward the most important layers, is applied to both the vision encoder and the language model so a pretrained vision-language backbone is adapted without full fine-tuning.

What would settle it

Re-run the subjective study on the same edited images at each model's native output resolution and recompute LMM4Edit's rank correlation with the new human scores; if the numbers fall materially below the reported values, the claimed human alignment is an artifact of downsampling to 512x512.

Watch

Extended reading notes

Core claim

The central discovery is that a single instruction-tuned multimodal language model, fed the source image, the edited image, and a prompt that asks for three scores and one yes/no answer, can predict human preference scores for text-guided image editing on all three evaluation dimensions better than any previously tested metric. The authors build EBench-18K, a large benchmark with 18,360 edited images, 55,080 MOSs, and 18,360 QA pairs from 15 raters per image, and fine-tune LMM4Edit on it. The resulting model reports rank correlations with human MOS of 0.9136 for perceptual quality, 0.8830 for editing alignment, and 0.9048 for attribute preservation, exceeding every comparison metric in Table 3, and it reaches 87.8% QA accuracy in-domain plus the best zero-shot QA accuracy among tested LMMs on the I2EBench dataset.

Load-bearing premise

The load-bearing premise is that the human MOS labels, collected at 512x512 on one calibrated monitor and rescaled per subject with Z-scores, are the correct ground truth for editing quality, even though most of the 17 editing models generate images at higher resolutions.

Editorial extensions

If this is right

  • A single LMM4Edit forward pass gives the three scalar scores and a yes/no task check, so per-dimension specialist metrics are no longer needed for TIE evaluation on this benchmark.
  • The same metric can rank whole editing models: its model-level rankings agree with human rankings more closely than any baseline tested in Table 5.
  • Fine-tuning on EBench-18K lifts editing-alignment correlation from about 0.41 for a strong zero-shot LMM to 0.88, showing that alignment judgments can be learned from MOS data.
  • Zero-shot QA accuracy on I2EBench indicates that the model's edit-checking skill transfers to unseen tasks and instructions without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported correlations survive full-resolution testing, editing-model releases could be screened automatically, with human review reserved for a shortlist of candidates; the paper itself does not test this workflow.
  • The 512x512 viewing condition in the subjective study may under-weight artifacts that only appear at native resolutions, so an immediate next experiment is to collect MOSs at 1024x1024 and check whether model rankings shift.
  • The two-stage training recipe appears portable to other generation-evaluation settings with per-dimension MOS data, such as text-to-video or text-to-3D quality assessment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 9 minor

Summary. The paper introduces EBench-18K, a text-guided image editing benchmark with 1,080 source images, 18,360 edited images from 17 TIE models, and 1,101,600 human annotations in the form of MOSs for perceptual quality, editing alignment, and attribute preservation plus task-specific yes/no QA pairs. It also proposes LMM4Edit, a fine-tuned Qwen2.5-VL-based metric with AdaLoRA that takes source image, edited image, and prompt as input and outputs scores and QA answers. The paper reports state-of-the-art correlations with human MOSs on EBench-18K (SRCC 0.9136/0.8830/0.9048 in Table 3) and best zero-shot QA accuracy on I2EBench among compared LMMs (Figure 7).

Significance. If the results hold, EBench-18K is a substantial community resource: it is larger than existing TIE benchmarks, covers 21 tasks and both description- and instruction-based models, and its subjective protocol is reported in unusually specific detail (15 annotators per image, ITU-R BT.500-13 outlier rejection, Z-score rescaling). LMM4Edit is also a plausible step toward replacing per-dimension specialist metrics with a single instruction-tuned LMM, and the external I2EBench check is a genuine attempt at cross-dataset validation. However, the central generalization claim currently rests on an ambiguous train/test split and on a zero-shot evaluation that only measures QA accuracy, not score prediction on an external benchmark.

major comments (4)
  1. [§5.5, Figure 7] The training and testing split is not specified at the correct granularity. Section 5.1 states only that learning-based models use a 4:1 split, but it does not say whether the split is performed over the 1,080 source images or over the 18,360 edited images. Since LMM4Edit explicitly conditions on the source image (Eq. 2) and the dataset contains 17 edited images per source, a random edited-image split would place outputs of the same source in both training and test sets. A large LMM can then memorize source-specific priors about quality, alignment, and preservation, inflating the headline SRCCs in Table 3 and making the reported margins over fine-tuned baselines uninterpretable for the claimed task of scoring new source images. Please specify the split granularity; if the split is by edited image, rerun the comparison with a source-level 4:1 split and report whether the margins over InternVL2.5 remain.
  2. [§3.3, Table 2] The zero-shot validation on I2EBench reports only QA accuracy, not correlation with human scores for perceptual quality, editing alignment, or attribute preservation. The abstract and conclusion describe LMM4Edit as showing strong generalization ability for evaluating TIE models, but the score-prediction dimensions—which are the core contribution—are not validated outside EBench-18K. If the claim is that LMM4Edit is an all-in-one metric that generalizes, the paper should either report zero-shot score correlations on any external benchmark with human MOSs or explicitly limit the generalization claim to QA accuracy.
  3. [§4.1, Eq. (3), Table 5] The subjective experiment presents all edited images at 512×512 resolution on the display, while Table 2 lists several models that generate at 768×768 or 1024×1024 and the benchmark includes super-resolution and other low-level tasks. Downsampling to 512×512 may compress or hide quality differences that are visible at native resolution, and for super-resolution it is unclear what a human MOS at 512×512 means. The paper should provide evidence that the MOSs are robust to this presentation choice, at least for the low-level task subsets, or discuss this as a limitation of the benchmark.
  4. [§5.2, Table 4] The overall rank used to compare metric-to-human agreement in Table 5 depends on the ad hoc weights in Eq. (3) (0.3/0.4/0.3). No sensitivity analysis is given, and different weights could change the ranking of metrics. At minimum, the authors should justify the weights and show that the main conclusion is stable across reasonable weight choices.
minor comments (9)
  1. [§4.1] There is a duplicated word in the text: 'where where r_ij is the raw rating...'
  2. [Table 3] The text refers to 'QwenLM2 [73]' and 'Qwen2-VL' in places, while Table 6 and the figure use Qwen2.5-VL; please make the backbone naming consistent.
  3. [Table 4] Please verify the TOPIQ row: the reported KRCC for editing alignment (0.6692) is much higher than the corresponding PLCC (0.3848), which is unusual and may be a typo.
  4. [Table 6] In the Remove column for Qwen2-VL, the reported accuracy appears as '0404' instead of '0.404'.
  5. [§2.2] The ablation table uses checkmarks without a clear legend; it is not obvious which model is used as the backbone in each row or which components are frozen versus trainable. Add a legend or spell out the configurations in the caption.
  6. [§5.1] The text cites '[25]' for EditEval, but reference [25] is listed as a survey on diffusion-based image editing; the citation for EditEval should be corrected.
  7. [§3.3] For learning-based methods, the paper states that 'the same training and testing split (4:1)' is used as in previous literature, but the prior literature is not identified; please cite the source of this split protocol.
  8. [§3.1] No inter-annotator agreement statistic is reported; given that the benchmark's value depends on MOS reliability, a brief report of agreement (e.g., ICC or Krippendorff's alpha) would strengthen the paper.
  9. [§3.3] The claim of being 'the first large-scale TIE benchmark' should be qualified against I2EBench and IE-Bench, which are discussed in related work; please state explicitly what EBench-18K adds beyond them (MOSs for three dimensions, 21 tasks, both prompt types).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LMM4Edit is a supervised fit to human MOS evaluated on a held-out split, with zero-shot external validation on I2EBench; self-citations are not load-bearing.

full rationale

The paper's derivation chain is standard supervised metric learning rather than a self-referential derivation. EBench-18K MOSs come from 15 human annotators per image (Section 3.3), LMM4Edit is trained to regress those MOSs via cross-entropy and MSE losses (Section 4.2, Figure 5), and the headline SRCC/KRCC/PLCC numbers in Table 3 are computed on a 4:1 train/test split applied equally to LMM4Edit and the fine-tuned baselines (Section 5.1, Table 3). The test labels are human ratings, not outputs of LMM4Edit or of the authors' prior metrics, so the reported correlations are empirical fits on held-out data, not identities. The zero-shot I2EBench QA comparison (Section 5.5, Figure 7) uses an external benchmark built by other authors and provides independent evidence of generalization. The paper's self-citations ([16], [68], [69], [70]) appear only in the related-work motivation for LMM-based quality assessment and are not load-bearing for the central claim. One ambiguity exists: Section 5.1 says 'the same training and testing split (4:1)' without specifying whether the split is by source image or by edited image, and LMM4Edit takes the source image as input (Eq. 2), so a split by edited image could allow source-identity memorization across train and test. However, this is a potential leakage risk rather than demonstrated circularity; the paper does not exhibit the reduction needed to establish that the reported SRCCs are forced by construction, and the external I2EBench validation would remain informative even if EBench-18K numbers were inflated. The principal caveats are external-scope issues (512x512 viewing resolution despite 14 models generating at higher resolutions, and Z-score rescaling across 21 task subsets), which are correctness concerns, not circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical or theoretical entities are postulated. The free parameters are limited to a hand-chosen weighting convention and standard training hyperparameters. The load-bearing assumptions are the validity of the human labels, the comparability of MOSs after Z-score standardization, and the representativeness of the model/task selection.

free parameters (2)
  • Overall score weights in Eq. 3 = 0.3, 0.4, 0.3
    The combined model ranking score gives editing alignment weight 0.4 and the other two dimensions 0.3 each. The weights are chosen by the authors to emphasize editing alignment, not fitted to human data, so they are a hand-chosen convention that affects the reported overall rank alignment.
  • Training hyperparameters (learning rate 1e-4, epochs 1 and 3) = 1e-4, 1 epoch pretraining, 3 epochs fine-tuning
    Standard training choices, not fitted to the test labels, but they influence the final model performance and are stated without a sensitivity analysis.
assumptions (3)
  • domain assumption Human MOS labels from the subjective study are valid ground truth for TIE quality, alignment, and preservation.
    Section 3.3 states that 30 professional annotators rated images on a 5-point continuous scale and 15 annotators assessed each image. The entire metric training and evaluation rests on these labels being reliable.
  • standard math The ITU-R BT.500-13 outlier rejection and Z-score transformation preserve meaningful cross-task comparability.
    Section 3.4 applies ITU-R BT.500-13 guidelines with a 2-sigma rejection criterion and Z-score normalization before rescaling to [0,100]. The paper assumes this standard processing yields comparable MOSs across the 21 tasks.
  • domain assumption The 17 selected TIE models and 21 hand-designed task types are representative of the TIE landscape.
    Section 3.2 describes the selection of models and tasks. The benchmark's claims about 'human preference for TIE' generalize only to the extent that this pool of models, prompts, and tasks is representative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs." pith.science (2026). https://pith.science/paper/377A7M57

@misc{pith2026250716193,
  author       = {Pith},
  title        = {Pith review of: LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/377A7M57}},
  note         = {Machine review of arXiv:2507.16193}
}
read the original abstract

The rapid advancement of Text-guided Image Editing (TIE) enables image modifications through text prompts. However, current TIE models still struggle to balance image quality, editing alignment, and consistency with the original image, limiting their practical applications. Existing TIE evaluation benchmarks and metrics have limitations on scale or alignment with human perception. To this end, we introduce EBench-18K, the first large-scale image Editing Benchmark including 18K edited images with fine-grained human preference annotations for evaluating TIE. Specifically, EBench-18K includes 1,080 source images with corresponding editing prompts across 21 tasks, 18K+ edited images produced by 17 state-of-the-art TIE models, 55K+ mean opinion scores (MOSs) assessed from three evaluation dimensions, and 18K+ question-answering (QA) pairs. Based on EBench-18K, we employ outstanding LMMs to assess edited images, while the evaluation results, in turn, provide insights into assessing the alignment between the LMMs' understanding ability and human preferences. Then, we propose LMM4Edit, a LMM-based metric for evaluating image Editing models from perceptual quality, editing alignment, attribute preservation, and task-specific QA accuracy in an all-in-one manner. Extensive experiments show that LMM4Edit achieves outstanding performance and aligns well with human preference. Zero-shot validation on the other datasets also shows the generalization ability of our model. The dataset and code are available at https://github.com/IntMeGroup/LMM4Edit.

Figures

Figures reproduced from arXiv: 2507.16193 by the authors.

Figure 1
Figure 1. We present the text-guided image editing (TIE) evaluation database and model, termed EBench-18K and LMM4Edit, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Distribution of perceptual quality, editing alignment and attribute preservation MOSs. (b) Counts and average [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of TIE models across the dimensions of perceptual quality, editing alignment, attribute preservation [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of human evaluations on 17 TIE models across 21 tasks in terms of perceptual quality, editing alignment, [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: An overview of LMM4Edit. The proposed model takes edited image, source image, and instruction prompt as input, [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Comparison of quality score correlation with MOSs and QA accuracy of different LMM models in terms of perceptual [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Zero-shot cross-dataset performance comparison [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

103 extracted references · 53 canonical work pages

  1. [1]

    Abdelrahman Abdelhamed, Stephen Lin, and Michael S Brown. 2018. A high- quality denoising dataset for smartphone cameras. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1692–1700

    2018

  2. [2]

    AI Meta. 2024. Llama 3.2: Revolutionizing Edge AI and Vision with Open, Cus- tomizable Models.Meta AI Blog(20 December 2024)

    2024

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, et al

  4. [4]

    Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kasten, and Tali Dekel

  5. [5]

    Samyadeep Basu, Mehrdad Saberi, Shweta Bhardwaj, Atoosa Malemir Chegini, Daniela Massiceti, Maziar Sanjabi, et al. 2023. EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methods.arXiv preprint arXiv:2310.02426 (2023)

    arXiv 2023

  6. [6]

    Bosse, D

    S. Bosse, D. Maniry, K.-R. Muller, T. Wiegand, and W. Samek. 2017. Deep neural networks for no-reference and full-reference image quality assessment.IEEE Transactions on Image Processing (TIP)27, 1 (2017), 206–219

    2017

  7. [7]

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. 2023. Instructpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 18392–18402

    2023

  8. [8]

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. 2023. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. InProceedings of the IEEE/CVF international confer- ence on computer vision (CVPR). 22560–22570

    2023

Show all 103 references
  1. [9]

    Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. 2024. TOPIQ: A Top-Down Approach From Seman- tics to Distortions for Image Quality Assessment.IEEE Transactions on Image Processing (TIP)33 (2024), 2404–2418

    2024

  2. [10]

    Wei-Ting Chen, Hao-Yu Fang, Cheng-Lin Hsieh, Cheng-Che Tsai, I Chen, Jian- Jiun Ding, Sy-Yen Kuo, et al. 2021. ALL Snow Removed: Single Image Desnowing Algorithm Using Hierarchical Dual-Tree Complex Wavelet Representation and Contradict Channel Loss. InProceedings of the IEEE/...

    2021

  3. [11]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jin- guo Zhu, Shenglong Ye, Hao Tian, et al. 2025. Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling. arXiv preprint arXiv:2412.05271(2025)

    2025 arXiv

  4. [13]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, et al

  5. [14]

    Wenhan Yang Jiaying Liu Chen Wei, Wenjing Wang. 2018. Deep Retinex Decom- position for Low-Light Enhancement. InProceedings of the British Machine Vision Conference (BMVC)

    2018

  6. [15]

    Damera-Venkata, T

    N. Damera-Venkata, T. D. Kite, W. S. Geisler, B. L. Evans, and A. C. Bovik. 2000. Image quality assessment based on a degradation model.IEEE Transactions on Image Processing (TIP)9, 4 (2000), 636–650

    2000

  7. [16]

    Huiyu Duan, Qiang Hu, Jiarui Wang, Liu Yang, Zitong Xu, Lu Liu, Xiongkuo Min, Chunlei Cai, Tianxiao Ye, Xiaoyun Zhang, and Guangtao Zhai. 2025. FineVQ: Fine-Grained User Generated Content Video Quality Assessment. InProceedings of the IEEE/CVF Conference on Computer Vision and...

    2025

  8. [17]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 24185–24198

  9. [18]

    X. Fu, J. Huang, X. Ding, Y. Liao, and J. Paisley. 2017. Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal.IEEE Transactions on Image Processing (TIP)26, 6 (2017), 2944–2956

    2017

  10. [19]

    Daniel Garibi, Or Patashnik, Andrey Voynov, Hadar Averbuch-Elor, and Daniel Cohen-Or. 2024. ReNoise: Real Image Inversion Through Iterative Noising.arXiv preprint arXiv:2403.14602(2024)

    2024 arXiv

  11. [20]

    Abhijay Ghildyal and Feng Liu. 2022. Shift-Tolerant Perceptual Similarity Metric. InProceedings of the European Conference on Computer Vision (ECCV). 91–107

    2022

  12. [21]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, et al. 2024. Scaling rectified flow transformers for high-resolution im- age synthesis. InProceedings of the International Conference on Machine Learning (ICML)

    2024

  13. [22]

    K. Gu, G. Zhai, X. Yang, and W. Zhang. 2012. An improved full-reference image quality metric based on structure compensation. InProceedings of the conference on Asia-Pacific Signal and Information Processing Association (APSIPA). 1–6

    2012

  14. [23]

    Amir Hertz, Kfir Aberman, and Daniel Cohen-Or. 2023. Delta Denoising Score. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR). 2328–2337

    2023

  15. [24]

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. Clipscore: A Reference-Free Evaluation Metric for Image Captioning.arXiv preprint arXiv:2104.08718(2021)

    2021 arXiv

  16. [25]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks.Communications of the ACM (CACM)63, 11 (Oct. 2020), 139–144

    2020

  17. [26]

    Inbar Huberman-Spiegelglas, Vladimir Kulikov, and Tomer Michaeli. 2024. An edit friendly DDPM noise space: Inversion and manipulations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 12469–12478

    2024

  18. [27]

    Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang, et al. 2024. HQ-Edit: A High-Quality Dataset for Instruction-Based Image Editing. arXiv preprint arXiv:2404.09990(2024)

    2024 arXiv

  19. [28]

    2012.Methodology for the Subjec- tive Assessment of the Quality of Television Pictures

    International Telecommunication Union (ITU). 2012.Methodology for the Subjec- tive Assessment of the Quality of Television Pictures. Technical Report Rec. ITU-R BT.500-13. International Telecommunication Union (ITU)

    2012

  20. [29]

    Yi Huang, Jiancheng Huang, Yifan Liu, Mingfu Yan, Jiaxi Lv, Jianzhuang Liu, et al. 2024. Diffusion Model-Based Image Editing: A Survey.IEEE transactions on pattern analysis and machine intelligence (TPAMI)PP (2024)

    2024

  21. [30]

    Le Kang, Peng Ye, Yi Li, and David Doermann. 2014. Convolutional Neural Networks for No-Reference Image Quality Assessment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2014

  22. [31]

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, et al. 2023. Imagic: Text-Based Real Image Editing with Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2023

  23. [32]

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. 2023. Pick-a-Pic: An Open Dataset of User Preferences for Text-to- Image Generation. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS). 36652–36663

    2023

  24. [33]

    Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. 2024. PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code. InProceedings of the International Conference on Learning Representations (ICLR)

    2024

  25. [34]

    Vladimir Kulikov, Matan Kleiner, Inbar Huberman-Spiegelglas, and Tomer Michaeli. 2024. FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Models.arXiv preprint arXiv:2412.08629(2024)

    2024 arXiv

  26. [35]

    Mingi Kwon, Jaeseok Jeong, and Youngjung Uh. 2023. Diffusion Models Already Have a Semantic Latent Space. InProceedings of the International Conference on Learning Representations (ICLR)

    2023

  27. [36]

    Shanshan Lao, Yuan Gong, Shuwei Shi, Sidi Yang, Tianhe Wu, Jiahao Wang, et al

  28. [37]

    Juil Koo, Chanho Park, and Minhyuk Sung. 2024. Posterior Distillation Sam- pling. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13352–13361

    2024

  29. [38]

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, et al. 2024. LLaVA-Next-Interleave: Tackling Multi-Image, Video, and 3D in Large Multi- modal Models.arXiv preprint arXiv:2407.07895(2024)

    2024 arXiv

  30. [39]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. InProceedings of the International Conference on Machine Learning (ICML). 12888–12900

    2022

  31. [40]

    Shufan Li, Harkanwar Singh, and Aditya Grover. 2023. Instructany2pix: Flex- ible Visual Editing via Multimodal Instruction Following.arXiv preprint arXiv:2312.06738(2023)

    2023 arXiv

  32. [41]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

    Attentions Help CNNs See Better: Attention-Based Hybrid Image Quality Assessment Network. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 1140–1149

  33. [42]

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, et al. 2024. Evaluating and Improving Compositional Text-to-Visual Generation. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  34. [43]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023. Improved Baselines with Visual Instruction Tuning.arXiv preprint arXiv:2310.03744(2023)

    2023 arXiv

  35. [44]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual In- struction Tuning. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vol. 36. 34892–34916

    2023

  36. [45]

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, et al. 2024. DeepSeek-VL: Towards Real-World Vision-Language Understanding.arXiv preprint arXiv:2403.055256 (2024)

    2024 arXiv

  37. [46]

    Shanglin Li, Bohan Zeng, Yutang Feng, Sicheng Gao, Xuhui Liu, Jiaming Liu, et al. 2023. Zone: Zero-Shot Instruction-Guided Local Editing.arXiv preprint arXiv:2312.16794(2023)

    2023 arXiv

  38. [47]

    Yuheng Li, Haotian Liu, Mu Cai, Yijun Li, Eli Shechtman, Zhe Lin, et al . 2024. Removing Distributional Discrepancies in Captions Improves Image-Text Align- ment.arXiv preprint arXiv:2410.00905(2024)

    2024 arXiv

  39. [48]

    Moorthy, and Alan C

    Anish Mittal, Anush K. Moorthy, and Alan C. Bovik. 2011. Blind/Referenceless Image Spatial Quality Evaluator. InProceedings of the Asilomar Conference on Signals, Systems and Computers (ACSSC). 723–727

    2011

  40. [49]

    Com- pletely Blind

    Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. 2013. Making a “Com- pletely Blind” Image Quality Analyzer.IEEE Signal Processing Letters (SPL)20, 3 (2013), 209–212

    2013

  41. [50]

    A. K. Moorthy and A. C. Bovik. 2009. A Modular Framework for Constructing Blind Universal Quality Indices.IEEE Signal Processing Letters (SPL)(2009)

    2009

  42. [51]

    Yiwei Ma, Jiayi Ji, Ke Ye, Weihuang Lin, Yonghan Zheng, Qiang Zhou, et al. 2024. I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing. In Proceedings of the Advances in Neural Information Processing Systems (NeurPIS)

    2024

  43. [52]

    Chaojie Mao, Jingfeng Zhang, Yulin Pan, Zeyinzi Jiang, Zhen Han, Yu Liu, et al

  44. [53]

    MM ’25, October 27–31, 2025, Dublin, Ireland

    ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Filling.arXiv preprint arXiv:2501.02487(2025). MM ’25, October 27–31, 2025, Dublin, Ireland

    2025 arXiv

  45. [54]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, et al . 2024. GPT-4 Technical Report.arXiv preprint arXiv:2303.08774(2024)

    2024 arXiv

  46. [55]

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, et al. 2023. Sdxl: Improving Latent Diffusion Models for High-Resolution Image Synthesis.arXiv preprint arXiv:2307.01952(2023)

    2023 arXiv

  47. [56]

    Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever

    Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. InProceedings of t...

    2021

  48. [57]

    Anush Krishna Moorthy and Alan Conrad Bovik. 2011. Blind Image Quality As- sessment: From Natural Scene Statistics to Perceptual Quality.IEEE Transactions on Image Processing (TIP)20, 12 (2011), 3350–3364

    2011

  49. [58]

    Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. 2017. Deep Multi-Scale Convolutional Neural Network for Dynamic Scene Deblurring. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2017

  50. [59]

    Hyelin Nam, Gihyun Kwon, Geon Yeong Park, and Jong Chul Ye. 2024. Contrastive Denoising Score for Text-guided Latent Diffusion Image Editing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9192–9201

    2024

  51. [60]

    H. R. Sheikh, A. C. Bovik, and G. De Veciana. 2005. An information fidelity crite- rion for image quality assessment using natural scene statistics.IEEE Transactions on Image Processing (TIP)14, 12 (2005), 2117–2128

    2005

  52. [61]

    Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. 2024. Emu Edit: Precise Image Editing via Recognition and Generation Tasks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (...

    2024

  53. [62]

    S. Su, Q. Yan, Y. Zhu, C. Zhang, X. Ge, J. Sun, et al. 2020. Blindly assess image quality in the wild guided by a self-adaptive hyper network. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  54. [63]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10684–10695

    2022

  55. [64]

    Saad, Alan C

    Michele A. Saad, Alan C. Bovik, and Christophe Charrier. 2012. Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain. IEEE Transactions on Image Processing (TIP)21, 8 (2012), 3339–3352

    2012

  56. [65]

    H. R. Sheikh and A. C. Bovik. 2006. Image information and visual quality.IEEE Transactions on Image Processing (TIP)15, 2 (2006), 430–444

    2006

  57. [66]

    Bram Wallace, Akash Gokul, and Nikhil Naik. 2022. EDICT: Exact Diffusion Inversion via Coupled Transformations.arXiv preprint arXiv:2211.12446(2022)

    2022 arXiv

  58. [67]

    Chan, and Chen Change Loy

    Jianyi Wang, Kelvin C.K. Chan, and Chen Change Loy. 2023. Exploring CLIP for assessing the look and feel of images. InProceedings of the Conference on Association for the Advancement of Artificial Intelligence (AAAI)

    2023

  59. [68]

    Jiarui Wang, Huiyu Duan, Jing Liu, Shi Chen, Xiongkuo Min, and Guangtao Zhai. 2023. AIGCIQA2023: A Large-Scale Image Quality Assessment Database for AI Generated Images: From the Perspectives of Quality, Authenticity and Correspondence. InProceedings of the CAAI International ...

    2023

  60. [69]

    Shangkun Sun, Bowen Qu, Xiaoyu Liang, Songlin Fan, and Wei Gao. 2025. IE- Bench: Advancing the Measurement of Text-Driven Image Editing for Human Perception Alignment.arXiv preprint arXiv:2501.09927(2025)

    2025 arXiv

  61. [70]

    Talebi and P

    H. Talebi and P. Milanfar. 2018. NIMA: Neural image assessment.IEEE Transac- tions on Image Processing (TIP)27, 8 (2018), 3998–4011

    2018

  62. [71]

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. 2023. Emergent Correspondence from Image Diffusion. InProceedings of the Conference on Neural Information Processing Systems (NeurIPS)

    2023

  63. [72]

    Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. 2024. Taming Rectified Flow for Inversion and Editing.arXiv preprint arXiv:2411.04746(2024)

    2024 arXiv

  64. [73]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, et al. 2024. Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution.arXiv preprint arXiv:2409.12191(2024)

    2024 arXiv

  65. [74]

    Puyi Wang, Wei Sun, Zicheng Zhang, Jun Jia, Yanwei Jiang, Zhichao Zhang, et al. 2024. Large Multi-modality Model Assisted AI-Generated Image Quality Assessment. InProceedings of the ACM International Conference on Multimedia (ACM MM). 7803–7812

    2024

  66. [75]

    Jiarui Wang, Huiyu Duan, Guangtao Zhai, and Xiongkuo Min. 2025. Quality Assessment for AI Generated Images with Instruction Tuning.IEEE Transactions on Multimedia (TMM)(2025)

    2025

  67. [76]

    Jiarui Wang, Huiyu Duan, Guangtao Zhai, Juntong Wang, and Xiongkuo Min

  68. [77]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMM. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  69. [78]

    Jifeng Wang, Xiang Li, and Jian Yang. 2018. Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Re- moval. InProceedings of the IEEE/CVF International Conference on Computer Vision (CVPR)

    2018

  70. [79]

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, et al

  71. [80]

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, et al

  72. [81]

    Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, and Joyce Chai. 2024. Inversion- Free Image Editing with Natural Language. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR). 9192–9201

    2024

  73. [82]

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing (TIP)13, 4 (2004), 600–612

    2004

  74. [83]

    Z. Wang, E. P. Simoncelli, and A. C. Bovik. 2003. Multiscale structural similarity for image quality assessment. InProceedings of the Asilomar Conference on Signals, Systems & Computers (ACSSC), Vol. 2. 1398–1402

    2003

  75. [84]

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Chunyi Li, Liang Liao, et al. 2023. Q-Align: Teaching LMMs for Visual Scoring via Discrete Text- Defined Levels.arXiv preprint arXiv:2312.17090(2023)

    2023 arXiv

  76. [85]

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. 2023. Human Preference Score: Better Aligning Text-to-Image Models with Human Preference. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2023

  77. [86]

    Guanghao Yin, Wei Wang, Zehuan Yuan, Chuchu Han, Wei Ji, Shouqian Sun, et al. 2022. Content-variant reference image quality assessment via knowledge distillation. InProceedings of the Conference on Association for the Advancement of Artificial Intelligence (AAAI), Vol. 36. 3134–3142

    2022

  78. [87]

    DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding.arXiv preprint arXiv:2412.10302(2024)

    2024 arXiv

  79. [88]

    Zhang, L

    L. Zhang, L. Zhang, X. Mou, and D. Zhang. 2011. FSIM: A feature similarity index for image quality assessment.IEEE Transactions on Image Processing (TIP)20, 8 (2011), 2378–2386

    2011

  80. [89]

    Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, et al

  81. [90]

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, et al. 2023. Adaptive Budget Allocation for Parameter-Efficient Fine- Tuning. InProceedings of the International Conference on Learning Representations (ICLR)

    2023

  82. [91]

    W. Xue, L. Zhang, X. Mou, and A. C. Bovik. 2013. Gradient magnitude similarity deviation: A highly efficient perceptual image quality index.IEEE Transactions on Image Processing (TIP)23, 2 (2013), 684–695

    2013

  83. [92]

    Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, et al. 2022. MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1191–1200

    2022

  84. [93]

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, et al

  85. [94]

    MiniCPM-V: A GPT-4V Level MLLM on Your Phone.arXiv preprint arXiv:2408.01800(2024)

    2024 arXiv

  86. [95]

    Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, et al. 2024. mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multimodal Large Language Models. InProceedings of the International Conference on Learning Representations (ICLR)

    2024

  87. [97]

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. 2023. Magicbrush: A manually annotated dataset for instruction-guided image editing. InProceedings of the Advances in Neural Information Processing Systems (NeurPIS)

    2023

  88. [100]

    InternLM-XComposer: A Vision-Language Large Model for Advanced Text-Image Comprehension and Composition.arXiv preprint arXiv:2309.15112 (2023)

    2023 arXiv

  89. [102]

    Zhang, P

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. 2018. The unreason- able effectiveness of deep features as a perceptual metric. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2018

  90. [103]

    Zhang, K

    W. Zhang, K. Ma, J. Yan, D. Deng, and Z. Wang. 2020. Blind image quality assessment using a deep bilinear convolutional neural network.IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)30, 1 (2020), 36–47

    2020

  91. [104]

    S. Zhao, L. Zhang, et al. 2019. Evaluation of Defogging: A Real-World Benchmark Dataset, a New Criterion and Baselines. InProceedings of the IEEE International Conference on Multimedia and Expo (ICME). 1840–1845

    2019

  92. [2022]

    InProceedings of the European Conference on Computer Vision (ECCV)

    Text2live: Text-driven layered image and video editing. InProceedings of the European Conference on Computer Vision (ECCV). 707–723

  93. [2023]

    InProceedings of the International Conference on Neural Information Processing Systems (NeurIPS)

    ImageReward: learning and evaluating human preferences for text-to-image generation. InProceedings of the International Conference on Neural Information Processing Systems (NeurIPS). 15903–15935

  94. [2024]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 24185–24198

  95. [2025]

    Qwen2.5-VL Technical Report.arXiv preprint arXiv:2502.13923(2025)

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.