Pith. sign in

REVIEW 3 major objections 6 minor 75 references

NTIRE 2025 challenge on Text to Image Generation Model Quality Assessment

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read All submitted challenge systems beat the published baselines on fine-grained text-to-image quality assessment.

desk verdict A useful new dataset and benchmark, but the alignment-track winner trained on validation images that are partly in the test set, so the headline ranking is not established. read the letter →

arxiv 2505.16314 v1 pith:T4F3EJPD submitted 2025-05-22 cs.CV cs.AI

classification cs.CVcs.AI
keywords text-to-imagegenerationimagequalityassessmentimage-textalignmentstructuraldistortiondetectionchallengereportmultimodallargelanguagemodelsEvalMuse-40KMuse-Structure
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a structured competition on fine-grained quality assessment of text-to-image (T2I) generation models. Its central claim is that current assessment methods can be pushed well past published baselines: every finalist beat the FGA-BLIP2 baseline on element-level image-text alignment or the RAHF baseline on structural distortion localization, and the top methods improved the main score by 0.1295 and 0.0862 respectively. The winning approaches fine-tune large multimodal models and combine them with specialized detection or segmentation components. The paper also contributes EvalMuse-Structure, a 10K-image dataset with bounding-box structural distortion annotations.

What carries the argument

The central machinery is the paired evaluation protocol, which uses Spearman Rank-order Correlation Coefficient (SRCC), Pearson Linear Correlation Coefficient (PLCC), element-presence accuracy (ACC), and the F1 score between predicted and human distortion masks. For alignment, Main Score = $0.5 \times \text{ACC} + 0.25 \times (\text{SRCC} + \text{PLCC})$; for structure, Main Score = $0.7 \times \text{F1} + 0.15 \times (\text{SRCC} + \text{PLCC})$. The winning methods carry the argument through fine-tuned multimodal large language models that output both global alignment scores and per-element presence predictions, and through hybrid CNN-Transformer plus instance-segmentation pipelines for structural distortion detection.

What would settle it

Re-run the test phase with the validation-contaminated portion removed and a freshly annotated held-out set, before teams see any feedback, then compare rankings; if the margins of the top teams shrink or reverse, the reported superiority is partly an artifact of leaderboard tuning.

Watch

Extended reading notes

Core claim

The challenge establishes empirically that fine-grained T2I quality assessment is tractable and that published baselines are not ceilings. In the alignment track, the top team, an ensemble of fine-tuned multimodal language models with instruction-augmented element matching, reaches a main score of 0.8551 versus the FGA-BLIP2 baseline of 0.7256; six alignment teams exceed 0.8. In the structure track, the winning team, combining a CNN-Transformer hybrid quality scorer with instance segmentation for distortion localization, reaches 0.6927 versus the RAHF baseline of 0.6065. Across both tracks, all 20 submitted systems outperform their track's baseline.

Load-bearing premise

The test set is a fair measure of generalization even though part of it is drawn from the validation set whose leaderboard scores were shown to participants during development, so teams could have tuned to those images.

Editorial extensions

If this is right

  • Fine-grained element-level alignment scoring improves substantially when multimodal models are fine-tuned with element-specific prompts, chain-of-thought queries, pseudo-label augmentation, and ensembling.
  • Structural distortion detection is more effective when framed as instance segmentation or heatmap prediction with separate quality-score regression than as a single semantic segmentation task.
  • Scoring and localization can be decoupled: the top structure team uses separate models for the quality score and the distortion mask, suggesting the two subtasks have different failure modes.
  • The new EvalMuse-Structure dataset provides a reusable benchmark that combines a structure score and a localization mask under one evaluation protocol.
  • Because every final team beat the published baselines, the baseline methods leave clear headroom in both fine-grained alignment and structural distortion detection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the test set includes validation images whose leaderboard scores were shown to participants during development, part of the reported gap over baselines may reflect leaderboard tuning rather than pure generalization; a fully withheld test set would separate these effects.
  • The element-presence accuracy metric rewards marking an element present if any ensemble member votes yes, which can inflate recall; reporting precision-recall tradeoffs per element type would sharpen the comparison.
  • The F1 mask metric derived from majority-overlap annotation may under-penalize small or boundary-detail distortions; instance-level average precision or boundary-aware metrics could reorder the structure track rankings.
  • The reported improvements are aggregate over prompts and generative models; it remains unknown whether gains are uniform or concentrated in specific element categories such as quantity, spatial relations, or color.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This report describes the NTIRE 2025 challenge on text-to-image generation model quality assessment, with two tracks: an image-text alignment track using EvalMuse-40K and a structural distortion track using the new EvalMuse-Structure dataset. The evaluation protocol uses SRCC, PLCC (after third-order polynomial fitting), ACC for element-level alignment, and F1 for structural masks, combined into a weighted Main Score for each track. The paper reports that all participating teams outperformed the baselines (FGA-BLIP2 and RAHF), with IH-VQA winning the alignment track and HNU-VPAI winning the structure track. Section 5 provides brief descriptions of the 20 submitted methods, including fine-tuned MLLMs, ensembles, pseudo-labeling, and task-specific segmentation/regression architectures.

Significance. If the results are valid, the challenge provides a useful comparison for fine-grained T2I quality assessment, introduces EvalMuse-Structure as a new benchmark, and demonstrates that fine-tuned MLLM ensembles can surpass existing published baselines on alignment scoring and distortion localization. The paper includes standard metrics, full result tables, and team fact sheets, which are valuable for the community. However, the significance is conditional on the test set being a clean held-out measure; as discussed below, the alignment track test construction and the winning team's validation-set training strategy create a direct overlap concern that must be resolved before the headline ranking can be accepted.

major comments (3)
  1. [§3.1, §3.3, §5.1.1] The alignment track test set is not fully held out. Section 3.1 states that the test set is composed of 'additional image-text pairs... combined with a portion of the validation set,' while Section 3.3 says participants received the validation set (without annotations) and obtained 'immediate feedback' from the validation leaderboard. Section 5.1.1 then reports that the winning team IH-VQA used a 'validation set augmentation strategy' in which the model generates pseudo-labels on the validation set and merges them into the training set. Under the natural reading that the released validation set is the same split folded into the test set, IH-VQA trained on images that later appear in the test set. This means the reported margin of 0.1295 over the FGA-BLIP2 baseline (Section 4) does not establish generalization; it may reflect memorization of validation-derived test images. The authors must clarify whether the validation set used for pseudo-labeling is identical to the validation portion included in the test set, and must report rankings computed on the additional (cleanly held-out) test images only.
  2. [§3.1, §3.3] Even for teams that did not explicitly pseudo-label the validation set, the challenge design allowed all participants to receive immediate feedback on the validation set, which is partly included in the test set. This permits validation-based tuning (e.g., checkpoint selection, hyperparameter choice, ensemble weighting) on images that later appear in the test set, potentially inflating scores and altering rankings for all alignment-track teams. The authors should quantify the effect by reporting results on the portion of the test set that was not derived from the released validation set, and should state whether any teams were restricted from using validation images in training. Without this, the claim in Section 4 that 'all submitted results... achieve better performance than the baseline' and the relative ordering of teams are not reliable evidence of generalization.
  3. [§3.1] The reliability of the ground-truth annotations is not substantiated. The paper reports that each image-text pair is annotated by three annotators with re-annotation only for 'significant disagreements' in alignment scores, and that structural masks are derived from regions overlapping in at least two of three annotators, but no inter-annotator agreement statistics (e.g., Cohen's kappa, Krippendorff's alpha, or mask IoU between annotators) are provided. Since all challenge scores are evaluated against these annotations, the absence of agreement metrics leaves the quality of the benchmark itself unverified. The authors should report agreement statistics for both tracks, including agreement on element-level alignment labels and on structural distortion masks.
minor comments (6)
  1. [§1] The introduction states '16 valid participating teams in the image track and 12 valid participating teams in the video track,' but the abstract and Section 4 report 12 teams in the alignment track and 8 teams in the structure track; 'image track' and 'video track' should be 'alignment track' and 'structure track.'
  2. [§3.2] The phrase 'Person Linear Correlation Coefficient' is a typo; it should be 'Pearson Linear Correlation Coefficient.'
  3. [§5.2.8 and Appendix C] The team name is written as 'Tenryu Badu' in the heading of Section 5.2.8 and as 'Tenryu Babu' in the team list in Appendix C; the spelling should be consistent.
  4. [References] Several references are duplicated: [1] and [2] are the same Qwen2.5-VL technical report, [57] and [58] are the same Q-Align paper, and [69] and [70] are the same DeQA-Score paper. These should be consolidated into single entries.
  5. [§5.1.3] The statement that 'they average the element-wise scores in original dataset to produce a single score per field' is unclear; it is not obvious what 'field' refers to, and the sentence should be rephrased for clarity.
  6. [§4] Adjacent teams in the structure track have very small Main Score differences (e.g., ranks 2–6 in Table 2 span only 0.0167), and no confidence intervals or significance tests are reported; a sentence acknowledging the uncertainty in close rankings would strengthen the presentation.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; the paper is a challenge report whose results are measurements against human annotations. The main self-citations are to the organizers' own benchmark and baseline, which is normal, and the validation/test overlap is a validity caveat rather than a circular step.

full rationale

This paper is a challenge report rather than a derivation. The central claims in Section 4 are measurements of submitted systems against human-annotated test labels, so there is no fitted parameter that is later relabeled as a prediction by the paper itself. The main self-citations are to the organizers' own EvalMuse-40K benchmark and FGA-BLIP2 baseline [17]; using one's own dataset as the challenge benchmark and as a comparison baseline is a normal part of a challenge report and does not constitute circular reasoning, because the winning methods' scores are not derived from [17]. A caveat worth noting for correctness rather than circularity: Section 3.1 states that the test set is 'combined with a portion of the validation set,' Section 3.3 says participants received immediate validation feedback, and Section 5.1.1 reports that IH-VQA used a 'validation set augmentation strategy' that merges pseudo-labeled validation images into training. This creates a potential train/test overlap that could inflate the reported margins, but it is an experimental-design limitation, not a case of the paper's predictions being equivalent to its inputs by construction. No circular step is exhibited in the paper's equations or protocol definitions.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper makes no theoretical derivation; its content is empirical. The free parameters are the design choices that define the benchmark and its ground truth, plus the standard PLCC fitting polynomial. The axioms are the usual IQA assumptions (human annotations as truth, correlation metrics as validity) and the untested assumption that validation-set feedback does not contaminate the test measurement. No new physical or conceptual entities are postulated; EvalMuse-Structure is a new dataset, not an invented entity.

free parameters (4)
  • Alignment track main-score weights = 0.5 * ACC + 0.25 * (SRCC + PLCC)
    Hand-chosen weights in Eq. (1) that define the ranking; different weights would change the leaderboard order.
  • Structure track main-score weights = 0.7 * F1 + 0.15 * (SRCC + PLCC)
    Hand-chosen weights in Eq. (2) that define the ranking.
  • Structural mask annotator agreement threshold = regions overlapping in at least 2 of 3 annotators
    Section 3.1 defines ground-truth masks by overlapping regions from at least two of three annotators; an ad hoc rule that sets the target for F1 evaluation.
  • PLCC fitting polynomial = third-order polynomial, coefficients fit per method
    Section 3.2 applies third-order polynomial nonlinear regression before computing PLCC; standard IQA practice, but a data fit applied to every method's predictions.
assumptions (4)
  • domain assumption Human annotations (alignment scores, element presence, structural masks) are treated as ground truth.
    Section 3.1 describes the annotation process but reports no inter-annotator agreement statistics, so annotation reliability is assumed rather than demonstrated.
  • domain assumption SRCC, PLCC, ACC, and F1 are valid proxies for fine-grained T2I quality.
    Section 3.2 defines the metrics; the paper does not justify that the weighted main score corresponds to what users or developers care about.
  • ad hoc to paper Immediate validation feedback does not materially inflate scores on the validation-derived portion of the test set.
    Section 3.3 gives participants immediate validation feedback; Section 3.1 places part of the validation set into the test set. The paper proceeds without testing for contamination.
  • domain assumption Pretrained foundation models (Qwen2.5-VL, InternVL, CLIP, SAM, Florence-2) provide reliable visual representations.
    All top methods in Section 5 are fine-tuned from these pretrained models; the paper does not re-validate the base models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NTIRE 2025 challenge on Text to Image Generation Model Quality Assessment." pith.science (2026). https://pith.science/paper/T4F3EJPD

@misc{pith2026250516314,
  author       = {Pith},
  title        = {Pith review of: NTIRE 2025 challenge on Text to Image Generation Model Quality Assessment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T4F3EJPD}},
  note         = {Machine review of arXiv:2505.16314}
}
read the original abstract

This paper reports on the NTIRE 2025 challenge on Text to Image (T2I) generation model quality assessment, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2025. The aim of this challenge is to address the fine-grained quality assessment of text-to-image generation models. This challenge evaluates text-to-image models from two aspects: image-text alignment and image structural distortion detection, and is divided into the alignment track and the structural track. The alignment track uses the EvalMuse-40K, which contains around 40K AI-Generated Images (AIGIs) generated by 20 popular generative models. The alignment track has a total of 371 registered participants. A total of 1,883 submissions are received in the development phase, and 507 submissions are received in the test phase. Finally, 12 participating teams submitted their models and fact sheets. The structure track uses the EvalMuse-Structure, which contains 10,000 AI-Generated Images (AIGIs) with corresponding structural distortion mask. A total of 211 participants have registered in the structure track. A total of 1155 submissions are received in the development phase, and 487 submissions are received in the test phase. Finally, 8 participating teams submitted their models and fact sheets. Almost all methods have achieved better results than baseline methods, and the winning methods in both tracks have demonstrated superior prediction performance on T2I model quality assessment.

Figures

Figures reproduced from arXiv: 2505.16314 by the authors.

Figure 1
Figure 1. Overview of team IH-VQA proposed iMatch. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Overview of team Evalthon proposed method. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overview of team HCMUS proposed method. 5.1.2. Evalthon Team Evalthon [76] win second place in the image￾text alignment track. They train multiple Large Vi￾sion Language Models(LVLMs) and integrate them us￾ing an XGBoost model. The baselines are Qwen2.5-VL￾7B,InternVL2.5-26B and InternVL2.5-38B[7]. As depicted in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Overview of team MICV proposed method. pseudo-labeled datasets with original EvalMuse dataset for training Qwen2.5-VL-72B-Instruct. They denote this dataset as External, while they also construct External Private dataset by removing the type attribute. Specifically, th…
Figure 5
Figure 5. Figure 5: Overview of team SJTU-MMLab proposed method. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Overview of team SJTUMM proposed method. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Overview of team YAG proposed method [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Overview of team AIIG proposed method. combining training of overall and element alignment scores. They use the ITM setup of BLIP2[30] to concatenate the query and embedded text, and then cross-focus with the im￾age. The final alignment score is obtained by a two-class…
Figure 9
Figure 9. Figure 9: Overview of team out of memory proposed method. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Overview of team I2 Group proposed method. Y˜ k of Xk as the pseudo label based on the model Mk. Fi￾nally, they take the average of the true label and the pseudo label as the label for model training. Besides, they also adopt the strategy of prompt rewriting to make t…
Figure 11
Figure 11. Figure 11: Overview of team Brute Force Wins proposed method. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Overview of team Wecan EvalAIG proposed method. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Overview of team Tenryu Badu proposed method. [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

75 extracted references · 51 canonical work pages

  1. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. 5

  2. [3]

    Xgboost: A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794, 2016. 6

  3. [4]

    Altclip: Altering the lan- guage encoder in clip for extended language capabilities

    Zhongzhi Chen, Guang Liu, Bo-Wen Zhang, Fulong Ye, Qinghong Yang, and Ledell Wu. Altclip: Altering the lan- guage encoder in clip for extended language capabilities. arXiv preprint arXiv:2211.06679, 2022. 14

  4. [5]

    Teacher-guided learning for blind image quality assessment

    Zewen Chen, Juan Wang, Bing Li, Chunfeng Yuan, Wei- hua Xiong, Rui Cheng, and Weiming Hu. Teacher-guided learning for blind image quality assessment. In Proceedings of the Asian Conference on Computer Vision , pages 2457– 2474, 2022. 14

  5. [6]

    Seagull: No-reference image quality assess- ment for regions of interest via vision-language instruction tuning

    Zewen Chen, Juan Wang, Wen Wang, Sunhan Xu, Hang Xiong, Yun Zeng, Jian Guo, Shuxun Wang, Chunfeng Yuan, Bing Li, et al. Seagull: No-reference image quality assess- ment for regions of interest via vision-language instruction tuning. arXiv preprint arXiv:2411.10161, 2024. 14

  6. [7]

    Expanding performance boundaries of open-source multimodal models with model, data, and test- time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhang- wei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test- time scaling. arXiv preprint arXiv:2412.05271, 2024. 6

  7. [8]

    NTIRE 2025 challenge on image super-resolution (×4): Methods and results

    Zheng Chen, Kai Liu, Jue Gong, Jingkai Wang, Lei Sun, Zongwei Wu, Radu Timofte, Yulun Zhang, et al. NTIRE 2025 challenge on image super-resolution (×4): Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Work- shops, 2025. 2

  8. [9]

    Promptiqa: Boosting the performance and generalization for no-reference image quality assessment via prompts

    Zewen Chen, Haina Qin, Juan Wang, Chunfeng Yuan, Bing Li, Weiming Hu, and Liang Wang. Promptiqa: Boosting the performance and generalization for no-reference image quality assessment via prompts. In European Conference on Computer Vision, pages 247–264. Springer, 2025. 14

Show all 75 references
  1. [10]

    NTIRE 2025 challenge on real-world face restoration: Methods and results

    Zheng Chen, Jingkai Wang, Kai Liu, Jue Gong, Lei Sun, Zongwei Wu, Radu Timofte, Yulun Zhang, et al. NTIRE 2025 challenge on real-world face restoration: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Work- shops...

  2. [11]

    NTIRE 2025 challenge on raw image restoration and super-resolution

    Marcos Conde, Radu Timofte, et al. NTIRE 2025 challenge on raw image restoration and super-resolution. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025. 2

  3. [12]

    Raw image reconstruc- tion from RGB on smartphones

    Marcos Conde, Radu Timofte, et al. Raw image reconstruc- tion from RGB on smartphones. NTIRE 2025 challenge re- port. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) Workshops ,

  4. [13]

    NTIRE 2025 challenge on night photography rendering

    Egor Ershov, Sergey Korchagin, Alexei Khalin, Artyom Pan- shin, Arseniy Terekhin, Ekaterina Zaychenkova, Georgiy Lobarev, Vsevolod Plokhotnyuk, Denis Abramov, Elisey Zhdanov, Sofia Dorogova, Yasin Mamedov, Nikola Banic, Georgii Perevozchikov, Radu Timofte, et al. NTIRE 2025 ch...

  5. [14]

    Eva: Exploring the limits of masked visual represen- tation learning at scale

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual represen- tation learning at scale. arXiv preprint arXiv:2211.07636 ,

  6. [15]

    NTIRE 2025 challenge on cross-domain few-shot object detection: Methods and results

    Yuqian Fu, Xingyu Qiu, Bin Ren Yanwei Fu, Radu Timofte, Nicu Sebe, Ming-Hsuan Yang, Luc Van Gool, et al. NTIRE 2025 challenge on cross-domain few-shot object detection: Methods and results. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition ...

  7. [16]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. 11

  8. [17]

    Evalmuse-40k: A reliable and fine-grained benchmark with comprehensive human annotations for text- to-image generation model evaluation, 2024

    Shuhao Han, Haotian Fan, Jiachen Fu, Liang Li, Tao Li, Jun- hui Cui, Yunqiu Wang, Yang Tai, Jingwei Sun, Chunle Guo, and Chongyi Li. Evalmuse-40k: A reliable and fine-grained benchmark with comprehensive human annotations for text- to-image generation model evaluation, 2024. 2...

  9. [18]

    NTIRE 2025 challenge on text to image generation model quality assess- ment

    Shuhao Han, Haotian Fan, Fangyuan Kong, Wenjie Liao, Chunle Guo, Chongyi Li, Radu Timofte, et al. NTIRE 2025 challenge on text to image generation model quality assess- ment. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) Workshops ,

  10. [19]

    Latent video diffusion models for high-fidelity video generation with arbitrary lengths

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity video generation with arbitrary lengths. arXiv preprint arXiv:2211.13221, 2022. 2

  11. [20]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 6

  12. [21]

    Tifa: Accu- rate and interpretable text-to-image faithfulness evaluation with question answering

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accu- rate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, page...

  13. [22]

    NTIRE 2025 challenge on video quality enhancement for video con- ferencing: Datasets, methods and results

    Varun Jain, Zongwei Wu, Quan Zou, Louis Florentin, Hen- rik Turbell, Sandeep Siddhartha, Radu Timofte, et al. NTIRE 2025 challenge on video quality enhancement for video con- ferencing: Datasets, methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision...

  14. [23]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Ad- vances in Neural Information Processing Systems, 36, 2024. 2

  15. [24]

    NTIRE 2025 challenge on efficient burst hdr and restoration: Datasets, methods, and results

    Sangmin Lee, Eunpil Park, Angel Canelo, Hyunhee Park, Youngjo Kim, Hyungju Chun, Xin Jin, Chongyi Li, Chun-Le Guo, Radu Timofte, et al. NTIRE 2025 challenge on efficient burst hdr and restoration: Datasets, methods, and results. In Proceedings of the IEEE/CVF Conference on Com...

  16. [25]

    Genai-bench: Evaluating and improv- ing compositional text-to-visual generation

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Gra- ham Neubig, et al. Genai-bench: Evaluating and improv- ing compositional text-to-visual generation. arXiv preprint arXiv:2406.13743, 2024. 2, 6

  17. [26]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. 9

  18. [27]

    Agiqa-3k: An open database for ai-generated image quality assessment

    Chunyi Li, Zicheng Zhang, Haoning Wu, Wei Sun, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, and Weisi Lin. Agiqa-3k: An open database for ai-generated image quality assessment. IEEE Transactions on Circuits and Sys- tems for Video Technology, 2023. 2

  19. [28]

    Aigiqa-20k: A large database for ai- generated image quality assessment

    Chunyi Li, Tengchuan Kou, Yixuan Gao, Yuqin Cao, Wei Sun, Zicheng Zhang, Yingjie Zhou, Zhichao Zhang, Haon- ing Wu, Weixia Zhang, Xiaohong Liu, Xiongkuo Min, and Guangtao Zhai. Aigiqa-20k: A large database for ai- generated image quality assessment. In Proceedings of the IEEE/...

  20. [29]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 2, 11

  21. [30]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 2, 10

  22. [31]

    NTIRE 2025 challenge on day and night raindrop removal for dual-focused images: Methods and results

    Xin Li, Yeying Jin, Xin Jin, Zongwei Wu, Bingchen Li, Yufei Wang, Wenhan Yang, Yu Li, Zhibo Chen, Bihan Wen, Robby Tan, Radu Timofte, et al. NTIRE 2025 challenge on day and night raindrop removal for dual-focused images: Methods and results. In Proceedings of the IEEE/CVF Conf...

  23. [32]

    NTIRE 2025 challenge on short-form ugc video quality assessment and enhancement: Kwaisr dataset and study

    Xin Li, Xijun Wang, Bingchen Li, Kun Yuan, Yizhen Shao, Suhang Yao, Ming Sun, Chao Zhou, Radu Timofte, and Zhibo Chen. NTIRE 2025 challenge on short-form ugc video quality assessment and enhancement: Kwaisr dataset and study. In Proceedings of the IEEE/CVF Conference on Com- p...

  24. [33]

    NTIRE 2025 challenge on short-form ugc video quality assessment and enhancement: Methods and results

    Xin Li, Kun Yuan, Bingchen Li, Fengbin Guan, Yizhen Shao, Zihao Yu, Xijun Wang, Yiting Lu, Wei Luo, Suhang Yao, Ming Sun, Chao Zhou, Zhibo Chen, Radu Timofte, et al. NTIRE 2025 challenge on short-form ugc video quality assessment and enhancement: Methods and results. In Pro- c...

  25. [34]

    NTIRE 2025 the 2nd restore any image model (RAIM) in the wild challenge

    Jie Liang, Radu Timofte, Qiaosi Yi, Zhengqiang Zhang, Shuaizheng Liu, Lingchen Sun, Rongyuan Wu, Xindong Zhang, Hui Zeng, Lei Zhang, et al. NTIRE 2025 the 2nd restore any image model (RAIM) in the wild challenge. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion a...

  26. [35]

    Rich human feedback for text-to-image generation

    Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, et al. Rich human feedback for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

  27. [36]

    Evaluating text-to-visual generation with image-to-text gen- eration

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text gen- eration. In European Conference on Computer Vision, pages 366–384. Springer, 2024. 2

  28. [37]

    NTIRE 2025 XGC quality assessment chal- lenge: Methods and results

    Xiaohong Liu, Xiongkuo Min, Qiang Hu, Xiaoyun Zhang, Jie Guo, et al. NTIRE 2025 XGC quality assessment chal- lenge: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025. 2

  29. [38]

    NTIRE 2025 challenge on low light image enhancement: Methods and results

    Xiaoning Liu, Zongwei Wu, Florin-Alexandru Vasluianu, Hailong Yan, Bin Ren, Yulun Zhang, Shuhang Gu, Le Zhang, Ce Zhu, Radu Timofte, et al. NTIRE 2025 challenge on low light image enhancement: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion ...

  30. [39]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021. 14

  31. [40]

    Ovis: Structural em- bedding alignment for multimodal large language model

    Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. Ovis: Structural em- bedding alignment for multimodal large language model. arXiv:2405.20797, 2024. 5

  32. [41]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

  33. [42]

    The tenth NTIRE 2025 efficient super- resolution challenge report

    Bin Ren, Hang Guo, Lei Sun, Zongwei Wu, Radu Timo- fte, Yawei Li, et al. The tenth NTIRE 2025 efficient super- resolution challenge report. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025. 2

  34. [43]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

  35. [44]

    NTIRE 2025 challenge on UGC video enhancement: Meth- ods and results

    Nickolay Safonov, Alexey Bryntsev, Andrey Moskalenko, Dmitry Kulikov, Dmitriy Vatolin, Radu Timofte, et al. NTIRE 2025 challenge on UGC video enhancement: Meth- ods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Work- s...

  36. [45]

    NTIRE 2025 challenge on event-based image deblurring: Methods and results

    Lei Sun, Andrea Alfarano, Peiqi Duan, Shaolin Su, Kaiwei Wang, Boxin Shi, Radu Timofte, Danda Pani Paudel, Luc Van Gool, et al. NTIRE 2025 challenge on event-based image deblurring: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

  37. [46]

    The tenth ntire 2025 image denoising challenge report

    Lei Sun, Hang Guo, Bin Ren, Luc Van Gool, Radu Timo- fte, Yawei Li, et al. The tenth ntire 2025 image denoising challenge report. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025. 2

  38. [47]

    Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed database training

    Wei Sun, Xiongkuo Min, Danyang Tu, Siwei Ma, and Guangtao Zhai. Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed database training. IEEE Journal of Selected Topics in Sig- nal Processing, 2023. 2

  39. [48]

    Siglip 2: Multilingual vision-language en- coders with improved semantic understanding, localization, and dense features

    Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muham- mad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language en- coders with improved semantic understanding, localization, ...

  40. [49]

    NTIRE 2025 image shadow removal challenge report

    Florin-Alexandru Vasluianu, Tim Seizinger, Zhuyun Zhou, Cailian Chen, Zongwei Wu, Radu Timofte, et al. NTIRE 2025 image shadow removal challenge report. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025. 2

  41. [50]

    NTIRE 2025 ambi- ent lighting normalization challenge

    Florin-Alexandru Vasluianu, Tim Seizinger, Zhuyun Zhou, Zongwei Wu, Radu Timofte, et al. NTIRE 2025 ambi- ent lighting normalization challenge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025. 2

  42. [51]

    Hierarchical curriculum learning for no-reference image quality assessment

    Juan Wang, Zewen Chen, Chunfeng Yuan, Bing Li, Wentao Ma, and Weiming Hu. Hierarchical curriculum learning for no-reference image quality assessment. International Jour- nal of Computer Vision, pages 1–20, 2023. 14

  43. [52]

    Aigciqa2023: A large-scale image quality assessment database for ai generated images: from the perspectives of quality, authenticity and correspondence

    Jiarui Wang, Huiyu Duan, Jing Liu, Shi Chen, Xiongkuo Min, and Guangtao Zhai. Aigciqa2023: A large-scale image quality assessment database for ai generated images: from the perspectives of quality, authenticity and correspondence. In CAAI International Conference on Artificial...

  44. [53]

    NTIRE 2025 challenge on light field image super-resolution: Methods and results

    Yingqian Wang, Zhengyu Liang, Fengyuan Zhang, Lvli Tian, Longguang Wang, Juncheng Li, Jungang Yang, Radu Timofte, Yulan Guo, et al. NTIRE 2025 challenge on light field image super-resolution: Methods and results. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision a...

  45. [54]

    Unified reward model for multimodal understanding and generation

    Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multimodal understanding and generation. arXiv preprint arXiv:2503.05236, 2025. 9

  46. [55]

    Diffu- siondb: A large-scale prompt gallery dataset for text-to- image generative models

    Zijie J Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffu- siondb: A large-scale prompt gallery dataset for text-to- image generative models. arXiv preprint arXiv:2210.14896,

  47. [56]

    Revisiting text-to-image evaluation with gecko: On met- rics, prompts, and human ratings

    Olivia Wiles, Chuhan Zhang, Isabela Albuquerque, Ivana Kaji´c, Su Wang, Emanuele Bugliarello, Yasumasa Onoe, Pinelopi Papalampidi, Ira Ktena, Chris Knutsen, et al. Revisiting text-to-image evaluation with gecko: On met- rics, prompts, and human ratings. arXiv preprint arXiv:24...

  48. [58]

    Q-align: Teaching lmms for visual scoring via discrete text-defined levels

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023. 2, 3, 7, 8

  49. [59]

    Human preference score: Better aligning text- to-image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hong- sheng Li. Human preference score: Better aligning text- to-image models with human preference. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 2096–2105, 2023. 2

  50. [60]

    Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understanding

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302, 2024. 8

  51. [61]

    Florence-2: Advancing a unified representation for a variety of vision tasks

    Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. Florence-2: Advancing a unified representation for a variety of vision tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 481...

  52. [62]

    Sam2-unet: Segment anything 2 makes strong encoder for natural and medical image segmentation

    Xinyu Xiong, Zihuang Wu, Shuangyi Tan, Wenxue Li, Fei- long Tang, Ying Chen, Siying Li, Jie Ma, and Guanbin Li. Sam2-unet: Segment anything 2 makes strong encoder for natural and medical image segmentation. arXiv preprint arXiv:2408.08870, 2024. 14

  53. [63]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. Advances in Neural Information Pro- cessing Systems, 36, 2024. 2

  54. [64]

    Boosting image quality assessment through efficient transformer adaptation with lo- cal feature enhancement

    Kangmin Xu, Liang Liao, Jing Xiao, Chaofeng Chen, Haon- ing Wu, Qiong Yan, and Weisi Lin. Boosting image quality assessment through efficient transformer adaptation with lo- cal feature enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

  55. [65]

    A sanity check for ai-generated image detection

    Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xi- aolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai-generated image detection. arXiv preprint arXiv:2406.19435, 2024. 15

  56. [66]

    NTIRE 2025 challenge on single image reflection removal in the wild: Datasets, methods and results

    Kangning Yang, Jie Cai, Ling Ouyang, Florin-Alexandru Vasluianu, Radu Timofte, Jiaming Ding, Huiming Sun, Lan Fu, Jinlong Li, Chiu Man Ho, Zibo Meng, et al. NTIRE 2025 challenge on single image reflection removal in the wild: Datasets, methods and results. In Proceedings of th...

  57. [67]

    Maniqa: Multi-dimension attention network for no-reference image quality assessment

    Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. Maniqa: Multi-dimension attention network for no-reference image quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pag...

  58. [68]

    mplug- owl2: Revolutionizing multi-modal large language model with modality collaboration

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. mplug- owl2: Revolutionizing multi-modal large language model with modality collaboration. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition ,...

  59. [70]

    Teaching large language models to regress accurate image quality scores using score distribution

    Zhiyuan You, Xin Cai, Jinjin Gu, Tianfan Xue, and Chao Dong. Teaching large language models to regress accurate image quality scores using score distribution. arXiv preprint arXiv:2501.11561, 2025. 15

  60. [71]

    Instruction-augmented multimodal alignment for image-text and element matching

    Xinli Yue, JianHui Sun, Junda Lu, Liangchao Yao, Fan Xia, Tianyi Wang, Fengyun Rao, Jing Lyu, and Yuetang Deng. Instruction-augmented multimodal alignment for image-text and element matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ...

  61. [72]

    NTIRE 2025 challenge on hr depth from images of specular and transparent surfaces

    Pierluigi Zama Ramirez, Fabio Tosi, Luigi Di Stefano, Radu Timofte, Alex Costanzino, Matteo Poggi, Samuele Salti, Ste- fano Mattoccia, et al. NTIRE 2025 challenge on hr depth from images of specular and transparent surfaces. In Pro- ceedings of the IEEE/CVF Conference on Compu...

  62. [73]

    Blind image quality assessment using a deep bilinear convolutional neural network

    Weixia Zhang, Kede Ma, Jia Yan, Dexiang Deng, and Zhou Wang. Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Transactions on Cir- cuits and Systems for Video Technology, 30(1):36–47, 2020. 2

  63. [74]

    Blind image quality assessment via vision- language correspondence: A multitask learning perspective

    Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, and Kede Ma. Blind image quality assessment via vision- language correspondence: A multitask learning perspective. In IEEE Conference on Computer Vision and Pattern Recog- nition, pages 14071–14081, 2023. 2

  64. [75]

    A perceptual quality assessment exploration for aigc images

    Zicheng Zhang, Chunyi Li, Wei Sun, Xiaohong Liu, Xiongkuo Min, and Guangtao Zhai. A perceptual quality assessment exploration for aigc images. In 2023 IEEE In- ternational Conference on Multimedia and Expo Workshops (ICMEW), pages 440–445. IEEE, 2023. 2

  65. [76]

    Tokenfocus-vqa: Enhancing text-to-image alignment with position-aware focus and multi-perspective aggregations on lvlms, 2025

    Zijian Zhang, Xuhui Zheng, Xuecheng Wu, Chong Peng, and Xuezhi Cao. Tokenfocus-vqa: Enhancing text-to-image alignment with position-aware focus and multi-perspective aggregations on lvlms, 2025. 6

  66. [77]

    Study group learning: Improving retinal vessel segmentation trained with noisy labels

    Yuqian Zhou, Hanchao Yu, and Humphrey Shi. Study group learning: Improving retinal vessel segmentation trained with noisy labels. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October ...

  67. [78]

    Detrs with col- laborative hybrid assignments training

    Zhuofan Zong, Guanglu Song, and Yu Liu. Detrs with col- laborative hybrid assignments training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6748–6758, 2023. 11

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.