Pith. sign in

REVIEW 3 major objections 5 minor 86 references

STORM: Benchmarking Visual Rating of MLLMs with a Comprehensive Ordinal Regression Dataset

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read STORM shows that fine-tuning a multimodal language model on 655K ordinal question-answer pairs with coarse-to-fine reasoning yields one zero-shot visual rating model across five domains.

desk verdict Big useful dataset, but the zero-shot 'universal rating' claim needs a prompt-controlled comparison before I'd believe it. read the letter →

arxiv 2506.01738 v1 pith:OZ2F6XCB submitted 2025-06-02 cs.CV

Jinhong Wang , Shuo Tong , Jian liu , Dongqi Tang , Jintai Chen , Haochao Ying , Hongxia Xu , Danny Chen
show 1 more author
Jian Wu
This is my paper
classification cs.CV
keywords ordinalregressionvisualratingmultimodallargelanguagemodelschain-of-thoughtzero-shotgeneralizationimagequalityassessmentfacialageestimationmedicalgrading
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

STORM is a dataset, benchmark, and training pipeline built around a simple claim: visual rating tasks — judging image quality, image aesthetics, a person's age, a medical image's disease grade, or a historical photo's decade — are all ordinal regression problems, and a multimodal large language model (MLLM) fine-tuned on all of them together can learn one general rating ability. To test this, the authors gather 655K image-level question-answer pairs from 14 existing datasets across five domains, rewrite each numeric label as a text-plus-number definition, and insert an intermediate coarse-to-fine reasoning step into every answer. They fine-tune the Qwen2.5-VL-3B backbone on the collection, and the central evidence is that the resulting model beats the same backbone trained without the coarse-to-fine pipeline, particularly on datasets whose training splits were withheld, while training on all five domains beats training on any single one. If true, this means a single model can be pointed at a new rating task and perform it without per-task training, needing only a textual definition of the new task's labels.

What carries the argument

The load-bearing mechanism is the coarse-to-fine Chain-of-Thought (CoT) pipeline. For every sample the question is rewritten so the model first chooses a coarse merged category from text-labelled candidates (for age: Child, Teenager, Youth, Adult, Middle, Elder), then produces the fine text-plus-number answer, and is explicitly instructed to keep the coarse thought and final answer consistent. Candidate names and textual definitions of numeric labels are generated with GPT and manually adjusted before being stored in the dataset. This turns each rating problem into a two-level hierarchical classification with progressively pruned candidate sets, preserves the ordinal commonality of labels across tasks, and supplies the model with interpretable intermediate thoughts.

What would settle it

Train two models on the same STORM data — one with coarse-to-fine CoT answers, one with flat answers — and test both on a held-out task whose label definitions are deliberately unusual (e.g. a 7-level severity scale with nonstandard wording) and whose coarse candidates are withheld from the prompt. If the CoT advantage persists without the handcrafted candidates, the ordinal-thinking claim survives; if it disappears, the gains come from the per-task prompt text rather than from a learned general rating ability.

Watch

Extended reading notes

Core claim

The authors' central discovery is that the coarse-to-fine chain-of-thought data, not just the volume of rating data, is what transfers. Fine-tuning on the full STORM collection gives the best accuracy and mean absolute error on both in-domain and held-out datasets: on the held-out Aesthetic dataset the fine-tuned model reaches nearly 2.5 times the accuracy of the Qwen2.5-VL pipeline without CoT, and on the held-out HCI historical-dating domain accuracy rises from 0.273 to 0.424. The ablations attribute this to three choices working together: pairing text definitions with numeric labels (the two together beat either alone), inserting a coarse prediction stage with a consistency check, and joint training across all five domains rather than a single domain. The authors conclude that the model has learned a universal ordinal-thinking paradigm that applies to any rating protocol whose labels are ordered.

Load-bearing premise

The claim that STORM imparts a general ordinal-thinking paradigm assumes that the handcrafted coarse categories and text label definitions supplied at inference to each target dataset do not, by themselves, carry most of the task-specific information; if the zero-shot gains depend on these per-domain prompt templates rather than on learned ordinal understanding, the evidence for a universal visual-rating ability collapses.

Editorial extensions

If this is right

  • A model fine-tuned once on STORM can be applied to a new visual rating task without training on that task; the held-out HCI historical-dating domain is the paper's demonstration.
  • Writing labels as text-plus-number instead of numbers alone improves accuracy across task families, so future rating datasets should ship textual label definitions alongside numeric scores.
  • Training jointly across heterogeneous ordinal domains improves each domain's accuracy relative to single-domain training, making data diversity across rating tasks part of the recipe.
  • Full fine-tuning outperforms LoRA on this benchmark, indicating that parameter-efficient adaptation is not enough for visual rating in this setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A skeptical reading, supported by the paper's own Appendix F, is that the handcrafted per-domain text definitions and coarse candidates supplied at inference might be doing much of the transfer work; a test that withholds or randomizes those candidates would separate a learned ordinal disposition from prompt-conditioned label mapping.
  • If the ordinal-thinking story is right, the same coarse-to-fine recipe should transfer to rating tasks outside the five tested domains, such as video or audio quality scoring; a STORM-style model evaluated there is a direct out-of-distribution test.
  • The reported 3B and 7B variants perform nearly alike, suggesting the data recipe rather than model scale drives the gains; probing smaller backbones would show how far the recipe alone goes.
  • The coarse/fine consistency step could be reused at inference as a confidence signal: predictions where the coarse thought and fine answer disagree are plausibly low-confidence and could be routed to human review.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces STORM, a large-scale dataset and benchmark for training and evaluating multi-modal large language models (MLLMs) on ordinal regression visual rating tasks. STORM comprises 14 datasets across five domains (image quality, aesthetics, facial age, medical disease grading, historical date estimation) with 655K image-level VQA pairs, and proposes a coarse-to-fine Chain-of-Thought (CoT) pipeline in which the model first predicts a coarse category and then a fine-grained label, using both numeric and text-defined label candidates. The authors fine-tune Qwen2.5-VL-3B on this data and report accuracy and MAE on in-domain and held-out datasets, claiming that the resulting STORM-3B exhibits general, zero-shot visual rating ability. The paper also provides ablations on prompt strategies and training strategies, and releases code, data, and model checkpoints.

Significance. If the central claims hold, STORM is a valuable community resource: it is the first benchmark to unify multiple ordinal visual rating domains for MLLMs, and the release of code, data, and checkpoints supports reproducibility and downstream research. The ablations in Tables 4 and 5 provide useful evidence that the coarse-to-fine CoT and combined text-plus-numeric labels contribute to the model's performance. However, the paper's strongest claim—that fine-tuning on STORM imparts a universal, zero-shot ordinal-thinking paradigm—is weakened by an evaluation design that does not control for the task-specific candidate prompts given to the model at inference. The reported zero-shot gains may therefore reflect prompt information as much as a learned general ability. The dataset and empirical comparisons are still worthwhile contributions, but the central generality claim needs stronger evidence.

major comments (3)
  1. [Section 5.2, Appendix E] The claim that STORM delivers general zero-shot visual rating ability is confounded with the hand-crafted per-dataset candidate prompts. In the reported zero-shot comparisons, STORM is evaluated with detailed coarse and fine candidate lists (e.g., age groups in Fig. 2, IQA 'below fair/fair/above fair' in E.1, MDG 'normal/early/late' in E.4, HDE 'early/mid/late' in E.5), but the zero-shot baselines (Qwen2.5-VL Zero, LLaVA-1.5 Zero) are not documented as receiving the same candidate prompts. Since Appendix F concedes that performance varies with the text definitions of labels and demands accurate prompt construction by the user, the observed zero-shot gains (e.g., Aesthetics ACC 0.081→0.658; HCI ACC 0.265→0.341) could arise substantially from the prompt content rather than from a learned universal ordinal-thinking paradigm. Please provide a controlled comparison in which the same coarse-to-fine candidate prompts are given to the untuned baselines, and report whether the gains persist. This is necessary to support the central claim of a general paradigm.
  2. [Checklist item 3(c), Tables 2 and 3] The paper reports no error bars, confidence intervals, or significance tests, and the checklist explicitly answers 'No' to error bars. Many of the performance differences between STORM and the Qwen2.5-VL Lite baseline are small (e.g., DeepDR ACC 0.435 vs 0.433, MAE 1.280 vs 1.285; APTOS ACC 0.508 vs 0.506). Without multi-seed runs or a paired significance test, the Section 5.2 claim of 'remarkable improvement' is not statistically supported on those datasets. Please report mean and standard deviation over at least three independent runs (or equivalent) for the main tables and for the zero-shot datasets, and indicate which differences are significant.
  3. [Section 5.2, Tables 6-7] The term 'zero-shot' is used imprecisely. Most held-out datasets (KonIQ, UTK, Aesthetic, APTOS) are from domains that are present in the training split (IQA, FAE, IAA, MDG), so the evaluation demonstrates held-out-dataset generalization rather than cross-domain generalization. Only HCI is a genuinely unseen domain, and it has only 132 test images (Table 7). The claim of a universal ordinal-thinking paradigm therefore rests heavily on a single small dataset. Please either restrict the 'zero-shot' language to 'held-out dataset' where appropriate, or add experiments on additional fully unseen domains to support the universality claim.
minor comments (5)
  1. [Figure 1] The text '0-116 Yeas Old' appears in Figure 1; it should be 'Years'.
  2. [Appendix D.1] Tables 8 and 9 both have the caption 'Table 8'; the second should be labeled Table 9.
  3. [References] Reference [3] is a duplicate of [2] (both are the GPT-4 technical report), and references [46] and [47] are the same CLIP paper; please consolidate the duplicates.
  4. [Appendix E] The prompts in E.1 (IQA), E.2 (IAA), and E.4 (MDG) incorrectly say 'proceed to make a final age prediction' where the intended phrase is 'final quality prediction', 'final level prediction', or 'final stage prediction'. Please fix these copy-paste errors.
  5. [Section 5.2] The phrase 'nearly 2.5× performance' is defined without a metric; for Aesthetics, the ACC ratio is about 8× (0.658/0.081) and the MAE ratio is about 3.7× (1.337/0.360). Please specify the metric and the exact comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: STORM's zero-shot gains are empirical results, not reductions to its own inputs.

full rationale

The paper's central claims are empirical: fine-tuning Qwen2.5-VL on STORM's coarse-to-fine CoT data improves accuracy and MAE on held-out test splits (Tables 2 and 3) relative to untuned Qwen2.5-VL and LLaVA-1.5 baselines. Nothing in the derivation makes those numbers equal to the training labels by construction. The coarse categories are deterministic merges of the fine labels (Section 4: 'Coarse labels are equivalent to merged neighboring categories'), but at inference the coarse thought is generated by the model and is not provided from ground truth; the final answer can be wrong. The per-dataset prompts in Appendix E supply label candidates and text definitions as part of the proposed pipeline; this is a possible confound for the 'universal paradigm' interpretation, and the authors concede in Appendix F that performance 'fluctuat[es] according to different text definitions generated of labels.' However, this is a scope and evaluation-design limitation, not a circular derivation: the model must still map the image to the correct candidate from the supplied label set. The paper does not rely on load-bearing self-citations; its comparisons are against external models and external datasets. The nearest concern is overclaiming generality from held-out sets that share domain and label semantics with training data, which is a correctness risk rather than circularity. Therefore no circular step is exhibited, and the score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central benchmark claim rests on hand-chosen label binning and coarse-candidate prompts, plus assumptions about domain representativeness and zero-shot protocol fairness. No new physical entities are introduced, and no mathematical axioms beyond standard empirical evaluation are used.

free parameters (3)
  • MOS-to-level binning thresholds = not reported
    Section 3.2 converts continuous MOS values in IQA/IAA datasets into five ordinal levels; the exact thresholds are not given, yet they define the supervised target and the benchmark's ground truth.
  • Coarse-to-fine candidate intervals per task = e.g., FAE: Child (0-10), Teenager (11-20), Youth (21-30), Adult (31-44), Middle (45-60), Elder (60+); IQA: below fair…
    Hand-chosen in Section 3.2 and Fig. 2 to partition fine labels; they appear in both the training data and the inference prompt, so they act as manually set parameters that influence all results.
  • Text label definitions per rating level = e.g., 'excellent' for the top quality level; 'normal/mild/moderate/severe/extreme' for DR grades
    Generated by GPT and manually adjusted (Section 4, Text Candidate Generation); these definitions are fixed design choices that shape the model's semantics and are not derived from data.
assumptions (4)
  • domain assumption The five selected domains (IQA, IAA, FAE, MDG, HDE) represent 'universal visual rating'.
    This scope assumption underlies the universal claim and is introduced in Section 3.1 and the abstract.
  • domain assumption Binning continuous MOS scores into five text-defined levels preserves the task's ordinal signal.
    Section 3.2 uses five discrete levels for IQA/IAA; if the binning is arbitrary or inconsistent, the learned labels are only proxies for human ratings.
  • domain assumption Supplying the target label candidates and ranges during zero-shot evaluation is a fair test of generalization.
    Used in Fig. 2 and Appendix E; the model receives coarse and fine candidates specific to the evaluation dataset, which is task-specific information.
  • domain assumption The fine-tuned model's textual answers can be reliably parsed back into numeric labels.
    The paper reports ACC and MAE but does not describe output parsing or how malformed CoT responses are handled.

how reviews work

0 comments
Cite this review

Pith. "Pith review of STORM: Benchmarking Visual Rating of MLLMs with a Comprehensive Ordinal Regression Dataset." pith.science (2026). https://pith.science/paper/OZ2F6XCB

@misc{pith2026250601738,
  author       = {Pith},
  title        = {Pith review of: STORM: Benchmarking Visual Rating of MLLMs with a Comprehensive Ordinal Regression Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OZ2F6XCB}},
  note         = {Machine review of arXiv:2506.01738}
}
read the original abstract

Visual rating is an essential capability of artificial intelligence (AI) for multi-dimensional quantification of visual content, primarily applied in ordinal regression (OR) tasks such as image quality assessment, facial age estimation, and medical image grading. However, current multi-modal large language models (MLLMs) under-perform in such visual rating ability while also suffering the lack of relevant datasets and benchmarks. In this work, we collect and present STORM, a data collection and benchmark for Stimulating Trustworthy Ordinal Regression Ability of MLLMs for universal visual rating. STORM encompasses 14 ordinal regression datasets across five common visual rating domains, comprising 655K image-level pairs and the corresponding carefully curated VQAs. Importantly, we also propose a coarse-to-fine processing pipeline that dynamically considers label candidates and provides interpretable thoughts, providing MLLMs with a general and trustworthy ordinal thinking paradigm. This benchmark aims to evaluate the all-in-one and zero-shot performance of MLLMs in scenarios requiring understanding of the essential common ordinal relationships of rating labels. Extensive experiments demonstrate the effectiveness of our framework and shed light on better fine-tuning strategies. The STORM dataset, benchmark, and pre-trained models are available on the following webpage to support further research in this area. Datasets and codes are released on the project page: https://storm-bench.github.io/.

Figures

Figures reproduced from arXiv: 2506.01738 by the authors.

Figure 1
Figure 1. An overview of our STORM benchmark. STORM consists of four key components: 1) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A data example with the original VQA compared with our coarse-to-fine CoT VQA. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The model pipeline of STORM. It first extracts visual tokens from an input image and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Visualization results of coarse-to-fine CoT on different datasets. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Overview of Qwen-2.5-VL pipeline. B Framework details B.1 Model details For LLaVA-1.5-7B, we choose the pre-trained ViT-L/14 of CLIP [47] as the vision encoder and Vicuna-7B [10] as our LLM, which has better instruction following capabilities in language tasks compared…
Figure 6
Figure 6. Figure 6: Statistics of the IQA domain datasets. IAA Aesthetic AVA TAD66K [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Statistics of the IAA domain datasets. C.2 Datasets Distribution visualization. In this section, we provide a visualization of the data statistics. We partition the category distribution of each dataset in [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Statistics of the MDG domain datasets. HCI Adience HDE FAE [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Statistics of the FAE and HDE domain datasets. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Confusion matrixes visualization results of the STORM on the KonIQ dataset. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Confusion matrixes visualization results of the STORM on the Aesthetic dataset. [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Confusion matrixes visualization results of the STORM on the Adience dataset. [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Confusion matrixes visualization results of the STORM on the APTOS dataset. [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Confusion matrixes visualization results of the STORM on the HCI dataset. [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

86 extracted references · 54 canonical work pages

  1. [1]

    ITU-R Rec

    Recommendation 500-10: Methodology for the subjective assessment of the quality of television pictures. ITU-R Rec. BT.500, 2000

    2000

  2. [3]

    GPT-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [4]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  4. [5]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv:2308.12966, 2023

    arXiv 2023

  5. [6]

    Face recognition and retrieval using cross-age reference coding with cross-age celebrity dataset

    Bor-Chun Chen, Chu-Song Chen, and Winston H Hsu. Face recognition and retrieval using cross-age reference coding with cross-age celebrity dataset. IEEE Transactions on Multimedia, 17(6):804–815, 2015

    2015

  6. [7]

    Using ranking-CNN for age estimation

    Shixing Chen, Caojin Zhang, Ming Dong, Jialiang Le, and Mike Rao. Using ranking-CNN for age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5183–5192, 2017

    2017

  7. [8]

    PaLI: A jointly-scaled multilingual language-image model

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. PaLI: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209.06794, 2022

    arXiv 2022

  8. [9]

    LongLoRA: Efficient fine-tuning of long-context large language models

    Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. LongLoRA: Efficient fine-tuning of long-context large language models. arXiv preprint arXiv:2309.12307, 2023

    arXiv 2023

Show all 86 references
  1. [10]

    Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality. See https://vicuna. lmsys. org (accessed 14 April 2023), 2023

    2023

  2. [11]

    Soft labels for ordinal regression

    Raul Diaz and Amit Marathe. Soft labels for ordinal regression. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4738–4747, 2019

    2019

  3. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv...

    2010 arXiv

  4. [13]

    Teach CLIP to develop a number sense for ordinal regression

    Yao Du, Qiang Zhai, Weihang Dai, and Xiaomeng Li. Teach CLIP to develop a number sense for ordinal regression. In European Conference on Computer Vision, pages 1–17. Springer, 2024

    2024

  5. [14]

    Diabetic retinopathy detection (2015)

    Emma Dugas, Jorge Jared, and Will Cukierski. Diabetic retinopathy detection (2015). URL https://kaggle. com/competitions/diabetic-retinopathy-detection, 7

    2015

  6. [15]

    Perceptual quality assessment of smartphone photography

    Yuming Fang, Hanwei Zhu, Yan Zeng, Kede Ma, and Zhou Wang. Perceptual quality assessment of smartphone photography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3677–3686, 2020

    2020

  7. [16]

    A simple approach to ordinal classification

    Eibe Frank and Mark Hall. A simple approach to ordinal classification. In European Conference on Machine Learning, pages 145–156. Springer, 2001. 10

    2001

  8. [17]

    Deep ordinal regression network for monocular depth estimation

    Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao. Deep ordinal regression network for monocular depth estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2002–2011, 2018

    2002

  9. [18]

    Facial age estimation by learning from label distribu- tions

    Xin Geng, Chao Yin, and Zhi-Hua Zhou. Facial age estimation by learning from label distribu- tions. TPAMI, 2013

    2013

  10. [19]

    Massive online crowdsourced study of subjective and objective picture quality

    Deepti Ghadiyaram and Alan C Bovik. Massive online crowdsourced study of subjective and objective picture quality. IEEE Transactions on Image Processing, 25(1):372–387, 2015

    2015

  11. [20]

    A V A: A video dataset of spatio-temporally localized atomic visual actions

    Chunhui Gu, Chen Sun, David A Ross, Carl V ondrick, Caroline Pantofaru, Yeqing Li, Sud- heendra Vijayanarasimhan, George Toderici, Susanna Ricco, Rahul Sukthankar, et al. A V A: A video dataset of spatio-temporally localized atomic visual actions. In Proceedings of the IEEE Co...

    2018

  12. [21]

    Rethinking image aesthetics assessment: Models, datasets and benchmarks

    Shuai He, Yongchang Zhang, Rui Xie, Dongxiang Jiang, and Anlong Ming. Rethinking image aesthetics assessment: Models, datasets and benchmarks. In IJCAI, pages 942–948, 2022

    2022

  13. [22]

    KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment

    Vlad Hosu, Hanhe Lin, Tamas Sziranyi, and Dietmar Saupe. KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Transactions on Image Processing, 29:4041–4056, 2020

    2020

  14. [23]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. LoRA: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  15. [24]

    and Tamirat Tesafaye

    Karl Ricanek Jr. and Tamirat Tesafaye. MORPH: A longitudinal image database of normal adult age-progression. In FG, 2006

    2006

  16. [25]

    APTOS 2019 blindness detection

    Maggie Karthik and Sohier Dane. APTOS 2019 blindness detection. Kaggle https://kaggle. com/competitions/aptos2019-blindness-detection Go to reference in, 5, 2019

    2019

  17. [26]

    Generating images with multimodal language models

    Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Generating images with multimodal language models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  18. [27]

    LISA: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. LISA: Reasoning segmentation via large language model. arXiv preprint arXiv:2308.00692, 2023

    2023 arXiv

  19. [28]

    Deep repulsive clustering of ordered data based on order- identity decomposition

    Seon-Ho Lee and Chang-Su Kim. Deep repulsive clustering of ordered data based on order- identity decomposition. In International Conference on Learning Representations, 2020

    2020

  20. [29]

    Age and gender classification using convolutional neural networks

    Gil Levi and Tal Hassner. Age and gender classification using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 34–42, 2015

    2015

  21. [30]

    BLIP-2: Bootstrapping language- image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023

    2023 arXiv

  22. [31]

    BLIP: Bootstrapping language- image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language- image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, pages 12888–12900. PMLR, 2022

    2022

  23. [32]

    Ordinal regression by extended binary classification

    Ling Li and Hsuan-Tien Lin. Ordinal regression by extended binary classification. Advances in Neural Information Processing Systems, 19, 2006

    2006

  24. [33]

    Learning probabilistic ordi- nal embeddings for uncertainty-aware regression

    Wanhua Li, Xiaoke Huang, Jiwen Lu, Jianjiang Feng, and Jie Zhou. Learning probabilistic ordi- nal embeddings for uncertainty-aware regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13896–13905, 2021

    2021

  25. [34]

    OrdinalCLIP: Learning rank prompts for language-guided ordinal regression

    Wanhua Li, Xiaoke Huang, Zheng Zhu, Yansong Tang, Xiu Li, Jie Zhou, and Jiwen Lu. OrdinalCLIP: Learning rank prompts for language-guided ordinal regression. Advances in Neural Information Processing Systems, 35:35313–35325, 2022. 11

    2022

  26. [35]

    LLaMA-VID: An image is worth 2 tokens in large language models

    Yanwei Li, Chengyao Wang, and Jiaya Jia. LLaMA-VID: An image is worth 2 tokens in large language models. arXiv preprint arXiv:2311.17043, 2023

    2023 arXiv

  27. [36]

    Order learning and its application to age estimation

    Kyungsun Lim, Nyeong-Ho Shin, Young-Yoon Lee, and Chang-Su Kim. Order learning and its application to age estimation. In International Conference on Learning Representations, 2019

    2019

  28. [37]

    Improved baselines with visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023

    2023

  29. [38]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023

    2023

  30. [39]

    DeepDRiD: Diabetic retinopathy—grading and image quality estimation challenge

    Ruhan Liu, Xiangning Wang, Qiang Wu, Ling Dai, Xi Fang, Tao Yan, Jaemin Son, Shiqi Tang, Jiang Li, Zijian Gao, et al. DeepDRiD: Diabetic retinopathy—grading and image quality estimation challenge. Patterns, 3(6), 2022

    2022

  31. [40]

    Deep ordinal regression based on data relationship for small datasets

    Yanzhu Liu, Adams Wai-Kin Kong, and Chi Keong Goh. Deep ordinal regression based on data relationship for small datasets. In IJCAI, pages 2372–2378, 2017

    2017

  32. [41]

    A constrained deep neural network for ordinal regression

    Yanzhu Liu, Adams Wai Kin Kong, and Chi Keong Goh. A constrained deep neural network for ordinal regression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 831–839, 2018

    2018

  33. [42]

    Probabilistic deep ordinal regression based on Gaussian processes

    Yanzhu Liu, Fan Wang, and Adams Wai Kin Kong. Probabilistic deep ordinal regression based on Gaussian processes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5301–5309, 2019

    2019

  34. [43]

    Cheap and quick: Efficient vision-language instruction tuning for large language models

    Gen Luo, Yiyi Zhou, Tianhe Ren, Shengxin Chen, Xiaoshuai Sun, and Rongrong Ji. Cheap and quick: Efficient vision-language instruction tuning for large language models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  35. [44]

    Dating historical color images

    Frank Palermo, James Hays, and Alexei A Efros. Dating historical color images. In European Conference on Computer Vision, pages 499–512. Springer, 2012

    2012

  36. [45]

    StraIT: Non-autoregressive generation with stratified image Transformer

    Shengju Qian, Huiwen Chang, Yuanzhen Li, Zizhao Zhang, Jiaya Jia, and Han Zhang. StraIT: Non-autoregressive generation with stratified image Transformer. arXiv preprint arXiv:2303.00750, 2023

    2023 arXiv

  37. [46]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021

    2021

  38. [47]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pa...

    2021

  39. [48]

    Deep expectation of real and apparent age from a single image without facial landmarks

    Rasmus Rothe, Radu Timofte, and Luc Van Gool. Deep expectation of real and apparent age from a single image without facial landmarks. IJCV, 2018

    2018

  40. [49]

    HuggingGPT: Solving ai tasks with ChatGPT and its friends in hugging face

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. HuggingGPT: Solving ai tasks with ChatGPT and its friends in hugging face. Advances in Neural Information Processing Systems, 36, 2024

    2024

  41. [50]

    Moving window regression: A novel approach to ordinal regression

    Nyeong-Ho Shin, Seon-Ho Lee, and Chang-Su Kim. Moving window regression: A novel approach to ordinal regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18760–18769, 2022

    2022

  42. [51]

    Gemini: A family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  43. [52]

    LLaMA: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 12

    2023 arXiv

  44. [53]

    Ord2Seq: Regarding ordinal regression as label sequence prediction

    Jinhong Wang, Yi Cheng, Jintai Chen, TingTing Chen, Danny Chen, and Jian Wu. Ord2Seq: Regarding ordinal regression as label sequence prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865–5875, 2023

    2023

  45. [54]

    Learning-to- rank meets language: Boosting language-driven ordering alignment for ordinal classification

    Rui Wang, Peipei Li, Huaibo Huang, Chunshui Cao, Ran He, and Zhaofeng He. Learning-to- rank meets language: Boosting language-driven ordering alignment for ordinal classification. Advances in Neural Information Processing Systems, 36, 2023

    2023

  46. [55]

    CogVLM: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. CogVLM: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079, 2023

    2023 arXiv

  47. [56]

    VisionLLM: Large language model is also an open-ended decoder for vision-centric tasks

    Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Yu Qiao, et al. VisionLLM: Large language model is also an open-ended decoder for vision-centric tasks. Advances in Neural Information Processing Systems, 36, 2024

    2024

  48. [57]

    Visual ChatGPT: Talking, drawing and editing with visual foundation models

    Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual ChatGPT: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671, 2023

    2023 arXiv

  49. [58]

    Q-Bench: A benchmark for general-purpose foundation models on low-level vision

    Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, and Weisi Lin. Q-Bench: A benchmark for general-purpose foundation models on low-level vision. 2023

    2023

  50. [59]

    Q-Instruct: Improving low-level visual abilities for multi-modality foundation models

    Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, et al. Q-Instruct: Improving low-level visual abilities for multi-modality foundation models. arXiv preprint arXiv:2311.06783, 2023

    2023 arXiv

  51. [60]

    Q-Align: Teaching LMMs for visual scoring via discrete text-defined levels

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-Align: Teaching LMMs for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023

    2023 arXiv

  52. [61]

    MM-REACT: Prompting ChatGPT for multimodal reasoning and action

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. MM-REACT: Prompting ChatGPT for multimodal reasoning and action. arXiv preprint arXiv:2303.11381, 2023

    2023 arXiv

  53. [62]

    Video-LLaMA: An instruction-tuned audio-visual language model for video understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-LLaMA: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.02858, 2023

    2023 arXiv

  54. [63]

    GPT4RoI: Instruction tuning large language model on region-of-interest

    Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo. GPT4RoI: Instruction tuning large language model on region-of-interest. arXiv preprint arXiv:2307.03601, 2023

    2023 arXiv

  55. [64]

    Age progression/regression by conditional adver- sarial autoencoder

    Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/regression by conditional adver- sarial autoencoder. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5810–5818, 2017

    2017

  56. [65]

    MiniGPT-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. MiniGPT-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  57. [66]

    MoV A: Adapting mixture of vision experts to multimodal context.arXiv preprint arXiv:2404.13046, 2024

    Zhuofan Zong, Bingqi Ma, Dazhong Shen, Guanglu Song, Hao Shao, Dongzhi Jiang, Hongsheng Li, and Yu Liu. MoV A: Adapting mixture of vision experts to multimodal context.arXiv preprint arXiv:2404.13046, 2024. 13 NeurIPS Paper Checklist

    2024 arXiv

  58. [67]

    For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? [Yes] (b) Did you describe the limitations of your work? [Yes] See Appendix F. (c) Did you discuss any potential negative societal impacts of...

  59. [68]

    (a) Did you state the full set of assumptions of all theoretical results? [NA] (b) Did you include complete proofs of all theoretical results? [NA]

    If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results? [NA] (b) Did you include complete proofs of all theoretical results? [NA]

  60. [69]

    for benchmarks)

    If you ran experiments (e.g. for benchmarks)... (a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)? [Yes] We have provided the related details in Appendix. The code, trainin...

  61. [70]

    If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? [Yes] (b) Did you mention the license of the assets? [Yes] (c) Did you include any new assets either in the supplem...

  62. [71]

    If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable? [NA] (b) Did you describe any potential participant risks, with links to Institutional Review Board (I...

  63. [73]

    Make the coarse prediction with the candidates:rate0-1(Below Fair), rate2(Fair), rate3-4(Above Fair)

  64. [74]

    Based on the coarse classification, proceed to make a final age prediction with the candidates: rate0(Bad), rate1(Poor), rate2(Fair), rate3(Good), rate4(Excellent)

  65. [75]

    Please note that the coarse thoughts and the final answer should be consistent. Answer: [Coarse answer], [Final answer] E.2 Generating the dataset for IAA <image> You are now an advanced Aesthetic Evaluation Evaluator, and your task is to assess the aes- thetic quality of the ...

  66. [76]

    Make the coarse prediction with the candidates:level0-1(Below Average), level2(Average), level3- 4(Above Average)

  67. [77]

    Based on the coarse classification, proceed to make a final age prediction with the candidates: level0(Unacceptable), level1(Flawed), level2(Average), level3(Professional), level4(Excellent)

  68. [78]

    Please note that the coarse thoughts and the final answer should be consistent. Answer: [Coarse answer], [Final answer] 10 E.3 Generating the dataset for FAE <image> You are an experienced facial analysis expert, and you need to estimate the age group of the person in the prov...

  69. [79]

    Make the coarse prediction with the candidates: Teenager(16-24 years old), Adult(25-47 years old), Elder(48+ years old)

  70. [80]

    Based on the coarse classification, proceed to make a final age prediction with the candidates: from 16 to 77 years old

  71. [81]

    Please note that the coarse thoughts and the final answer should be consistent. Answer: Coarse answer], [Predicted Age] E.4 Generating the dataset for MDG <image> You are an experienced ophthalmologist, and you need to perform disease grading on the provided fundus image. Thes...

  72. [82]

    Make the coarse prediction with the candidates:Normal(stage0), Early(stage1-2), Late(stage3-4)

  73. [83]

    Based on the coarse classification, proceed to make a final age prediction with the candidates: stage0(no retinopathy), stage1(mild NPDR), stage2(moderate NPDR), stage3(severe NPDR) and stage4(PDR)

  74. [84]

    Please note that the coarse thoughts and the final answer should be consistent. Answer: [Coarse answer], [Predicted grade] E.5 Generating the dataset for HDE <image> You are now an advanced history researcher, and you need to grade the provided images by decade. These are all ...

    1930

  75. [85]

    Make the coarse prediction with the candidates: Early(phase0-phase1), Mid(phase2), Late(phase3- phase4)

  76. [86]

    Based on the coarse classification, proceed to make a final age prediction with the candidates: phase0(1930s), phase1(1940s), phase2(1950s), phase3(1960s), and phase4(1970s)

    1930

  77. [87]

    Answer: [Coarse answer], [Predicted Phase] 11 F Limitations The definitions of labels for different domain tasks are quite diverse

    Please note that the coarse thoughts and the final answer should be consistent. Answer: [Coarse answer], [Predicted Phase] 11 F Limitations The definitions of labels for different domain tasks are quite diverse. In scenarios where the definitions of labels for different domain...

  78. [128]

    To conserve GPU memory during fine-tuning, we employ FSDP (Full Shard Data Parallel) with ZeRO3-style

    The Adam optimizer with zero weight decay and a cosine learning rate scheduler are utilized. To conserve GPU memory during fine-tuning, we employ FSDP (Full Shard Data Parallel) with ZeRO3-style. All models are trained using 32 × A100s. In the case of training the setting with...

    2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.