REVIEW 3 major objections 6 minor 105 references
SNAP: A Benchmark for Testing the Effects of Capture Conditions on Fundamental Vision Tasks
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Camera settings and lighting, not just scene content, drive what vision models see: all 52 tested models lose accuracy on under- and over-exposed images, and nearly identical photos taken with slightly different settings flip their answers.
desk verdict Solid controlled benchmark and exposure-level findings, but the 'sensitivity to minute settings' claim is undercut by a metric that reduces to below-chance accuracy. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
SNAP is the load-bearing object: a dataset of 37,558 images of 10 everyday object categories photographed with a Canon DSLR at one-stop intervals across shutter speeds 1/4000 to 30 seconds, ISO 100 to 6400, and F-numbers 5.6 to 22, under 1000 lux and 10 lux illumination, with lighting, white balance, and focal length fixed. Two derived quantities carry the analysis. The EV offset re-indexes each image by exposure relative to the camera's auto setting, using $EV = \log_2(F^2/t) - \log_2(\mathrm{ISO}/100)$ per illumination condition, so that 0 means well-exposed and each step means doubled or halved light; this turns exposure equivalence (many parameter triplets producing the same light) into a single axis. The parameter sensitivity metric then measures, within sets of same-scene, same-EV-offset images, the percentage of sets whose coefficient of variation exceeds 1, quantifying instability from nearly imperceptible settings changes.
What would settle it
Recover capture metadata for images currently missing Exif tags in one analyzed dataset, for example by re-fetching a random sample of OpenImages' Flickr sources and extracting F-number, ISO, and shutter speed, and compare the EV-offset distribution with the tagged 0.1% subset. If the untagged images turn out to be dominated by edited, synthetic, or camera-unknown images with different exposure statistics, the claim that computer vision datasets are significantly biased by capture conditions would hold only for camera-origin photos. A second test: digitally re-expose well-exposed SNAP images to the same EV offsets; if the accuracy curves do not reproduce, the effect belongs to the physical capture pipeline (sensor noise, tone mapping, JPEG compression) rather than to pixel brightness alone.
Extended reading notes
Core claim
The central discovery, stated on the paper's own terms, is that capture conditions constitute a measurable, systematic source of error that current benchmarks ignore. Models are susceptible to image perturbations caused by slight variations of camera parameters: even within a fixed exposure level, sets of near-identical images yield inconsistent predictions, with parameter sensitivity exceeding 20% for some classifiers. Exposure level itself acts as a global performance curve: peak top-1 accuracy lies between EV offset $-2$ and $+2$, with under-exposure hurting classifiers more than over-exposure, while object detection fails hardest on over-exposed images and is dominated by misclassification errors inherited from ImageNet-pretrained backbones. The paper further claims that common vision datasets are significantly biased: across 13 datasets with over a billion images combined, F-number, ISO, and shutter-speed distributions are long-tailed with peaks at a few one-stop values, most photos were taken on auto, and capture dates cluster in the past, and this bias propagates into downstream tasks. Finally, on the VQA task, humans and machines separate in a specific way: four VLMs match or beat average human accuracy, yet no model reaches the 89% human peak on well-exposed images, while humans collapse on heavily under-exposed images that models still partially read.
Load-bearing premise
The whole dataset-bias argument rests on the small fraction of images that carry Exif tags for shutter speed, F-number, and ISO, only 3.3% of the 1.3 billion images overall and as little as 0.1% for OpenImages, being representative of what those datasets actually contain.
Editorial extensions
If this is right
- Model accuracy reported on standard benchmarks is partly a statement about the camera settings in the training data, so capture-condition coverage should be reported alongside accuracy.
- Exposure-robustness curves become a reproducible diagnostic: every model's SNAP curve is unimodal around EV offset 0 with asymmetric tails, so new models can be compared on that axis without re-running the full benchmark.
- Object-detection failures under extreme exposure are mostly misclassifications inherited from ImageNet-pretrained backbones, meaning fixes to the backbone, not the detection head, are the lever.
- Scaling data and parameters helps but does not cure the problem: the largest CLIP-pretrained models still waver on near-identical images and none matches the human peak on well-exposed VQA images.
Reading between the lines
- The natural experiment the paper does not run is a treatment group: fine-tune models on SNAP-style capture-balanced data and remeasure parameter sensitivity; the paper's own analysis of LAION's exposure diversity predicts sensitivity would shrink, which would turn the diagnostic into a training recipe.
- If digitally re-exposing well-exposed SNAP images to the same EV offsets reproduces the accuracy curves, exposure robustness can be built cheaply as a data-augmentation layer; if it does not, camera-in-the-loop collection like SNAP is the only route.
- The human-machine asymmetry on over-exposed images suggests models' high-level features are more sensitive to clipped highlights than human perception is, which points toward explicit tone-mapping or exposure-normalization modules as the targeted fix.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces SNAP, a new image dataset collected with a DSLR under controlled lighting, densely sampling shutter speed, ISO, and F-number at 1-stop intervals across 100 scenes from 10 object categories. The authors evaluate 23 image classifiers, 16 object detectors, and 13 vision-language models, and run a 43-subject human VQA study with brief masked presentation. They also analyze Exif metadata from 13 large vision datasets and report systematic capture bias. The main reported findings are that model accuracy drops on under- and over-exposed images relative to well-exposed images, that models are sensitive to "minute variations" of camera parameters at matched exposure, and that current training datasets under-represent capture diversity.
Significance. If the exposure-level findings hold, SNAP is a valuable benchmark: it provides real-camera parameter coverage at a density not available in existing robustness datasets, releases code and data, and evaluates a broad model zoo across three tasks with a human baseline. The consistent peak at well-exposed EV offsets across tasks is a useful, falsifiable observation, and the manual collection effort is substantial. However, the matched-exposure sensitivity claim rests on a statistically invalid metric, and the dataset-bias conclusion is extrapolated from very small Exif subsets; both need re-analysis before the paper's central claims can be accepted.
major comments (3)
- [3.3 (Parameter sensitivity)] For the binary per-image metrics used in image classification and VQA hard accuracy, the coefficient of variation is a deterministic function of the mean: for a set with mean accuracy p, CV = sqrt((1-p)/p), so the condition CV > 1 is exactly p < 0.5. Therefore PS is the percentage of (scene, EV-offset) sets in which the model's accuracy is below 50%, not a measure of how much predictions fluctuate across camera settings. This directly affects the matched-exposure claims in Section 4.1 ("All models are susceptible to image perturbations caused by slight variations of camera parameters"), Section 4.3 ("All VLMs are sensitive to camera parameter variations"), and the abstract's "minute variations" statement. A set in which the model is wrong three times out of five contributes to PS, while a set whose per-image outcomes alternate substantially around a mean of 0.55 does not. Please replace PS with a metric that actually compares per-image outcomes across different parameter settings within a scene (for example, the variance of per-image accuracy across settings, or a permutation test against within-setting variability), or remove the matched-exposure conclusion.
- [3.1 / Table 1] The capture-bias analysis is performed on Exif-tagged subsets whose coverage varies from 61.6% (YFCC15M) down to 0.1% (OpenImages, Wukong) and 1.4% (LAION400M). The paper states in Section 3.1 that "Overall, we found significant biases across all datasets" and uses the resulting distributions in Figs. 1 and A.4 to support the abstract's claim that "computer vision datasets are significantly biased." No evidence is provided that images with all three Exif tags are representative of the full dataset; for Common Crawl-derived corpora, images without metadata may be screenshots, stripped web images, or non-camera files. Please either supply a representativeness check (for example, comparing a labeled sample of images without Exif on content or source statistics) or explicitly restrict the dataset-bias conclusions to the Exif-tagged subpopulations.
- [3.4 / 4.3 (Human experiment)] The human baseline used 200 ms image presentation followed by a mask, while the VLMs were given the full image with no time limit and could generate lengthy answers. The paper uses this baseline to state that "4 models reach or surpass the average accuracy of human subjects" (Section 4.3) and that humans peak at 89% on well-exposed images. The time-limited protocol may substantially lower human accuracy, especially on under-exposed images, so the model-vs-human comparisons are not apples-to-apples. Since the VLMs are not feedforward and are not subject to the same presentation constraint, the claim that models "reach or surpass" humans should be rephrased as "surpass a brief-presentation human baseline," or additional human data without the time limit should be collected. The conclusion that models do not reach human accuracy on well-exposed images is conservative under this mismatch, but the "surpass human" statements are not.
minor comments (6)
- [3.2.1] The text says "We captured 10 scenes with 2–5 objects from each object category," but Appendix B states there are 100 unique scenes (10 per category); please clarify the wording.
- [4.3 / Appendix F.4] Question numbering is inconsistent: Section 3.2.2 defines Q3 as multiple-choice categorization and Q4 as multiple-choice counting, but Section 4.3 refers to "MC categorization (Q5)" and Fig. F.4(d) is labeled "Q4: Categorization, multiple-choice." Please renumber or re-label consistently.
- [4.2] The text says "SP is highest for the oLRP FN component" and should read "PS is highest."
- [4.1 / Fig. 4] The sentence "This is evident from high parameter sensitivity (PS) w.r.t. top-1 accuracy that reaches over 20% some models (Fig. 4)" appears to cite the wrong figure; Fig. 4 shows mean top-1 accuracy versus EV offset, while the PS values are the blue circles in Fig. 3.
- [References] Reference [6] for OpenImages v7 points to a semantic segmentation paper (Benenson and Ferrari); please cite the actual OpenImages dataset.
- [3 (Methodology)] The opening line "This session discusses" should be "This section discusses."
Circularity Check
The 'parameter sensitivity' metric (PS) reduces, for binary metrics, to the fraction of sets with below-50% accuracy, so the central claim of sensitivity to minute camera variations is an artifact of the metric definition; the paper's other empirical findings remain self-contained.
-
self definitional
[Section 3.3, 'Parameter sensitivity (PS)' definition; applied in Section 4.1 to claim 'All models are susceptible...']
"Parameter sensitivity (PS) measures sensitivity of the metrics to camera parameters... to measure fluctuations in models’ results within each set, we compute coefficient of variation (CV), defined as the ratio of the std to mean (CV = σ µ) of the metric; ... we compute PS as the percentage of sets with CV > 1. ... All models are susceptible to image perturbations caused by slight variations of camera parameters. This is evident from high parameter sensitivity (PS)... Even the top-5 performing models inconsistently classify nearly 10% of the scenes that look essentially the same."
For top-1 classification accuracy and VQA hard accuracy, each image contributes a binary 0/1. For a set with mean p, σ = sqrt(p(1−p)), so CV = sqrt((1−p)/p). Therefore CV > 1 if and only if p < 0.5. By construction, PS is the percentage of same-scene/same-EV-offset sets whose mean accuracy is below 50%, not the percentage of sets whose results fluctuate across camera parameter combinations. A set with p=0.1 (mostly wrong) counts as 'sensitive', while a set with p=0.8 and genuinely mixed success does not. Hence the conclusion that models are susceptible to 'minute variations of camera settings' and 'inconsistently classify' scenes reduces, through the metric definition, to below-50% mean accuracy; the threshold of 1 has no statistical justification.
full rationale
The paper is an empirical benchmark rather than a derivation. The capture-bias analysis (Section 3.1) is a direct measurement of Exif metadata using the standard EV formula; the SNAP dataset is collected and annotated independently; the model evaluations and the human study are external measurements. No fitted parameter is renamed as a prediction, and the few self-citations (e.g., [2,86,93]) are background related work, not load-bearing. The one significant definitional circularity is the PS metric: because top-1 and hard accuracy are binary, CV collapses to a deterministic function of the mean, so the CV>1 threshold selects exactly the sets with accuracy below 50%. The paper's headline claim about sensitivity to 'minute variations of camera settings' rests on this PS value and therefore reduces by construction to a statement about low mean accuracy, not about variability across camera parameters. The major-exposure results (accuracy vs EV offset in Figs. 4, 6, 9) are independent empirical evidence and are not affected by this issue, which is why the overall circularity is partial rather than total.
Assumptions & free parameters
free parameters (1)
- Parameter sensitivity CV threshold =
1.0
assumptions (3)
- domain assumption The subset of images with complete Exif metadata is representative of each analyzed dataset's capture-condition distribution.
- standard math The standard exposure value formula EV = log2(F^2/t) - log2(ISO/100) plus the auto-mode reference correctly identifies well-exposed images independent of illumination.
- domain assumption Images with the same EV offset and lighting 'look essentially the same' to models, so accuracy variation within such bins is attributable to minute camera-setting differences rather than visible scene differences.
Cite this review
Pith. "Pith review of SNAP: A Benchmark for Testing the Effects of Capture Conditions on Fundamental Vision Tasks." pith.science (2026). https://pith.science/paper/O35ITQVQ
@misc{pith2026250515628,
author = {Pith},
title = {Pith review of: SNAP: A Benchmark for Testing the Effects of Capture Conditions on Fundamental Vision Tasks},
year = {2026},
howpublished = {\url{https://pith.science/paper/O35ITQVQ}},
note = {Machine review of arXiv:2505.15628}
}
abstract
Generalization of deep-learning-based (DL) computer vision algorithms to various image perturbations is hard to establish and remains an active area of research. The majority of past analyses focused on the images already captured, whereas effects of the image formation pipeline and environment are less studied. In this paper, we address this issue by analyzing the impact of capture conditions, such as camera parameters and lighting, on DL model performance on 3 vision tasks -- image classification, object detection, and visual question answering (VQA). To this end, we assess capture bias in common vision datasets and create a new benchmark, SNAP (for $\textbf{S}$hutter speed, ISO se$\textbf{N}$sitivity, and $\textbf{AP}$erture), consisting of images of objects taken under controlled lighting conditions and with densely sampled camera settings. We then evaluate a large number of DL vision models and show the effects of capture conditions on each selected vision task. Lastly, we conduct an experiment to establish a human baseline for the VQA task. Our results show that computer vision datasets are significantly biased, the models trained on this data do not reach human accuracy even on the well-exposed images, and are susceptible to both major exposure changes and minute variations of camera settings. Code and data can be found at https://github.com/ykotseruba/SNAP
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
http://www.gphoto.com/
gPhoto2. http://www.gphoto.com/
-
[2]
On sensor bias in experimental methods for comparing interest-point, saliency, and recognition algorithms
Alexander Andreopoulos and John K Tsotsos. On sensor bias in experimental methods for comparing interest-point, saliency, and recognition algorithms. TPAMI, 34(1):110–126, 2011
2011
-
[3]
Aharon Azulay and Yair Weiss. Why do deep convolutional networks generalize so poorly to small image transformations? arXiv:1805.12177, 2018
arXiv 2018
-
[4]
Unexplored faces of robustness and out-of-distribution: Covariate shifts in environment and sensor domains
Eunsu Baek, Keondo Park, Jiyoon Kim, and Hyung-Sin Kim. Unexplored faces of robustness and out-of-distribution: Covariate shifts in environment and sensor domains. In CVPR, 2024
2024
-
[5]
ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. In NeurIPS, 2019
2019
-
[6]
From colouring-in to pointillism: revisiting semantic segmentation supervision
Rodrigo Benenson and Vittorio Ferrari. From colouring-in to pointillism: revisiting semantic segmentation supervision. arXiv:2210.14142, 2022
arXiv 2022
-
[7]
Are we done with ImageNet? arXiv:2006.07159, 2020
Lucas Beyer, Olivier J Hénaff, Alexander Kolesnikov, Xiaohua Zhai, and Aäron van den Oord. Are we done with ImageNet? arXiv:2006.07159, 2020
arXiv 2006
-
[8]
PaliGemma: A versatile 3B VLM for transfer
Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. PaliGemma: A versatile 3B VLM for transfer. arXiv:2407.07726, 2024
arXiv 2024
Show all 105 references
-
[9]
A digital image processing pipeline for modelling of realistic noise in synthetic images
Oleksandra Bielova, Ronny Hansch, Andreas Ley, and Olaf Hellwich. A digital image processing pipeline for modelling of realistic noise in synthetic images. In CVPRW, 2019
2019
-
[10]
YOLOv4: Optimal speed and accuracy of object detection
Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. YOLOv4: Optimal speed and accuracy of object detection. arXiv:2004.10934, 2020
2004 arXiv
-
[11]
COYO-700M: Image-text pair dataset
Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim. COYO-700M: Image-text pair dataset. https://github.com/kakaobrain/coyo-dataset, 2022
2022
-
[12]
End-to-end object detection with Transformers
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with Transformers. In ECCV, 2020
2020
-
[13]
Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts
Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In CVPR, 2021
2021
-
[14]
On the generalization mystery in deep learning
Satrajit Chatterjee and Piotr Zielinski. On the generalization mystery in deep learning. arXiv:2203.10036, 2022
2022 arXiv
-
[15]
GazeGAN: A generative adversarial saliency model based on invariance analysis of human gaze during scene free viewing
Zhaohui Che, Ali Borji, Guangtao Zhai, Xiongkuo Min, Guodong Guo, and Patrick Le Callet. GazeGAN: A generative adversarial saliency model based on invariance analysis of human gaze during scene free viewing. arXiv:1905.06803, 2019
1905 arXiv
-
[16]
Benchmarking robustness of adaptation methods on pre-trained vision-language models
Shuo Chen, Jindong Gu, Zhen Han, Yunpu Ma, Philip Torr, and V olker Tresp. Benchmarking robustness of adaptation methods on pre-trained vision-language models. In NeurIPS, 2023
2023
-
[17]
PaLI: A jointly-scaled multilingual language-image model
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. PaLI: A jointly-scaled multilingual language-image model. In ICLR, 2023
2023
-
[18]
Reproducible scaling laws for contrastive language-image learning
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In CVPR, 2023
2023
-
[19]
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, 2009
2009
-
[20]
Understanding how image quality affects deep neural networks
Samuel Dodge and Lina Karam. Understanding how image quality affects deep neural networks. In QoMEX, 2016
2016
-
[21]
A study and comparison of human and deep learning recognition performance under visual distortions
Samuel Dodge and Lina Karam. A study and comparison of human and deep learning recognition performance under visual distortions. In ICCCN, 2017. 10
2017
-
[22]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021
2021
-
[23]
In search of robust measures of generalization
Gintare Karolina Dziugaite, Alexandre Drouin, Brady Neal, Nitarshan Rajkumar, Ethan Caballero, Linbo Wang, Ioannis Mitliagkas, and Daniel M Roy. In search of robust measures of generalization. In NeurIPS, 2020
2020
-
[24]
The Pascal Visual Object Classes (VOC) Challenge
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The Pascal Visual Object Classes (VOC) Challenge. IJCV, 88:303–338, 2010
2010
-
[25]
Data filtering networks
Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander T Toshev, and Vaishaal Shankar. Data filtering networks. In ICLR, 2024
2024
-
[26]
Yolov3: An incremental improvement
Ali Farhadi and Joseph Redmon. Yolov3: An incremental improvement. arXiv:1804.02767, 2018
2018 arXiv
-
[27]
Shortcut learning in deep neural networks
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020
2020
-
[28]
Generalisation in humans and deep neural networks
Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. In NeurIPS, 2018
2018
-
[29]
Fast R-CNN
Ross Girshick. Fast R-CNN. In ICCV, 2015
2015
-
[30]
Truth or backpropaganda? an empirical investigation of deep learning theory
Micah Goldblum, Jonas Geiping, Avi Schwarzschild, Michael Moeller, and Tom Goldstein. Truth or backpropaganda? an empirical investigation of deep learning theory. In ICLR, 2020
2020
-
[31]
Strengths and weaknesses of deep learning models for face recognition against image degradations
Klemen Grm, Vitomir Štruc, Anais Artiges, Matthieu Caron, and Hazım K Ekenel. Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biometrics, 7(1):81–89, 2017
2017
-
[32]
Wukong: A 100 million large-scale Chinese cross-modal pre-training benchmark
Jiaxi Gu, Xiaojun Meng, Guansong Lu, Lu Hou, Niu Minzhe, Xiaodan Liang, Lewei Yao, Runhui Huang, Wei Zhang, Xin Jiang, et al. Wukong: A 100 million large-scale Chinese cross-modal pre-training benchmark. In NeurIPS, 2022
2022
-
[33]
ExifTool
Phil Harvey. ExifTool. https://exiftool.org/, 2016
2016
-
[34]
Recent advances in deep learning theory
Fengxiang He and Dacheng Tao. Recent advances in deep learning theory. arXiv:2012.10931, 2020
2012 arXiv
-
[35]
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022
2022
-
[36]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016
2016
-
[37]
Benchmarking neural network robustness to common corruptions and perturbations
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In ICLR, 2019
2019
-
[38]
Natural adversarial examples
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR, 2021
2021
-
[39]
Scene recognition with CNNs: Objects, scales and dataset bias
Luis Herranz, Shuqiang Jiang, and Xiangyang Li. Scene recognition with CNNs: Objects, scales and dataset bias. In CVPR, 2016
2016
-
[40]
A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1–55, 2025
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Info...
2025
-
[41]
Visual Genome: Connecting language and vision using crowdsourced dense image annotations
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual Genome: Connecting language and vision using crowdsourced dense image annotations. IJCV, 123:32–73, 2017
2017
-
[42]
Robustness and repeatability of saliency models subjected to visual degradations
Olivier Le Meur. Robustness and repeatability of saliency models subjected to visual degradations. In ICIP, 2011
2011
-
[43]
Can multiple-choice questions really be useful in detecting the abilities of llms? In LREC-COLING, 2024
Wangyue Li, Liangzhi Li, Tong Xiang, Xiao Liu, Wei Deng, and Noa Garcia. Can multiple-choice questions really be useful in detecting the abilities of llms? In LREC-COLING, 2024. 11
2024
-
[44]
Exploring plain vision transformer backbones for object detection
Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In ECCV, 2022
2022
-
[45]
VILA: On pre-training for visual language models
Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. VILA: On pre-training for visual language models. In CVPR, 2024
2024
-
[46]
Microsoft COCO: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014
2014
-
[47]
Improved baselines with visual instruction tuning
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, 2024
2024
-
[48]
Evaluation of LBP and deep texture descriptors with a new robustness benchmark
Li Liu, Paul Fieguth, Xiaogang Wang, Matti Pietikäinen, and Dewen Hu. Evaluation of LBP and deep texture descriptors with a new robustness benchmark. In ECCV, 2016
2016
-
[49]
Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection. In ECCV, 2024
2024
-
[50]
SSD: Single shot multibox detector
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. SSD: Single shot multibox detector. In ECCV, 2016
2016
-
[51]
Swin Transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021
2021
-
[52]
A decade’s battle on dataset bias: Are we there yet? In ICLR, 2025
Zhuang Liu and Kaiming He. A decade’s battle on dataset bias: Are we there yet? In ICLR, 2025
2025
-
[53]
A convnet for the 2020s
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, 2022
2022
-
[54]
DeepSeek-VL: Towards real-world vision-language understand- ing
Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. DeepSeek-VL: Towards real-world vision-language understand- ing. arXiv:2403.05525, 2024
2024 arXiv
-
[55]
Benchmarking robustness in object detection: Autonomous driving when winter is coming
Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking robustness in object detection: Autonomous driving when winter is coming. arXiv:1907.07484, 2019
1907 arXiv
-
[56]
Simple open-vocabulary object detection
Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, et al. Simple open-vocabulary object detection. In ECCV, 2022
2022
-
[57]
MNIST-C: A robustness benchmark for computer vision
Norman Mu and Justin Gilmer. MNIST-C: A robustness benchmark for computer vision. arXiv:1906.02337, 2019
1906 arXiv
-
[58]
Uniform convergence may be unable to explain generalization in deep learning
Vaishnavh Nagarajan and J Zico Kolter. Uniform convergence may be unable to explain generalization in deep learning. In NeurIPS, 2019
2019
-
[59]
Evaluation of visual saliency analysis algorithms in noisy images
Yuzhen Niu, Lingling Ke, and Wenzhong Guo. Evaluation of visual saliency analysis algorithms in noisy images. Machine Vision and Applications, 27(6):915–927, 2016
2016
-
[60]
Localization recall precision (LRP): A new performance metric for object detection
Kemal Oksuz, Baris Can Cam, Emre Akbas, and Sinan Kalkan. Localization recall precision (LRP): A new performance metric for object detection. In ECCV, 2018
2018
-
[61]
One metric to measure them all: Localisation recall precision (LRP) for evaluating visual detection tasks
Kemal Oksuz, Baris Can Cam, Sinan Kalkan, and Emre Akbas. One metric to measure them all: Localisation recall precision (LRP) for evaluating visual detection tasks. TPAMI, 44(12):9446–9463, 2021
2021
-
[62]
Im2Text: Describing images using 1 million captioned photographs
Vicente Ordonez, Girish Kulkarni, and Tamara Berg. Im2Text: Describing images using 1 million captioned photographs. In NeurIPS, 2011
2011
-
[63]
NMS strikes back
Jeffrey Ouyang-Zhang, Jang Hyun Cho, Xingyi Zhou, and Philipp Krähenbühl. NMS strikes back. arXiv:2212.06137, 2022
2022 arXiv
-
[64]
PsychoPy2: Experiments in behavior made easy
Jonathan Peirce, Jeremy R Gray, Sol Simpson, Michael MacAskill, Richard Höchenberger, Hiroyuki Sogo, Erik Kastman, and Jonas Kristoffer Lindeløv. PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51:195–203, 2019. 12
2019
-
[65]
Dataset issues in object recognition
Jean Ponce, Tamara L Berg, Mark Everingham, David A Forsyth, Martial Hebert, Svetlana Lazebnik, Marcin Marszalek, Cordelia Schmid, Bryan C Russell, Antonio Torralba, C Williams, and A Zisserman. Dataset issues in object recognition. In Toward Category-level Object Recognition,...
2006
-
[66]
Basics Photography 07: Exposure
David Präkel. Basics Photography 07: Exposure . A V A Publishing, 2009
2009
-
[67]
Dataset growth
Ziheng Qin, Zhaopan Xu, Yukun Zhou, Zangwei Zheng, Zebang Cheng, Hao Tang, Lei Shang, Baigui Sun, Xiaojiang Peng, Radu Timofte, et al. Dataset growth. In ECCV, 2024
2024
-
[68]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021
2021
-
[69]
Camera exposure determination
Sidney F Ray et al. Camera exposure determination. In Ralph Jacobson, Sidney Ray, Geoffrey G. Attridge, and Norman Axford, editors, The Manual of Photography, pages 310–318. Routledge, 2000
2000
-
[70]
ImageNet-21K pretraining for the masses
Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor. ImageNet-21K pretraining for the masses. arXiv:2104.10972, 2021
2021 arXiv
-
[71]
Fine-grained recognition in the noisy wild: Sensitivity analysis of convolutional neural networks approaches
Erik Rodner, Marcel Simon, Robert B Fisher, and Joachim Denzler. Fine-grained recognition in the noisy wild: Sensitivity analysis of convolutional neural networks approaches. arXiv:1610.06756, 2016
2016 arXiv
-
[72]
Effects of degradations on deep neural network architectures
Prasun Roy, Subhankar Ghosh, Saumik Bhattacharya, and Umapada Pal. Effects of degradations on deep neural network architectures. arXiv:1807.10108, 2018
2018 arXiv
-
[73]
LAION-400M: Open dataset of CLIP-filtered 400 million image-text pairs
Christoph Schuhmann, Robert Kaczmarczyk, Aran Komatsuzaki, Aarush Katta, Richard Vencu, Romain Beaumont, Jenia Jitsev, Theo Coombes, and Clayton Mullis. LAION-400M: Open dataset of CLIP-filtered 400 million image-text pairs. In NeurIPSW, 2021
2021
-
[74]
Objects365: A large-scale, high-quality dataset for object detection
Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In CVPR, 2019
2019
-
[75]
Conceptual Captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual Captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACL, 2018
2018
-
[76]
Assessing visually-continuous corruption robustness of neural networks relative to human performance
Huakun Shen, Boyue Caroline Hu, Krzysztof Czarnecki, Lina Marsso, and Marsha Chechik. Assessing visually-continuous corruption robustness of neural networks relative to human performance. In WACV, 2025
2025
-
[77]
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015
2015
-
[78]
WIT: Wikipedia- based image text dataset for multimodal multilingual machine learning
Krishna Srinivasan, Karthik Raman, Jiecao Chen, Michael Bendersky, and Marc Najork. WIT: Wikipedia- based image text dataset for multimodal multilingual machine learning. In SIGIR, 2021
2021
-
[79]
Convnets and ImageNet beyond accuracy: Understanding mistakes and uncovering biases
Pierre Stock and Moustapha Cisse. Convnets and ImageNet beyond accuracy: Understanding mistakes and uncovering biases. In ECCV, 2018
2018
-
[80]
A survey on statistical theory of deep learning: Approximation, training dynamics, and generative models
Namjoon Suh and Guang Cheng. A survey on statistical theory of deep learning: Approximation, training dynamics, and generative models. Annual Review of Statistics and Its Application , 12, 2024
2024
-
[81]
YFCC100M: The new data in multimedia research
Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. YFCC100M: The new data in multimedia research. Communications of the ACM, 59(2):64–73, 2016
2016
-
[82]
A deeper look at dataset bias
Tatiana Tommasi, Novi Patricia, Barbara Caputo, and Tinne Tuytelaars. A deeper look at dataset bias. In Domain Adaptation in Computer Vision Applications , pages 37–55. 2017
2017
-
[83]
Unbiased look at dataset bias
Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In CVPR, 2011
2011
-
[84]
A robustness analysis of deep q networks
Adam W Tow, Sareh Shirazi, Jürgen Leitner, Niko Sünderhauf, Michael Milford, and Ben Upcroft. A robustness analysis of deep q networks. Australasian Conference on Robotics and Automation, 2016
2016
-
[85]
From ImageNet to image classification: Contextualizing progress on benchmarks
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. From ImageNet to image classification: Contextualizing progress on benchmarks. In ICML, 2020
2020
-
[86]
Why does data-driven beat theory-driven computer vision? In ICCVW, 2019
John Tsotsos, Iuliia Kotseruba, Alexander Andreopoulos, and Yulong Wu. Why does data-driven beat theory-driven computer vision? In ICCVW, 2019. 13
2019
-
[87]
Probing the effect of selection bias on generalization: A thought experiment
John K Tsotsos and Jun Luo. Probing the effect of selection bias on generalization: A thought experiment. arXiv:2105.09934, 2021
2021 arXiv
-
[88]
CSPNet: A new backbone that can enhance learning capability of CNN
Chien-Yao Wang, Hong-Yuan Mark Liao, Yueh-Hua Wu, Ping-Yang Chen, Jun-Wei Hsieh, and I-Hau Yeh. CSPNet: A new backbone that can enhance learning capability of CNN. In CVPRW, 2020
2020
-
[89]
QwenV2-VL: Enhancing vision-language model’s perception of the world at any resolution
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. QwenV2-VL: Enhancing vision-language model’s perception of the world at any resolution. arXiv:2409.12191, 2024
2024 arXiv
-
[90]
Enhancing the reasoning ability of multimodal large language models via mixed preference optimization
Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, et al. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv:2411.10442, 2024
2024 arXiv
-
[91]
InternImage: Exploring large-scale vision foundation models with deformable convolutions
Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, et al. InternImage: Exploring large-scale vision foundation models with deformable convolutions. In CVPR, 2023
2023
-
[92]
Physical adversarial attack meets computer vision: A decade survey
Hui Wei, Hao Tang, Xuemei Jia, Zhixiang Wang, Hanxun Yu, Zhubo Li, Shin’ichi Satoh, Luc Van Gool, and Zheng Wang. Physical adversarial attack meets computer vision: A decade survey. TPAMI, 46(12):9797–9817, 2024
2024
-
[93]
Active control of camera parameters for object detection algorithms
Yulong Wu and John Tsotsos. Active control of camera parameters for object detection algorithms. arXiv:1705.05685, 2017
2017 arXiv
-
[94]
Statistic analysis of millions of digital photos
Dietmar Wueller and Reiner Fageth. Statistic analysis of millions of digital photos. InDigital Photography IV, volume 6817, page 68170L. International Society for Optics and Photonics, 2008
2008
-
[95]
Statistic analysis of millions of digital photos 2017
Dietmar Wueller and Reiner Fageth. Statistic analysis of millions of digital photos 2017. Electronic Imaging, 2018(5):1–4, 2018
2017
-
[96]
Does robustness on ImageNet transfer to downstream tasks? In CVPR, 2022
Yutaro Yamada and Mayu Otani. Does robustness on ImageNet transfer to downstream tasks? In CVPR, 2022
2022
-
[97]
Yocto-Light-V3 product page
Yoctopuce. Yocto-Light-V3 product page. https://www.yoctopuce.com/EN/products/ usb-environmental-sensors/yocto-light-v3 . Accessed April 5, 2024
2024
-
[98]
Understanding bias in large-scale visual datasets
Boya Zeng, Yida Yin, and Zhuang Liu. Understanding bias in large-scale visual datasets. In NeurIPS, 2024
2024
-
[99]
Sigmoid loss for language image pre-training
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, 2023
2023
-
[100]
Understanding deep learning (still) requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107–115, 2021
2021
-
[101]
DINO: DETR with improved denoising anchor boxes for end-to-end object detection
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. DINO: DETR with improved denoising anchor boxes for end-to-end object detection. In ICLR, 2023
2023
-
[102]
DETRs beat YOLOs on real-time object detection
Yian Zhao, Wenyu Lv, Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingqing Dang, Yi Liu, and Jie Chen. DETRs beat YOLOs on real-time object detection. In CVPR, 2024
2024
-
[103]
Exif as language: Learning cross-modal associations between images and camera metadata
Chenhao Zheng, Ayush Shrivastava, and Andrew Owens. Exif as language: Learning cross-modal associations between images and camera metadata. In CVPR, 2023
2023
-
[104]
Improving the robustness of deep neural networks via stability training
Stephan Zheng, Yang Song, Thomas Leung, and Ian Goodfellow. Improving the robustness of deep neural networks via stability training. In CVPR, 2016
2016
-
[105]
Manual” and ‘Manual exposure
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In CVPR, 2018. 14 A Computer vision dataset properties VOC2007ImageNet SBUCOCO OpenImages v7 YFCC15M CC3M ImageNet21K CC12M WIT LAION400M Wukong CO...
2018
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.