Pith. sign in

REVIEW 6 cited by

A Decade's Battle on Dataset Bias: Are We There Yet?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.08632 v2 pith:ASEKCBTD submitted 2024-03-13 cs.CV cs.LG

Zhuang Liu , Kaiming He This is my paper · ORCID
classification cs.CVcs.LG
keywords datasetaccuracybiasclassificationdatasetsdecadeneuralachieve
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We revisit the "dataset classification" experiment suggested by Torralba & Efros (2011) a decade ago, in the new era with large-scale, diverse, and hopefully less biased datasets as well as more capable neural network architectures. Surprisingly, we observe that modern neural networks can achieve excellent accuracy in classifying which dataset an image is from: e.g., we report 84.7% accuracy on held-out validation data for the three-way classification problem consisting of the YFCC, CC, and DataComp datasets. Our further experiments show that such a dataset classifier could learn semantic features that are generalizable and transferable, which cannot be explained by memorization. We hope our discovery will inspire the community to rethink issues involving dataset bias.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dataset-Origin Signatures and Shortcut Learning in Screening Mammography AI: A Cross-Dataset Case Study

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Pooling abnormal-enriched public mammography datasets into a screening cohort's training set lowers screening AUC, and dataset-of-origin is almost perfectly predictable despite identical preprocessing.

  2. Prompt the Unseen: Evaluating Visual-Language Alignment Beyond Supervision

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A new benchmark shows VLM projection layers retain most of their alignment accuracy on object classes never seen during alignment training, with mechanistic evidence pointing to FFN key-value memory.

  3. Understanding Dataset Bias in Medical Imaging: A Case Study on Chest X-rays

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Classifiers can identify the origin of chest X-rays across NIH, CheXpert, MIMIC-CXR, and PadChest with F1 scores up to about 99%, and the bias appears driven mainly by pixel intensity and texture.

  4. Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models

    cs.MM 2025-01 conditional novelty 6.0 of 10

    A tuning-free pipeline using frozen LLaMA-3, MiniGPT-v2, and Video-ChatGPT reports state-of-the-art zero-shot video moment retrieval on three benchmarks.

  5. Low-Biased General Annotated Dataset Generation

    cs.CV 2024-12 conditional novelty 6.0 of 10

    A CLIP-aligned, diffusion-generated 'low-biased' ImageNet improves backbone pre-training transfer accuracy and reduces shape, context, and background biases relative to real ImageNet and earlier synthetic datasets.

  6. Enhance Vision-Language Alignment with Noise

    cs.CV 2024-12 reject novelty 5.0 of 10

    Injected learned Gaussian noise, tuned with a variational loss, improves CLIP's few-shot classification accuracy slightly over CoOp and CLIP-Adapter.

Pith tools