Pith. sign in

REVIEW 5 major objections 6 minor 40 references

Quality over Quantity: Boosting Data Efficiency Through Ensembled Multimodal Data Curation

T0 review · 5 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A CLIP model trained on 3.5M carefully chosen image-text pairs beats one trained on the full 12.8M pool in DataComp's small-scale filtering track.

desk verdict A workmanlike data-curation ensemble that probably does help on DataComp small-scale, but the claimed SOTA margin is within typical training noise and the tuning protocol is under-specified. read the letter →

arxiv 2502.08211 v2 pith:2V6UDYAK submitted 2025-02-12 cs.LG cs.AI

Jinda Xu , Yuhao Song , Daming Wang , Weiwei Zhao , Minghua Chen , Kangliang Chen , Qinya Li This is my paper · ORCID
classification cs.LGcs.AI
keywords datacurationweaksupervisionmultimodallearningCLIPCompdeduplicationensembleimage-textpretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that data curation for web-scale image-text pretraining is best treated as a learning problem rather than a fixed set of heuristics. EcoDatum combines eight curation operators—image blur and geometry checks, language identification and caption-concreteness scoring, global CLIP-style alignment with horizontal and vertical flip variants, and local object-detection alignment—into a weak supervision ensemble in which each operator votes on every image-text pair and a learned model combines the votes into a quality score. On the DataComp small-scale track, a benchmark that fixes the training code and lets competitors choose the data, a CLIP trained on the resulting 3.5 million pairs from a 12.8 million Common Crawl pool averages 0.182 across 38 downstream tasks, above the unfiltered pool (0.132), the basic DataComp filter (0.142), and the previous top filters T-MARS and WS (0.180). If this holds, the paper establishes that smaller, well-selected data can outperform larger raw data for multimodal pretraining, with a large reduction in training compute.

What carries the argument

The load-bearing mechanism is the weak supervision ensemble. Eight operators are converted into labeling functions whose threshold bands produce labels of $1$ (selected), $0$ (filtered), or $-1$ (abstain), as in Eq. (2); the abstain option lets an operator stay silent when its score is ambiguous. A LabelModel—a probabilistic model of labeling-function accuracy and correlation—turns the resulting label matrix into a per-sample quality score. Search-based optimization chooses the labeling functions by maximizing $M=\alpha_1 F1_{\mathrm{tiny}}+\alpha_2 f_{\mathrm{Overlap}}-\alpha_3 f_{\mathrm{Conflict}}+\alpha_4 f_{\mathrm{Coverage}}$ on a tiny labeled dataset, where the F1 term measures agreement with hand-built clean/noisy labels and the other terms reward coverage and penalize conflicts. Quality-guided deduplication runs first: perceptual hashing groups near-duplicates and the CLIP score selects the best-aligned pair in each group. Flip-CLIP (horizontal and vertical flips before CLIP scoring) counters the tendency of CLIP scores to be inflated by text embedded in images.

What would settle it

Run the released EcoDatum pipeline on the DataComp small-scale pool twice, building two independently resampled tiny labeled sets from the same recipe; if the labeling-function combination that maximizes the composite metric on one tiny set scores below 0.180 on the 38 tasks, or below the combination selected on the other tiny set, the claimed transfer from the tiny set to full-scale curation fails.

Watch

Extended reading notes

Core claim

On its own terms, the discovery is that the agreement pattern among diverse curation operators carries enough signal to rank web-crawled image-text pairs by quality. EcoDatum converts each operator's continuous score into a ternary vote (select, filter, or abstain), feeds the votes into a probabilistic LabelModel, and learns weights that reflect how much each operator contributes. A search procedure tunes the vote thresholds by optimizing a composite metric on a tiny labeled set built from clean COCO images, random web images, and artificially swapped pairs. The curated subset of about 3.5 million pairs yields a CLIP model with 0.182 average performance over 38 evaluation datasets, the top published result on the DataComp small-scale filtering track and a 28% improvement over the DataComp basic filter.

Load-bearing premise

The load-bearing premise is that the tiny labeled set—clean COCO images, random web images, and swapped image-text pairs—faithfully represents what 'high quality' means for the full 12.8 million web-crawled pairs, so that optimizing against it improves the 38-task downstream benchmark.

Editorial extensions

If this is right

  • A CLIP trained on EcoDatum's 3.5M subset scores 0.182 on the 38-task benchmark, above the 12.8M unfiltered pool (0.132), the basic DataComp filter (0.142), and the prior best filters T-MARS and WS (0.180).
  • The 3.5M subset is 72% smaller than the 12.8M pool, so the same benchmark result comes with a substantial cut in downstream data-processing and training cost.
  • Global cross-modal alignment operators carry the largest learned weights, and removing the vertical flip variant degrades performance, so the largest gains come from alignment scoring rather than unimodal filters.
  • The search-based LF optimization is load-bearing: random LabelModel weight assignments produced 0.141–0.161 average scores, all below the optimized 0.182.
  • The improvement over no filtering grows with pool size (0.002 at 0.128M, 0.016 at 1.28M, 0.050 at 12.8M), which the paper reads as evidence that the curation gain scales.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the ensemble is operator-agnostic, the same weak-supervision search recipe could be transferred to other data modalities—video-text pairs, audio captions, or language-model pretraining corpora—provided a tiny labeled proxy of clean versus noisy examples can be built; the paper does not test these transfers.
  • Part of the gain likely comes from demoting pairs where text embedded in the image inflates vanilla CLIP similarity, which makes EcoDatum a natural complement to OCR-based filters rather than a replacement for them.
  • The paper's scaling table suggests the method is aimed at large noisy pools; on an already aggressively cleaned pool the operator votes would be more correlated, and one should not expect the same margin over unfiltered data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. EcoDatum is a data-curation pipeline for the DataComp small-scale filtering track. It combines quality-guided perceptual-hash deduplication, a set of unimodal (blur, geometry, language identification, caption concreteness) and multimodal (CLIP/Flip-CLIP global alignment and GroundingDINO local alignment) operators, and a weak-supervision LabelModel that converts operator scores into labels via threshold-based labeling functions. A search procedure over LF combinations is guided by a composite metric M = α1 F1_tiny + α2 fOverlap − α3 fConflict + α4 fCoverage evaluated on a tiny labeled set (COCO-clean, DataComp-noisy, and swapped pairs). The paper reports that the selected subset of 3.5M pairs from 12.8M yields an average score of 0.182 over 38 datasets, claiming 1st place on the DataComp leaderboard and a 28% improvement over the DataComp baseline.

Significance. If the result were statistically robust, EcoDatum would be a useful demonstration that ensembling complementary quality operators via weak supervision can curate web-scale data more efficiently than single-metric filters, and the proposed search-based LF optimization is a step toward automating threshold selection. The evaluation on the standardized DataComp benchmark with fixed training code is a strength, as is the availability of the code link. However, the current evidence does not establish the claimed SOTA: the reported margin over T-MARS and WS is 0.002 with no measure of run-to-run variance, and key hyperparameters (final data volume and composite-metric coefficients) appear to be tuned on the same 38-task benchmark used for the final score. These issues must be resolved before the central claim is credible.

major comments (5)
  1. [Table 1 and 'Result Analysis'] Table 1 reports a single run for EcoDatum (0.182) and for all baselines, with no seeds or error bars. The difference between EcoDatum and T-MARS/WS (both 0.180) is 0.002, which is within the run-to-run variance typically observed for CLIP training at this scale; the claim of state-of-the-art status is therefore not supported by the presented evidence. I request a multi-seed evaluation with means and standard deviations, or an explicit description of the DataComp leaderboard's official single-run protocol and its precision.
  2. [Implementation Details; Appendix A.1; Figure 7] The final data volume (top 40%, 3.5M) is selected by varying the filtered dataset size and comparing downstream performance on the same 38-task benchmark (Figure 7; 'conducted extensive experiments'). This makes the reported 0.182 the maximum of a tuning curve on the evaluation benchmark rather than a held-out estimate. To support the data-efficiency claim, the volume selection should be performed on an independent validation split (e.g., a subset of the 38 tasks) or via a nested procedure, and the final comparison should be reported on tasks not used for the volume selection.
  3. [Search-based Optimization, Eq. (3)] The manuscript is internally inconsistent about what was optimized using the 38-task benchmark. It states that the tiny-labeled dataset 'does not contribute to training the model or optimizing Eq (3) coefficients,' but the next paragraph says α1–α4 are 'determined through a few rounds of experiments,' and the subsequent paragraph describes a systematic search whose results are 'repeatedly evaluating their impact on the average performance across a diverse set of 38 benchmark tasks' (Figure 6). If any of these experiments used the 38-task benchmark, the LF selection and coefficient tuning are not independent of the final evaluation metric; if they did not, the text must be rewritten to state exactly which data were used for each tuning step and to report the chosen α values.
  4. [Figure 6 and 'Result Analysis'] The only evidence that the composite metric is a valid proxy for downstream quality is Figure 6, which plots the composite metric against the average performance on the same 38-task benchmark. This is a post-hoc correlation, not a predictive validation, and it cannot detect overfitting of the LF search to the benchmark. I recommend evaluating the selected LFs on a held-out set of tasks or on an independently labeled web-crawl quality set to establish that F1-tiny on the tiny labeled set transfers to unseen evaluation data.
  5. [Search-based Optimization, tiny-labeled dataset construction] The tiny labeled set is constructed by labeling COCO images as clean, randomly sampled DataComp images as noisy, and artificially swapped pairs as cross-modal noise. This construction may not cover the actual failure modes of web-crawl data (e.g., OCR text in images, non-English captions, near-duplicate images with different captions), and no evidence is given that F1-tiny on this set predicts curation quality on the benchmark. At minimum, report the composition and size of the tiny set and perform a sensitivity analysis over its construction (e.g., different noise ratios), or validate on an externally labeled web-crawl quality dataset.
minor comments (6)
  1. [Modality Operators Ensemble, Eq. (2)] The thresholds b_j and β_j in Eq. (2) and the TopK-based LF construction in Table 6 are not formally connected; please define how b_j and β_j are derived from the operator score distributions.
  2. [Ablation Study, Table 2] The row 'QGD+Ens.(Mul.)' is ambiguous; state explicitly which multimodal operators (global, local, or both) are included in each ablation row.
  3. [Appendix A.1] The repeated text 'uni000...' in Appendix A.1 appears to be a rendering artifact and should be removed or fixed.
  4. [Throughout] The spelling 'Datacomp' is used in several places (e.g., Table 1 caption, Related Work); unify to 'DataComp'.
  5. [Result Analysis] The claim of reducing 'computational overhead by 72%' should specify whether this refers to training compute or total curation compute, and should state how it is derived from the dataset-size reduction.
  6. [Figure 6] The caption says 'Downtasks'; it should read 'downstream tasks'.

Circularity Check

0 steps flagged · score 0.0 of 10

No formal circularity: EcoDatum's reported 0.182 average is an external DataComp benchmark measurement, and the LF search is validated on a tiny-labeled set that is not the evaluation benchmark.

full rationale

The paper's derivation chain is self-contained with respect to the definitional circularity patterns. The central claim is an empirical benchmark score from the external DataComp 38-task suite, not an algebraic consequence of the method's equations. The weak supervision LF search (Algorithm 1) optimizes the composite metric M in Eq. (3), which is constructed from F1 on a tiny-labeled dataset (COCO clean samples, random DataComp noisy samples, and swapped image-text pairs) plus LF overlap, conflict, and coverage statistics; none of these components is defined in terms of the 38-task average. The final 0.182 is therefore not forced by construction from the method's inputs. Two methodological concerns are worth flagging but do not amount to circularity. First, Appendix A.1 and Figure 7 show the final data volume (top 40%, 3.5M) was selected by varying dataset size and comparing downstream performance on the same 38-task benchmark used for the headline claim, so the reported number is a selected maximum of a tuning curve rather than an independent prediction; this is a benchmark-selection risk, not a definitional reduction. Second, the text around Eq. (3) is internally inconsistent: it states the tiny-labeled dataset "does not contribute to training the model or optimizing Eq (3) coefficients," yet immediately says the coefficients "are determined through a few rounds of experiments," leaving unclear whether downstream signal leaked into the composite metric. Again, this is a missing protocol detail and an overfitting risk, not a circular derivation. No load-bearing self-citations, imported uniqueness theorems, or ansatz-smuggling citations were found; the method borrows operators and the LabelModel from external prior work. Accordingly, the paper scores 0 on the circularity scale.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The ledger captures the main tunable quantities and hidden assumptions. No physically invented entities are introduced. The free parameters are the data-volume cutoff, composite metric coefficients, LF thresholds, detection threshold, and dedup threshold; all are fitted or hand-chosen rather than derived. The axioms are domain assumptions about the validity of using pretrained model scores and a synthetic tiny-label set to judge web data quality.

free parameters (5)
  • Selected data-volume threshold = top 40% of quality-scored samples; 3.5M of 8.8M after dedup
    Chosen after 'extensive experiments' and 'reviewed related works' to maximize benchmark performance (Implementation Details; Appendix A.1).
  • Composite metric coefficients alpha_1..alpha_4 = not reported
    Coefficients in Eq. (3) are 'determined through a few rounds of experiments'; no values are given, and they control the LF search objective.
  • LF cutoff thresholds (b_j, beta_j / TopK) = TopK values in Table 6, e.g., Top20-Top80 for global, ICC, local alignment
    Eq. (2) converts operator scores to weak labels using per-operator mean, std, and TopK cutoffs; the optimal combination is selected by Algorithm 1.
  • GroundingDINO confidence threshold t = 0.1
    Used in Eq. (1) to count detected objects; set in Implementation Details to retain more potentially aligned data.
  • Perceptual-hash duplicate threshold = unspecified
    Quality-guided dedup relies on perceptual hashing with an unstated similarity threshold to define duplicate groups (Method: Quality Guided Deduplication).
assumptions (5)
  • domain assumption Operator scores are informative monotone signals of data quality for CLIP pretraining.
    The whole ensembling scheme treats CLIP, H-CLIP, V-CLIP, GroundingDINO, ICC, FastText, and blur/geometry heuristics as noisy labels of 'high-quality'. No external evidence is given that these scores correlate with downstream task usefulness (Method: Modality Operators Ensemble).
  • domain assumption The tiny-labeled dataset (COCO clean, random DataComp noisy, swapped pairs) is an unbiased proxy for real web-crawl quality.
    Search-based optimization and Eq. (3) rely on F1-tiny over this hand-constructed set; if it is not representative, the selected LFs may not transfer (Method: Search-based Optimization; Appendix A.7).
  • domain assumption Vertical and horizontal flipping removes text-overlap bias in CLIP scores without losing useful visual signal.
    Flip-CLIP is motivated by observed overlap rates (40% in LAION, 20% in DataComp), but the paper gives no analysis that flipped scores remain valid quality indicators (Multimodal Curation Operators; Appendix A.4).
  • domain assumption Perceptual hashing can identify semantically redundant image-text pairs for deduplication.
    Quality-guided dedup assumes hash collisions correspond to duplicates and that retaining the highest-CLIP pair preserves diversity (Method: Quality Guided Deduplication).
  • standard math Snorkel LabelModel's generative model assumptions hold for these labeling functions.
    The paper relies on Ratner et al. 2017 and Bach et al. 2019 without verifying the LabelModel's assumptions of conditional independence or sufficient coverage for this particular set of operators (Method: Modality Operators Ensemble).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quality over Quantity: Boosting Data Efficiency Through Ensembled Multimodal Data Curation." pith.science (2026). https://pith.science/paper/2V6UDYAK

@misc{pith2026250208211,
  author       = {Pith},
  title        = {Pith review of: Quality over Quantity: Boosting Data Efficiency Through Ensembled Multimodal Data Curation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2V6UDYAK}},
  note         = {Machine review of arXiv:2502.08211}
}
read the original abstract

In an era overwhelmed by vast amounts of data, the effective curation of web-crawl datasets is essential for optimizing model performance. This paper tackles the challenges associated with the unstructured and heterogeneous nature of such datasets. Traditional heuristic curation methods often inadequately capture complex features, resulting in biases and the exclusion of relevant data. We introduce an advanced, learning-driven approach, Ensemble Curation Of DAta ThroUgh Multimodal Operators (EcoDatum), incorporating a novel quality-guided deduplication method to ensure balanced feature distributions. EcoDatum strategically integrates various unimodal and multimodal data curation operators within a weak supervision ensemble framework, utilizing automated optimization to score each data point effectively. EcoDatum, which significantly improves the data curation quality and efficiency, outperforms existing state-of-the-art (SOTA) techniques, ranked 1st on the DataComp leaderboard, with an average performance score of 0.182 across 38 diverse evaluation datasets. This represents a 28% improvement over the DataComp baseline method, demonstrating its effectiveness in improving dataset curation and model training efficiency.

Figures

Figures reproduced from arXiv: 2502.08211 by the authors.

Figure 1
Figure 1. Web-crawled image-text datasets often vary in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the EcoDatum Framework. EcoDatum utilizes quality-guided deduplication along with an ensemble [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Quality Guided Deduplication retains the samples [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Illustration of multimodal curation operators inte [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Overview of Search-Based LFs Combinations Optimization. This method optimizes LFs to create a more accurate [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Composite Metric Validation with Repeated Ex [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Vary the filtered dataset quantity and compare the [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 9
Figure 9. Figure 9: Hard Bad Cases Mined by V-CLIP Operator Thus, moving away from random deduplication to a more deliberate, quality-guided approach significantly bolsters the effectiveness and reliability of multimodal datasets. A.4.Comparison With Different Global Feature Align￾ment St…
Figure 11
Figure 11. Figure 11: Low alignment quality data detected by the Local [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 10
Figure 10. Figure 10: High alignment quality data detected by the Local [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 13
Figure 13. Figure 13: Unimodal(Visual-based) Operators Inf. Scores’ [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 12
Figure 12. Figure 12: Multimodal Operators Inf. Scores’ Distributions [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 15
Figure 15. Figure 15: Selected “High Quality” Data Samples from EcoDatum. “High Quality” image-text data refers to pairs where the [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: Filtered “Low Quality” Data Samples from EcoDatum. “Low-quality” image-text data typically manifests as blurry [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 20 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Abbas, A.; Tirumala, K.; Simig, D.; Ganguli, S.; and Morcos, A. S. 2023. Semdedup: Data-efficient learning at web-scale through semantic deduplication. arXiv preprint arXiv:2303.09540

    arXiv 2023

  4. [4]

    Alayrac, J.-B.; Donahue, J.; Luc, P.; Miech, A.; Barr, I.; Hasson, Y.; Lenc, K.; Mensch, A.; Millican, K.; Reynolds, M.; et al. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35: 23716--23736

    2022

  5. [5]

    H.; Rodriguez, D.; Liu, Y.; Luo, C.; Shao, H.; Xia, C.; Sen, S.; Ratner, A.; Hancock, B.; Alborzi, H.; et al

    Bach, S. H.; Rodriguez, D.; Liu, Y.; Luo, C.; Shao, H.; Xia, C.; Sen, S.; Ratner, A.; Hancock, B.; Alborzi, H.; et al. 2019. Snorkel drybell: A case study in deploying weak supervision at industrial scale. In Proceedings of the 2019 International Conference on Management of Data, 362--375

    work page 2019

  6. [6]

    u rnberger, A.; and K \

    Beluch, W. H.; Genewein, T.; N \"u rnberger, A.; and K \"o hler, J. M. 2018. The power of ensembles for active learning in image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, 9368--9377

    work page 2018

  7. [7]

    Breiman, L. 1996. Bagging predictors. Machine learning, 24: 123--140

    work page 1996

  8. [8]

    Brown, T. B. 2020. Language models are few-shot learners. arXiv preprint ArXiv:2005.14165

    arXiv 2020

Show all 40 references
  1. [9]

    Chen, D.; Huang, Y.; Ma, Z.; Chen, H.; Pan, X.; Ge, C.; Gao, D.; Xie, Y.; Liu, Z.; Gao, J.; et al. 2024. Data-juicer: A one-stop data processing system for large language models. In Companion of the 2024 International Conference on Management of Data, 120--134

    2024

  2. [10]

    Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597--1607. PMLR

    2020

  3. [11]

    Devlin, J. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  4. [12]

    Farid, H. 2021. An overview of perceptual hashing. Journal of Online Trust and Safety, 1(1)

    2021

  5. [13]

    Freund, Y.; and Schapire, R. E. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1): 119--139

    1997

  6. [14]

    Y.; Ilharco, G.; Fang, A.; Hayase, J.; Smyrnis, G.; Nguyen, T.; Marten, R.; Wortsman, M.; Ghosh, D.; Zhang, J.; Orgad, E.; Entezari, R.; Daras, G.; Pratt, S

    Gadre, S. Y.; Ilharco, G.; Fang, A.; Hayase, J.; Smyrnis, G.; Nguyen, T.; Marten, R.; Wortsman, M.; Ghosh, D.; Zhang, J.; Orgad, E.; Entezari, R.; Daras, G.; Pratt, S. M.; Ramanujan, V.; Bitton, Y.; Marathe, K.; Mussmann, S.; Vencu, R.; Cherti, M.; Krishna, R.; Koh, P. W.; Sau...

    2023

  7. [15]

    Y.; Ilharco, G.; Fang, A.; Hayase, J.; Smyrnis, G.; Nguyen, T.; Marten, R.; Wortsman, M.; Ghosh, D.; Zhang, J.; et al

    Gadre, S. Y.; Ilharco, G.; Fang, A.; Hayase, J.; Smyrnis, G.; Nguyen, T.; Marten, R.; Wortsman, M.; Ghosh, D.; Zhang, J.; et al. 2024. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing Systems, 36

    2024

  8. [16]

    Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; Casas, D. d. L.; Hendricks, L. A.; Welbl, J.; Clark, A.; et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556

    2022 arXiv

  9. [17]

    J.; Adila, D.; and Sala, F

    Huang, T.-H.; Shin, C.; Tay, S. J.; Adila, D.; and Sala, F. 2024. Multimodal Data Curation via Object Detection and Filter Ensembles. arXiv preprint arXiv:2401.12225

    2024 arXiv

  10. [18]

    Jia, C.; Yang, Y.; Xia, Y.; Chen, Y.-T.; Parekh, Z.; Pham, H.; Le, Q.; Sung, Y.-H.; Li, Z.; and Duerig, T. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, 4904--4916. PMLR

    2021

  11. [19]

    Joulin, A.; Grave, E.; Bojanowski, P.; Douze, M.; J \'e gou, H.; and Mikolov, T. 2016. Fasttext. zip: Compressing text classification models. arXiv preprint arXiv:1612.03651

    2016 arXiv

  12. [20]

    Kim, W.; Chun, S.; Kim, T.; Han, D.; and Yun, S. 2024. HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts. arXiv preprint arXiv:2404.17507

    2024 arXiv

  13. [21]

    Li, J.; Li, D.; Xiong, C.; and Hoi, S. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, 12888--12900. PMLR

    2022

  14. [22]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 1...

    2014

  15. [23]

    Liu, S.; Zeng, Z.; Ren, T.; Li, F.; Zhang, H.; Yang, J.; Jiang, Q.; Li, C.; Yang, J.; Su, H.; et al. 2025. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, 38--55. Springer

    2025

  16. [24]

    Liu, S.; Zeng, Z.; Ren, T.; Li, F.; Zhang, H.; Yang, J.; Li, C.; Yang, J.; Su, H.; Zhu, J.; et al. 2023. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499

    2023 arXiv

  17. [25]

    C.; Kolter, J

    Maini, P.; Goyal, S.; Lipton, Z. C.; Kolter, J. Z.; and Raghunathan, A. 2023. T-mars: Improving visual representations by circumventing text feature learning. arXiv preprint arXiv:2307.03132

    2023 arXiv

  18. [26]

    Maini, P.; Yaghini, M.; and Papernot, N. 2021. Dataset inference: Ownership resolution in machine learning. arXiv preprint arXiv:2104.10706

    2021 arXiv

  19. [27]

    M.; Razzak, M

    Mindermann, S.; Brauner, J. M.; Razzak, M. T.; Sharma, M.; Kirsch, A.; Xu, W.; H \"o ltgen, B.; Gomez, A. N.; Morisot, A.; Farquhar, S.; et al. 2022. Prioritized training on points that are learnable, worth learning, and not yet learnt. In International Conference on Machine L...

    2022

  20. [28]

    Y.; Ilharco, G.; Oh, S.; and Schmidt, L

    Nguyen, T.; Gadre, S. Y.; Ilharco, G.; Oh, S.; and Schmidt, L. 2024. Improving multimodal datasets with image captioning. Advances in Neural Information Processing Systems, 36

    2024

  21. [29]

    Radenovic, F.; Dubey, A.; Kadian, A.; Mihaylov, T.; Vandenhende, S.; Patel, Y.; Wen, Y.; Ramanathan, V.; and Mahajan, D. 2023. Filtering, distillation, and hard negatives for vision-language pre-training. In Proceedings of the IEEE/CVF conference on computer vision and pattern...

    2023

  22. [30]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

    2021

  23. [31]

    Rasmus, A.; Berglund, M.; Honkala, M.; Valpola, H.; and Raiko, T. 2015. Semi-supervised learning with ladder networks. Advances in neural information processing systems, 28

    2015

  24. [32]

    H.; Ehrenberg, H.; Fries, J.; Wu, S.; and R \'e , C

    Ratner, A.; Bach, S. H.; Ehrenberg, H.; Fries, J.; Wu, S.; and R \'e , C. 2017. Snorkel: Rapid training data creation with weak supervision. In Proceedings of the VLDB endowment. International conference on very large data bases, volume 11, 269. NIH Public Access

    2017

  25. [33]

    Schuhmann, C.; Beaumont, R.; Vencu, R.; Gordon, C.; Wightman, R.; Cherti, M.; Coombes, T.; Katta, A.; Mullis, C.; Wortsman, M.; et al. 2022. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems,...

    2022

  26. [34]

    Schuhmann, C.; Vencu, R.; Beaumont, R.; Kaczmarczyk, R.; Mullis, C.; Katta, A.; Coombes, T.; Jitsev, J.; and Komatsuzaki, A. 2021. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114

    2021 arXiv

  27. [35]

    Song, H.; Kim, M.; Park, D.; Shin, Y.; and Lee, J.-G. 2022. Learning from noisy labels with deep neural networks: A survey. IEEE transactions on neural networks and learning systems, 34(11): 8135--8153

    2022

  28. [36]

    M.; and Reza, A

    Torkashvand, A.; Jameii, S. M.; and Reza, A. 2023. Deep learning-based collaborative filtering recommender systems: A comprehensive and systematic review. Neural Computing and Applications, 35(35): 24783--24827

    2023

  29. [37]

    Yanuka, M.; Alper, M.; Averbuch-Elor, H.; and Giryes, R. 2024. ICC: Quantifying Image Caption Concreteness for Multimodal Dataset Curation. arXiv preprint arXiv:2403.01306

    2024 arXiv

  30. [38]

    Yokoo, S.; Zhu, P.; Ishikawa, Y.; Tanaka, M.; Kondo, M.; and Kataoka, H. 2023. Leveraging Image-Text Similarity and Caption Modification for the DataComp Challenge: Filtering Track and BYOD Track. arXiv preprint arXiv:2310.14581

    2023 arXiv

  31. [39]

    Yu, H.; Tian, Y.; Kumar, S.; Yang, L.; and Wang, H. 2023. The devil is in the details: A deep dive into the rabbit hole of data filtering. arXiv preprint arXiv:2309.15954

    2023 arXiv

  32. [40]

    Zimek, A.; Schubert, E.; and Kriegel, H.-P. 2012. A survey on unsupervised outlier detection in high-dimensional numerical data. Statistical Analysis and Data Mining: The ASA Data Science Journal, 5(5): 363--387

    2012

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.