Pith. sign in

REVIEW 4 major objections 5 minor 92 references

LR0.FM: Low-Res Benchmark and Improving Robustness for Zero-Shot Classification in Foundation Models

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that small trainable low-resolution tokens, added to frozen transformer blocks and trained on synthetic diffusion images, restore much of the zero-shot classification accuracy lost at 16x16 and 32x32, and that a…

desk verdict A genuinely useful benchmark and a plausible but narrowly tested adaptation method; the WAR metric is partly circular and the real-world robustness claim outruns the evidence. read the letter →

arxiv 2502.03950 v3 pith:FMPJCFAM submitted 2025-02-06 cs.CV

Priyank Pathak , Shyam Marjit , Shruti Vyas , Yogesh S Rawat This is my paper · ORCID
classification cs.CV
keywords low-resolutionrobustnesszero-shotclassificationvision-languagefoundationmodelsCLIPvisualprompttuningsyntheticdiffusionimagesWeightedAggregatedbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that low-resolution inputs break vision-language foundation models in a predictable way, and that a small, trainable addition can repair much of the damage without touching the pre-trained weights. It builds a benchmark of 10 foundation models, 66 backbones, and 15 datasets, plus a new metric (Weighted Aggregated Robustness) that avoids inflated robustness scores from models that are already near random. The central method, LR-TK0, adds low-resolution tokens to frozen transformer blocks, trains them with multi-scale self-supervised distillation on synthetic diffusion-generated images, and reports consistent gains at 16x16 and 32x32 across several backbones. A reader should care because real cameras, surveillance footage, and satellite images routinely produce pixelated inputs, and the paper claims the fix does not require retraining the foundation model or exposing it to the target datasets.

What carries the argument

Two mechanisms carry the paper. First, LR tokens: extra trainable vectors, one set per transformer block, added to the spatial patch tokens while the pre-trained transformer stays frozen; under multi-scale self-supervised distillation, the frozen model on a high-resolution input acts as teacher, and the token-augmented model on high- and low-resolution inputs acts as student, pulled together by a CLIP-style contrastive loss. Second, Weighted Aggregated Robustness (WAR): a dataset-weighted average of an improved relative robustness score $\Gamma^D_n = \gamma^D_n(1 - e^{-\alpha (E^D)^2})$, which suppresses the inflated scores a near-random model receives and re-weights datasets so the final model ranking correlates with each dataset's ranking. The layer-similarity analysis, using L2 distances between layers of low- and high-resolution models, is what justifies putting tokens in every block, since early layers deviate most.

What would settle it

Train LR-TK0 exactly as described, then evaluate it on genuinely low-resolution images captured directly by low-end cameras, CCTV, or satellite sensors rather than on bicubic-downsampled high-resolution images. If the reported 16x16 and 32x32 gains over the frozen baseline shrink to near zero, the method's real-world robustness claim is refuted, even though it improves simulated low-resolution benchmarks.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that low-resolution inputs mainly corrupt the early layers of a vision-language transformer while semantic knowledge in the deeper layers survives, and that this corruption can be corrected by trainable tokens that learn the missing high-resolution features. LR-TK0 freezes the pre-trained model, adds LR tokens before every block, and distills the frozen high-resolution teacher's features into the token-augmented student at multiple scales using a contrastive loss; training is on diffusion-generated synthetic images, so no target dataset is seen. The paper reports consistent top-1 and robustness gains at 16x16 and 32x32 for EVA, MetaCLIP, and OpenCLIP backbones, with the largest gains at 16x16 and at most a one-to-two-point cost at full resolution. The accompanying LR0.FM benchmark establishes that larger models are more robust, that pre-training dataset quality matters more than quantity, and that fine-tuned or higher-resolution variants are less robust.

Load-bearing premise

The load-bearing premise is that bicubic downsampling, followed by upscaling, faithfully reproduces real low-resolution degradation, and that diffusion-generated synthetic images teach the same high-to-low-resolution mapping found in real photos.

Editorial extensions

If this is right

  • Zero-shot classifiers can be made robust to pixelation by adding small adapter tokens rather than fine-tuning, so deployed CLIP-style models can serve surveillance, satellite, and privacy-pixelated imagery with only a few percent extra parameters.
  • The benchmark's robustness ordering gives model selection guidance: prefer larger backbones and high-quality filtered pre-training data such as DataComp-1B over larger raw LAION-2B, and avoid fine-tuned or 336x336 variants when low-resolution inputs are expected.
  • Early layers are the main bottleneck, so future robustness methods can concentrate their learnable components on the first blocks.
  • WAR provides a more balanced way to compare robustness across datasets, changing rankings especially on EuroSAT and ImageNet-A, so robustness claims should report both simple aggregated robustness and WAR.
  • LR-TK0 combines with other zero-shot techniques such as visual prompt tuning and transfers across EVA backbone sizes, suggesting it is a general plug-in rather than a per-model fix.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims: if real-world low-resolution degradation includes sensor noise, compression, and motion blur, the bicubic-based training could miss those artifacts; testing LR-TK0 on native low-resolution camera frames is the natural check.
  • Beyond the paper's claims: the synthetic-caption training recipe could transfer to frozen encoders in other modalities, such as audio or video, since the high-to-low-resolution distillation is label-free and task-agnostic.
  • Beyond the paper's claims: the layer-wise high-resolution versus low-resolution similarity diagnostic could predict where to insert adapters in any frozen transformer, making token placement a measurable design choice rather than an empirical one.
  • Beyond the paper's claims: WAR weights optimized at 16x16 are not optimal at higher resolutions, as the paper's own correlation analysis shows, so a resolution-dependent weighting scheme may be needed for a single cross-resolution robustness score.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces LR0.FM, a benchmark for zero-shot image classification of foundation models under low-resolution inputs, covering 10 model families, 66 backbones, and 15 datasets with bicubic downsampling to 16x16, 32x32, 64x64, and 128x128. It proposes a new robustness metric, Weighted Aggregated Robustness (WAR), designed to correct two perceived flaws in prior metrics, and it reports benchmark findings on model size, pretraining data, fine-tuning, and layer-wise sensitivity. On the method side, the paper proposes LR-TK0, trainable low-resolution tokens inserted before each frozen transformer block, trained by multi-scale self-supervised distillation on synthetic diffusion-generated images, and reports consistent gains in low-resolution zero-shot accuracy without modifying pretrained weights.

Significance. If the reported results hold, the paper is a useful and timely contribution: it is, to my knowledge, the largest systematic study of very-low-resolution zero-shot classification for CLIP-style models, and it ships code and a broad set of measurements that the community can reuse. The LR-TK0 idea is simple, parameter-efficient, and well motivated by the paper's layer-wise analysis. The benchmark and method are both reproducible in principle. The main limitations are that the WAR metric is validated on the same data used to fit its weights, the method is trained and evaluated under the same synthetic bicubic degradation model without a held-out degradation test, and the reported gains lack error bars and statistical tests. These issues do not destroy the benchmark's value but they do weaken the strength of the central claims as currently stated.

major comments (4)
  1. [Sec. 3, Eq. (2) and Fig. 5] The validation of the WAR metric is partly circular. The dataset weights w_d^16 are optimized with Ax to maximize Spearman correlations between the weighted-aggregate ranking and the per-dataset rankings, and then the same correlations are reported as evidence that WAR improves representation of EuroSAT and ImageNet-A. Optimizing a quantity and then measuring it on the fitting data is not a valid test of the metric. Please reframe Fig. 5 as a fitting diagnostic, or validate the weights on held-out datasets, held-out model subsets, or via cross-validation.
  2. [Sec. 5.2 and Appendix E, Listing 1] LR-TK0 is trained and evaluated under the same degradation model: bicubic downsampling to a low resolution followed by bicubic upscaling to the model's native resolution. The only non-synthetic evidence is the qualitative real-world example in Appendix H (Fig. 29). As a result, the conclusion that LR-TK0 improves robustness in real-world low-resolution settings is not established. Please add quantitative evaluation on held-out degradations (e.g., JPEG compression, sensor noise, motion blur, or at least nearest-neighbor downsampling), or substantially temper the real-world generalization claims in the abstract and conclusion.
  3. [Tables 2-6 and Sec. 6.1] No error bars, random seeds, confidence intervals, or statistical tests are reported, and the training budgets differ across models (200 epochs for EVA vs. 10 epochs for MetaCLIP and OpenCLIP). Some of the reported improvements are small, for example EVA-B/16 at 32x32 with SAR 74.4 to 75.3 and WAR 64.8 to 66.4. Multiple seeds and a paired significance test are needed to establish that the gains are not due to training stochasticity or initialization, especially since token training is the central method claim.
  4. [Sec. 4, Figs. 5 and 6] The headline findings "model size positively correlates with robustness" and "pretraining dataset quality is more important than size" are correlational and confounded. The DataComp-1B vs. LAION-2B comparison in Fig. 6 does not control for model family, architecture, or training recipe, and Fig. 5 shows only weak correlations with many exceptions. Please either add controlled comparisons that isolate dataset size from model family, or soften the causal language to describe associations rather than drivers.
minor comments (5)
  1. [Sec. 4, first paragraph] The text says the correlations are computed "using SAR and WAR averaging of relative robustness Γ16", but Sec. 3 defines SAR on γ and WAR on Γ. Please reconcile this notation for consistency.
  2. [Sec. 3, Eq. (1)] The hyperparameter α=200 is selected by hand as a middle value between 100 and 500. The sensitivity plot in Fig. 21 shows the behavior, but the paper does not state a selection criterion or whether the benchmark conclusions are stable across a reasonable range of α.
  3. [Appendix F.3 and Table 13] The IDM evaluation is reported only on Pets because of a stated mismatch between the paper's weights and the official GitHub implementation, and Table 13 omits IDM from the main comparison. This is reasonable, but the sentence "IDM generalized Zero shot weights do not match their GitHub implementation" is unclear; please state exactly which weights were used and why IDM is excluded from Table 3.
  4. [Appendix G, Fig. 24] Figure 24 shows that weights optimized for 16x16 become identical to SAR at 64x64 and 128x128. The paper should state explicitly that WAR's advantage is concentrated at 16x16 and 32x32, and should not claim that WAR improves evaluation uniformly across all resolutions.
  5. [Sec. 1] The claim that "no prior work has explored this aspect of FMs" is stronger than needed and is not essential to the contribution. Consider softening the novelty claim or citing the closest existing low-resolution robustness studies for vision-language models.

Circularity Check

1 steps flagged · score 6.0 of 10

WAR's validation is partially circular: dataset weights are fit to maximize Spearman correlations, and the same correlations are reported as evidence; LR-TK0 and benchmark findings remain independent.

  1. fitted input called prediction [Section 3, 'Solution: Weighted Aggregated Robustness' (Eq. 2 and weight optimization); Section 4, 'Proposed WAR Metrics']
    ""We use Ax tool (Bakshy et al., 2018) for optimizing the weights of the dataset wd16 ∈ [0.1, 1] such that the Spearman correlation (SC) between the final model ranking obtained after the weighted averaging and individual dataset ranking is maximized... 0.95×SC(Imagenet)+SC(ImageNet-V2)+SC(DTD)+SC(ImageNet-A)+SC(EuroSAT) (2)" ... "WAR shows a slight decrease in avg. correlation (SAR-16 0.89 vs WAR-16 0.87), but it also improves the representation of EuroSAT & ImageNet-A. The correlation score for EuroSAT increased from a weak/no correlation of 0.26 to a moderate 0.49.""

    The WAR dataset weights are fitted parameters: they are optimized to maximize a weighted sum of Spearman correlations between the aggregate model ranking and individual dataset rankings (Eq. 2). Section 4 then reports the resulting improvements in EuroSAT and ImageNet-A Spearman correlations as evidence that WAR 'represents' datasets fairly. These are the same quantities being optimized, so the improvement is substantially by construction rather than an independent validation of the metric. The circularity is partial: the fitted weights are later transferred to other resolutions (Figure 24) and the LR-TK0 accuracy gains are measured independently, so the benchmark and method conclusions are not forced by this fit.

full rationale

I found one concrete circularity, confined to the WAR metric. The dataset weights in Eq. 2 are optimized to maximize a weighted sum of Spearman correlations between the aggregate ranking and individual dataset rankings; Section 4 then presents the resulting EuroSAT and ImageNet-A correlation increases as evidence that WAR 'represents' datasets fairly. That is a fitted input reported as a prediction. I did not score higher because the paper's other central claims are independent: LR-TK0's accuracy gains are measured on 15 target datasets never used in training; ablations control for HR degradation, token position, multi-scale buckets, and number of captions; and the weight transfer to 32x32/64x64/128x128 (Figure 24) provides a partial held-out check. The bicubic train/eval match for LR-TK0 is a real generalization limitation but not a definitional circularity, since the evaluation does not reduce to the training objective by construction. Self-citations (e.g., Schiappa et al. 2024) supply the baseline gamma/SAR metric being replaced, not an unverified load-bearing premise.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central method rests on a modest set of hand-chosen hyperparameters (alpha, WAR weights, training epochs, synthetic dataset size) and on domain assumptions about how well synthetic degradation approximates real low-resolution images. The most concerning fitted quantity is the WAR dataset-weight vector, which is optimized on the same benchmark it later evaluates.

free parameters (5)
  • WAR dataset weights w_d^n = 15 values between 0.01 and 1.0, see Table 11
    Optimized with the Ax tool to maximize a weighted Spearman correlation between aggregate model rankings and per-dataset rankings (Eq. 2). The weights are fit on the same 66 models and 15 datasets they are later used to evaluate.
  • Gamma penalty alpha = 200
    Chosen by hand in Section 3 as a middle ground between 100 and 500; controls how quickly improved relative robustness collapses near random accuracy.
  • Spearman objective coefficients = 0.95 for ImageNet, ImageNet-V2, DTD; 1.0 for ImageNet-A, EuroSAT
    In Eq. (2), these coefficients weight which per-dataset rankings should be reflected in the aggregate; chosen empirically.
  • LR-TK0 training epochs = 200 (EVA), 10 (MetaCLIP, OpenCLIP)
    Implementation details in Section 6; different per model, with no ablation on epoch count.
  • Synthetic dataset size = 7K captions, 30 images per caption
    Section 5.2; ablations show 2K captions also help, so this is a design choice rather than a fitted constant.
assumptions (4)
  • domain assumption Bicubic downsampling to LR, then upscaling to model resolution, is a valid proxy for real-world low-resolution images
    Used in Section 3 to define the benchmark task; real LR artifacts such as sensor noise, compression, and motion blur may differ.
  • domain assumption Frozen pre-trained teacher HR features are a good supervision target for LR students
    Section 5.1 anchors student LR features to teacher HR features under a contrastive loss; assumes teacher features are stable and desirable.
  • domain assumption PIXART-alpha generated images from random Conceptual Captions provide a task-agnostic training distribution
    Section 5.2 relies on these synthetic images to generalize to the 15 target datasets; assumes distributional coverage.
  • domain assumption Template-based zero-shot classification (CLIP protocol) is a valid measure of model performance
    All evaluations use CLIP-style text templates (Section 3); this is standard practice but still an assumption about what the benchmark measures.
invented entities (1)
  • LR tokens independent evidence
    purpose: Trainable tokens added to every frozen transformer block to compensate for loss of detail in low-resolution images
    Trained on synthetic data and evaluated on 15 unseen benchmark datasets; the accuracy gains at 16x16 and 32x32 provide an external falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LR0.FM: Low-Res Benchmark and Improving Robustness for Zero-Shot Classification in Foundation Models." pith.science (2026). https://pith.science/paper/FMPJCFAM

@misc{pith2026250203950,
  author       = {Pith},
  title        = {Pith review of: LR0.FM: Low-Res Benchmark and Improving Robustness for Zero-Shot Classification in Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FMPJCFAM}},
  note         = {Machine review of arXiv:2502.03950}
}
read the original abstract

Visual-language foundation Models (FMs) exhibit remarkable zero-shot generalization across diverse tasks, largely attributed to extensive pre-training on largescale datasets. However, their robustness on low-resolution/pixelated (LR) images, a common challenge in real-world scenarios, remains underexplored. We introduce LR0.FM, a comprehensive benchmark evaluating the impact of low resolution on the zero-shot classification performance of 10 FM(s) across 66 backbones and 15 datasets. We propose a novel metric, Weighted Aggregated Robustness, to address the limitations of existing metrics and better evaluate model performance across resolutions and datasets. Our key findings show that: (i) model size positively correlates with robustness to resolution degradation, (ii) pre-training dataset quality is more important than its size, and (iii) fine-tuned and higher resolution models are less robust against LR. Our analysis further reveals that the model makes semantically reasonable predictions at LR, and the lack of fine-grained details in input adversely impacts the model's initial layers more than the deeper layers. We use these insights and introduce a simple strategy, LR-TK0, to enhance the robustness of models without compromising their pre-trained weights. We demonstrate the effectiveness of LR-TK0 for robustness against low-resolution across several datasets and its generalization capability across backbones and other approaches. Code is available at https://github.com/shyammarjit/LR0.FM

Figures

Figures reproduced from arXiv: 2502.03950 by the authors.

Figure 1
Figure 1. Top-1 zero-shot classification accuracy (y-axis) vs resolution (x-axis): Backbones for foundation models are merged as shade, with average performance across backbones in the dark. ABSTRACT Visual-language foundation Models (FMs) exhibit remarkable zero-shot general￾ization across diverse tasks, largely attributed to extensive pre-training on large￾scale datasets. However, their robustness on low-resolution/pixelate… view at source ↗
Figure 2
Figure 2. Zero-Shot misclassifications: EVA-CLIP [2023a] correct classification at 224×224 (green) & misclassification at lower resolution (red). However, ImageNet labels-based mispredictions are semantically reasonable (humans), indicating viability of pre-trained weights at low resolution. data (Zhou et al., 2020) etc. In these cases, details crucial for accurate classification may be ob￾scured by artifacts like pixelation … view at source ↗
Figure 3
Figure 3. Left: Dataset: Size ∝ log # test images, and color gradient ∝ # of test classes orange is 10 & black is 1000 classes). Right: Zero Shot Evaluation: Food-101 image (32×32) generates image embeddings fImg, while class labels are filled in templates (1 shown) generating text embeddings (averaged across templates). The dot product of fImg with text features gives classification logits. tions, token dropping, training vi… view at source ↗
Figures from the paper (24 more)
Figure 4
Figure 4. Figure 4: Left: Improved ΓD n vs traditional γD n : Γ D n ≈γ D n except near random predictions (ED → 0). Mid: Correlation between the ordering of models after averaging of robustness (SAR) across datasets (γ D 16 & Γ D 16) with dataset’s true ordering. SAR final ranking ignores…
Figure 5
Figure 5. Figure 5: Evaluations at 16×16. Left: SAR vs WAR: WAR improves the correlation (between the ordering of models after aggregation with individual datasets) for EuroSAT (0.26 → 0.49 and ImageNet-A (0.56 → 0.68), both computed via Γ D 16. Right: i) Model Size & ii) Pre-training dat…
Figure 6
Figure 6. Figure 6: Left: DataComp-1B vs LAION-2B: Smaller DataComp-1B pre-training helps robustness. Models are ordered via size. Mid: Model comparison w/o Size: Models binned into size buckets (±30M). Right: Fine-tuning degrades robustness. (left & mid): bigger models are more robust. W…
Figure 7
Figure 7. Figure 7: Left: High Input Resolution Model are less robust. Mid: t-SNE of Dataset robustness Dataset represented via 66 models robustness (Γ D 16), indicates 3 clusters. Right: Layers-wise fea￾tures L2 similarity: n×n model layers similarity w/ 224×224 ones, for EVA02-B-16. For…
Figure 8
Figure 8. Figure 8: Feats t-SNE: EVA-02-CLIP-B/16 test features for Food-101, colored using class labels. With low resolutions (16×16, and 32×32), features become indistinguishable, thereby overlapping. AddSR BSRGAN ESRGAN IDM Inf-DiT Swinir HR: 224 x 224 LR: 16 x 16 AddSR BSRGAN ESRGAN I…
Figure 9
Figure 9. Figure 9: Super resolution at 16×16: Image from Pets (left) and Food102 (right). Models include AddSR [2024], BSRGAN [2021], ESRGAN [2018], IDM [2023], Inf-DiT [2024], and Swinir [2021]. Inside Model [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Fire (& ice) icons represent trainable (& frozen) parameters. [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Synthetic Images: (Left) Images generated using PIXART-α [2023] using randomly sampled captions from Conceptual Captions [2018]. (Right) Multiple images per caption. embeddings for both HR (f S HR) and LR image(s) (f S LR) as shown in fig. 10 (right). f S HR, f S LR(s…
Figure 12
Figure 12. Figure 12: Baseline vs LR-TK0: Top-1 accuracy for EVA-B/16 on 16×16. (more in Supplementary) [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]
Figure 14
Figure 14. Figure 14: LR-TK0 improves all EVA backbones: L@336 is L/14 with 336 input SAR -16 Position of LR Token Introduction Baseline LR-TK0 WAR -16 Position of LR Token Introduction Baseline LR-TK0 [PITH_FULL_IMAGE:figures/full_fig_p010_14.png]
Figure 16
Figure 16. Figure 16: LR token Grad-CAM: Baseline (EVA-B/16) attention is scattered at 16×16 (compared to 224×224). LR-TK0 focuses on the object, likely capturing fine-grained details. @: input resolution. # Images/Caption [PITH_FULL_IMAGE:figures/full_fig_p010_16.png]
Figure 17
Figure 17. Figure 17: Top-1 Accuracy drop: Drop in accuracy for all models for all the datasets. The color scheme same as [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: Vanilla vs LR-TK0 (Our): Top-1 accuracy for EVA-02-CLIP-B/16 model for dif￾ferent resolutions. VPT Implementation: VPT is the same as ours, instead of adding on top of spatial tokens, train￾able 50 tokens are concatenated to frozen spatial tokens before the first bloc…
Figure 19
Figure 19. Figure 19: Vanilla vs LR-TK0 (Our): Gamma Robustness for EVA-02-CLIP-B/16 model for different resolutions on each dataset [PITH_FULL_IMAGE:figures/full_fig_p024_19.png]
Figure 20
Figure 20. Figure 20: Images/ Caption : For ACC, and WAR, evaluation metrics on 16×16. SAR in the main paper. GAN Wang et al. (2018), SwinIR Liang et al. (2021), and AddSR Xie et al. (2024). All super￾resolution methods were employed in a zero-shot setting to ensure a fair comparison. Our …
Figure 21
Figure 21. Figure 21: Rate of robustness declines as accuracy approaches random prediction [PITH_FULL_IMAGE:figures/full_fig_p026_21.png]
Figure 22
Figure 22. Figure 22: Position of LR tokens introduction: No tokens were added after the position embed￾ding stage. [i]-th indicates the block from which LR tokens were introduced. Performance metrics variants of [PITH_FULL_IMAGE:figures/full_fig_p026_22.png]
Figure 23
Figure 23. Figure 23: Effect of LR token: ‘@’ is input resolution. Vanilla model attention is scattered at 16×16 (compared to 224×224), while our LR tokens focus on the object, capturing fine-grained details. Furthermore, we present a side-by-side comparison between the LR token introducti…
Figure 24
Figure 24. Figure 24: Spearman Correlation for weights derived for 16×16 for higher resolutions. Samples of Diffusion Generated Images: In [PITH_FULL_IMAGE:figures/full_fig_p027_24.png]
Figure 25
Figure 25. Figure 25: Spearman Correlation for different optimization function. The optimization objective is to maximize the mentioned dataset Spearman correlation (SC). For Example ‘Imagenet: 0.95, EuroSAT:1’ means: SC (Imagenet)×0.95 + SC (EuroSAT)×1. money falling from the sky dancer -…
Figure 26
Figure 26. Figure 26: Synthetic Images: Images generated using PIXART-α (Chen et al., 2023) using the captions randomly sampled from Conceptual Captions (Sharma et al., 2018). Left: Sample Images, while right shows multiple images per caption generated via different seeds. More examples of…
Figure 27
Figure 27. Figure 27: Synthetic Images2: Mutliple Images / Caption. More examples of [PITH_FULL_IMAGE:figures/full_fig_p029_27.png]
Figure 28
Figure 28. Figure 28: Semantically Correct Predictions: More examples of [PITH_FULL_IMAGE:figures/full_fig_p029_28.png]
Figure 29
Figure 29. Figure 29: Real World low-resolution images: Top-5 predictions for images taken from Google (true label shown, below image). Blue indicates Semantically reasonable prediction, Green indi￾cates correct prediction, and Red means wrong prediction. EVA-02-CLIP-B/16 model predic￾tion…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

92 extracted references · 42 canonical work pages

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35: 0 23716--23736, 2022

    2022

  2. [2]

    Ae: A domain-agnostic platform for adaptive experimentation

    Eytan Bakshy, Lili Dworkin, Brian Karrer, Konstantin Kashin, Ben Letham, Ashwin Murthy, and Shaun Singh. Ae: A domain-agnostic platform for adaptive experimentation. In NeurIPS Systems for ML Workshop, 2018. URL http://learningsys.org/nips18/assets/papers/87CameraReadySubmissionAE

    2018

  3. [3]

    Food-101--mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101--mining discriminative components with random forests. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, pp.\ 446--461. Springer, 2014

    2014

  4. [4]

    Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 3558--3568, 2021

    2021

  5. [5]

    An empirical study and analysis of generalized zero-shot learning for object recognition in the wild

    Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, and Fei Sha. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pp.\ 52--68. Springer, 2016

    2016

  6. [6]

    Pixart- : Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- : Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023

    2023

  7. [7]

    Robustsam: Segment anything robustly on degraded images

    Wei-Ting Chen, Yu-Jiet Vong, Sy-Yen Kuo, Sizhou Ma, and Jian Wang. Robustsam: Segment anything robustly on degraded images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 4081--4091, 2024

    2024

  8. [8]

    Pali: A jointly-scaled multilingual language-image model

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209.06794, 2022

    arXiv 2022

Show all 92 references
  1. [9]

    Learning resolution-invariant deep representations for person re-identification

    Yun-Chun Chen, Yu-Jhe Li, Xiaofei Du, and Yu-Chiang Frank Wang. Learning resolution-invariant deep representations for person re-identification. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp.\ 8215--8222, 2019

    2019

  2. [10]

    Low-resolution face recognition

    Zhiyi Cheng, Xiatian Zhu, and Shaogang Gong. Low-resolution face recognition. In Computer Vision--ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2--6, 2018, Revised Selected Papers, Part III 14, pp.\ 605--621. Springer, 2019

    2018

  3. [11]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 3606--3613, 2014

    2014

  4. [12]

    Mevid: Multi-view extended videos with identities for video person re-identification

    Daniel Davila, Dawei Du, Bryon Lewis, Christopher Funk, Joseph Van Pelt, Roderic Collins, Kellie Corona, Matt Brown, Scott McCloskey, Anthony Hoogs, et al. Mevid: Multi-view extended videos with identities for video person re-identification. In Proceedings of the IEEE/CVF Wint...

    2023

  5. [13]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009

    2009

  6. [14]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019. URL https://api.semanticscholar.org/CorpusID:52967399

    2019

  7. [15]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. International Conference on Learning Representations, 2021

    2021

  8. [16]

    Data filtering networks

    Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. Data filtering networks. arXiv preprint arXiv:2309.17425, 2023

    2023 arXiv

  9. [17]

    Datacomp: In search of the next generation of multimodal datasets

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Orgad, Rahim Entezari, Giannis Daras, Sarah Pratt, Vivek Ramanujan, Yonatan Bitton, Kalyani Marathe, Stephen Mussman...

    2023 arXiv

  10. [18]

    Implicit diffusion models for continuous super-resolution

    Sicheng Gao, Xuhui Liu, Bohan Zeng, Sheng Xu, Yanjing Li, Xiaoyan Luo, Jianzhuang Liu, Xiantong Zhen, and Baochang Zhang. Implicit diffusion models for continuous super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 10021...

    2023

  11. [19]

    Efficient low-resolution face recognition via bridge distillation

    Shiming Ge, Shengwei Zhao, Chenyu Li, Yu Zhang, and Jia Li. Efficient low-resolution face recognition via bridge distillation. IEEE Transactions on Image Processing, 29: 0 6898--6908, 2020. doi:10.1109/TIP.2020.2995049

    2020

  12. [20]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 15180--15190, 2023

    2023

  13. [21]

    Caltech-256 object category dataset

    Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. Caltech Technical Report, 2007

    2007

  14. [22]

    M2-encoder: Advancing bilingual image-text understanding by large-scale efficient pretraining, 2024

    Qingpei Guo, Furong Xu, Hanxiao Zhang, Wang Ren, Ziping Ma, Lin Ju, Jian Wang, Jingdong Chen, and Ming Yang. M2-encoder: Advancing bilingual image-text understanding by large-scale efficient pretraining, 2024. URL https://arxiv.org/abs/2401.15896

    2024 arXiv

  15. [23]

    Audioclip: Extending clip to image, text and audio

    Andrey Guzhov, Federico Raue, J \"o rn Hees, and Andreas Dengel. Audioclip: Extending clip to image, text and audio. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 976--980. IEEE, 2022

    2022

  16. [24]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  17. [25]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12 0 (7): 0 2217--2226, 2019

    2019

  18. [26]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization. ICCV, 2021 a

    2021

  19. [27]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. CVPR, 2021 b

    2021

  20. [28]

    Openclip, July 2021

    Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. URL https://doi.org/10.5281/zenodo.5143773

    2021 doi

  21. [29]

    Visual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, pp.\ 709--727. Springer, 2022

    2022

  22. [30]

    Resolution invariant face recognition using a distillation approach

    Syed Safwan Khalid, Muhammad Awais, Zhen-Hua Feng, Chi-Ho Chan, Ammarah Farooq, Ali Akbari, and Josef Kittler. Resolution invariant face recognition using a distillation approach. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2 0 (4): 0 410--420, 2020. doi:1...

    2020

  23. [31]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 4015--4026, 2023

    2023

  24. [32]

    Similarity of neural network representations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In International conference on machine learning, pp.\ 3519--3529. PMLR, 2019

    2019

  25. [33]

    Lsun-stanford car dataset: enhancing large-scale car image datasets using deep learning for usage in gan training

    Tin Kramberger and Bo z idar Poto c nik. Lsun-stanford car dataset: enhancing large-scale car image datasets using deep learning for usage in gan training. Applied Sciences, 10 0 (14): 0 4913, 2020

    2020

  26. [34]

    Visual genome: Connecting language and vision using crowdsourced dense image annotations

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer ...

    2017

  27. [35]

    Elevater: A benchmark and toolkit for evaluating language-augmented visual models

    Chunyuan Li, Haotian Liu, Liunian Li, Pengchuan Zhang, Jyoti Aneja, Jianwei Yang, Ping Jin, Houdong Hu, Zicheng Liu, Yong Jae Lee, and Jianfeng Gao. Elevater: A benchmark and toolkit for evaluating language-augmented visual models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belg...

    2022

  28. [36]

    Selvaraju, Akhilesh Deepak Gotmare, Shafiq Joty, Caiming Xiong, and Steven Hoi

    Junnan Li, Ramprasaath R. Selvaraju, Akhilesh Deepak Gotmare, Shafiq Joty, Caiming Xiong, and Steven Hoi. Align before fuse: Vision and language representation learning with momentum distillation. In NeurIPS, 2021

    2021

  29. [37]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, 2022 b

    2022

  30. [38]

    Face recognition in low quality images: A survey

    Pei Li, Loreto Prieto, Domingo Mery, and Patrick Flynn. Face recognition in low quality images: A survey. arXiv preprint arXiv:1805.11519, 2018

    2018 arXiv

  31. [39]

    Pei Li, Loreto Prieto, Domingo Mery, and Patrick J. Flynn. On low-resolution face recognition in the wild: Comparisons and new techniques. IEEE Transactions on Information Forensics and Security, 14 0 (8): 0 2000--2012, 2019. doi:10.1109/TIFS.2018.2890812

  32. [40]

    RECLIP : Resource-efficient CLIP by training with small images

    Runze Li, Dahun Kim, Bir Bhanu, and Weicheng Kuo. RECLIP : Resource-efficient CLIP by training with small images. Transactions on Machine Learning Research, 2023 a . ISSN 2835-8856. URL https://openreview.net/forum?id=Ufc5cWhHko

    2023

  33. [41]

    An inverse scaling law for clip training

    Xianhang Li, Zeyu Wang, and Cihang Xie. An inverse scaling law for clip training. In NeurIPS, 2023 b

    2023

  34. [42]

    Clipa-v2: Scaling clip training with 81.1 arXiv preprint arXiv:2306.15658, 2023 c

    Xianhang Li, Zeyu Wang, and Cihang Xie. Clipa-v2: Scaling clip training with 81.1 arXiv preprint arXiv:2306.15658, 2023 c

    2023 arXiv

  35. [43]

    Swinir: Image restoration using swin transformer

    Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 1833--1844, 2021

    2021

  36. [44]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Procee...

    2014

  37. [45]

    Frozen clip models are efficient video learners

    Ziyi Lin, Shijie Geng, Renrui Zhang, Peng Gao, Gerard De Melo, Xiaogang Wang, Jifeng Dai, Yu Qiao, and Hongsheng Li. Frozen clip models are efficient video learners. In European Conference on Computer Vision, pp.\ 388--404. Springer, 2022

    2022

  38. [46]

    Few-shot adaptation of multi-modal foundation models: A survey

    Fan Liu, Tianshu Zhang, Wenwen Dai, Wenwen Cai, Xiaocong Zhou, and Delong Chen. Few-shot adaptation of multi-modal foundation models: A survey. arXiv preprint arXiv:2401.01736, 2024

    2024 arXiv

  39. [47]

    Flower classification via convolutional neural network

    Yuanyuan Liu, Fan Tang, Dengwen Zhou, Yiping Meng, and Weiming Dong. Flower classification via convolutional neural network. In 2016 IEEE International Conference on Functional-Structural Plant Growth Modeling, Simulation, Visualization and Applications (FSPMA), pp.\ 110--116....

    2016

  40. [48]

    Luevano, Leonardo Chang, Heydi Méndez-Vázquez, Yoanna Martínez-Díaz, and Miguel González-Mendoza

    Luis S. Luevano, Leonardo Chang, Heydi Méndez-Vázquez, Yoanna Martínez-Díaz, and Miguel González-Mendoza. A study on the performance of unconstrained very low resolution face recognition: Analyzing current trends and new research directions. IEEE Access, 9: 0 75470--75493, 202...

    2021

  41. [49]

    Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning

    Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning. Neurocomputing, 508: 0 293--304, 2022

    2022

  42. [50]

    Fine-grained visual classification of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013

    2013 arXiv

  43. [51]

    End-to-end learning of visual representations from uncurated instructional videos

    Antoine Miech, Jean-Baptiste Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, and Andrew Zisserman. End-to-end learning of visual representations from uncurated instructional videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 987...

    2020

  44. [52]

    Rethinking image super-resolution from training data perspectives

    Go Ohtani, Ryu Tadokoro, Ryosuke Yamada, Yuki M Asano, Iro Laina, Christian Rupprecht, Nakamasa Inoue, Rio Yokota, Hirokatsu Kataoka, and Yoshimitsu Aoki. Rethinking image super-resolution from training data perspectives. arXiv preprint arXiv:2409.00768, 2024

    2024 arXiv

  45. [53]

    Im2text: Describing images using 1 million captioned photographs

    Vicente Ordonez, Girish Kulkarni, and Tamara Berg. Im2text: Describing images using 1 million captioned photographs. Advances in neural information processing systems, 24, 2011

    2011

  46. [54]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pp.\ 3498--3505. IEEE, 2012

    2012

  47. [55]

    Patil, R

    Jyoti S. Patil, R. S. Pawase, and Yogesh H. Dandawate. Classification of low resolution astronomical images using convolutional neural networks. 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), pp.\ 1168--...

    2017

  48. [56]

    Unsafebench: Benchmarking image safety classifiers on real-world and ai-generated images

    Yiting Qu, Xinyue Shen, Yixin Wu, Michael Backes, Savvas Zannettou, and Yang Zhang. Unsafebench: Benchmarking image safety classifiers on real-world and ai-generated images. arXiv preprint arXiv:2405.03486, 2024

    2024 arXiv

  49. [57]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    2019

  50. [58]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...

    2021

  51. [59]

    Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pp.\ 5389--5400

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pp.\ 5389--5400. PMLR, 2019

    2019

  52. [60]

    Probing conceptual understanding of large visual-language models

    Madeline Chantry Schiappa, Michael Cogswell, Ajay Divakaran, and Yogesh Singh Rawat. Probing conceptual understanding of large visual-language models. arXiv preprint arXiv:2304.03659, 2023

    2023 arXiv

  53. [61]

    Robustness analysis on foundational segmentation models

    Madeline Chantry Schiappa, Shehreen Azad, Sachidanand Vs, Yunhao Ge, Ondrej Miksik, Yogesh S Rawat, and Vibhav Vineet. Robustness analysis on foundational segmentation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 1786--1796, 2024

    2024

  54. [62]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021

    2021 arXiv

  55. [63]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Informa...

    2022

  56. [64]

    Dafnis, Zhixing Zhang, Shiyu Zhao, and Dimitris Metaxas

    Samuel Schulter, Vijay Kumar B G, Yumin Suh, Konstantinos M. Dafnis, Zhixing Zhang, Shiyu Zhao, and Dimitris Metaxas. Omnilabel: A challenging benchmark for language-based object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\...

    2023

  57. [65]

    Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of ACL, 2018

    2018

  58. [66]

    Test-time prompt tuning for zero-shot generalization in vision-language models

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. Advances in Neural Information Processing Systems, 35: 0 14274--14289, 2022

    2022

  59. [67]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012

    2012 arXiv

  60. [68]

    Eva-clip: Improved training techniques for clip at scale

    Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. arXiv preprint arXiv:2303.15389, 2023 a

    2023 arXiv

  61. [69]

    Eva-clip-18b: Scaling clip to 18 billion parameters

    Quan Sun, Jinsheng Wang, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, and Xinlong Wang. Eva-clip-18b: Scaling clip to 18 billion parameters. arXiv preprint arXiv:2402.04252, 2023 b

    2023 arXiv

  62. [70]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation ...

    2023 arXiv

  63. [71]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, pp.\ 5998–6008, 2017

    2017

  64. [72]

    Learning robust global representations by penalizing local predictive power

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, pp.\ 10506--10518, 2019

    2019

  65. [73]

    Magneto: A foundation transformer

    Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, and Furu Wei. Magneto: A foundation transformer. In Andreas Krause, Emma Brunskill, Kyunghyun Cho...

    2023

  66. [74]

    Esrgan: Enhanced super-resolution generative adversarial networks

    Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops, pp.\ 0--0, 2018

    2018

  67. [75]

    Q-bench: A benchmark for general-purpose foundation models on low-level vision

    Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, et al. Q-bench: A benchmark for general-purpose foundation models on low-level vision. arXiv preprint arXiv:2309.14181, 2023

    2023 arXiv

  68. [76]

    Zero-shot learning-the good, the bad and the ugly

    Yongqin Xian, Bernt Schiele, and Zeynep Akata. Zero-shot learning-the good, the bad and the ugly. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 4582--4591, 2017

    2017

  69. [77]

    Addsr: Accelerating diffusion-based blind super-resolution with adversarial diffusion distillation

    Rui Xie, Ying Tai, Kai Zhang, Zhenyu Zhang, Jun Zhou, and Jian Yang. Addsr: Accelerating diffusion-based blind super-resolution with adversarial diffusion distillation. arXiv preprint arXiv:2404.01717, 2024

    2024 arXiv

  70. [78]

    Demystifying CLIP data

    Hu Xu, Saining Xie, Xiaoqing Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Demystifying CLIP data. In The Twelfth International Conference on Learning Representations, 2024 a . URL https://openreview.ne...

    2024

  71. [79]

    ODISE: Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, and Shalini De Mello. ODISE: Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models . arXiv preprint arXiv: 2303.04803, 2023

    2023 arXiv

  72. [80]

    What is Next in Multimodal Foundation Models?

    Zhenlin Xu, Yi Zhu, Tiffany Deng, Abhay Mittal, Yanbei Chen, Manchen Wang, Paolo Favaro, Joe Tighe, and Davide Modolo. Benchmarking zero-shot recognition with vision-language models: Challenges on granularity and specificity. In CVPR 2024 Workshop on "What is Next in Multimoda...

    2024

  73. [81]

    Inf-dit: Upsampling any-resolution image with memory-efficient diffusion transformer

    Zhuoyi Yang, Heyang Jiang, Wenyi Hong, Jiayan Teng, Wendi Zheng, Yuxiao Dong, Ming Ding, and Jie Tang. Inf-dit: Upsampling any-resolution image with memory-efficient diffusion transformer. arXiv preprint arXiv:2405.04312, 2024

    2024 arXiv

  74. [82]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL https://openreview.net/forum?id=Ee277P3AYC

    2022

  75. [83]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 11975--11986, October 2023

    2023

  76. [84]

    Designing a practical degradation model for deep blind image super-resolution

    Kai Zhang, Jingyun Liang, Luc Van Gool, and Radu Timofte. Designing a practical degradation model for deep blind image super-resolution. In IEEE International Conference on Computer Vision, pp.\ 4791--4800, 2021

    2021

  77. [85]

    A progressive framework of vision-language knowledge distillation and alignment for multilingual scene

    Wenbo Zhang, Yifan Zhang, Jianfeng Lin, Binqiang Huang, Jinlu Zhang, and Wenhao Yu. A progressive framework of vision-language knowledge distillation and alignment for multilingual scene. arXiv preprint arXiv:2404.11249, 2024

    2024 arXiv

  78. [86]

    Regionclip: Region-based language-image pretraining

    Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region-based language-image pretraining. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, ...

    2022

  79. [87]

    Learning deep features for scene recognition using places database

    Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. Learning deep features for scene recognition using places database. Advances in neural information processing systems, 27, 2014

    2014

  80. [88]

    Privacy-sensitive objects pixelation for live video streaming

    Jizhe Zhou, Chi-Man Pun, and Yu Tong. Privacy-sensitive objects pixelation for live video streaming. In Proceedings of the 28th ACM International Conference on Multimedia, MM '20, pp.\ 3025–3033, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450379885....

    2020

  81. [89]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  82. [90]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  83. [91]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  84. [92]

    Text Encoders are mostly modified vanilla transformers (Tran.) NIPS2017_3f5ee243

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.