Pith. sign in

REVIEW 3 major objections 5 minor 39 references

Quality Text, Robust Vision: The Role of Language in Enhancing Visual Robustness of Vision-Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Using detailed image captions instead of class labels to generate adversarial examples gives CLIP the best reported zero-shot robustness and clean accuracy across 16 datasets.

desk verdict Caption-guided adversarial fine-tuning works on average, but the paper overclaims SOTA and never shows that caption semantics—rather than length and syntax—drive the gains. read the letter →

arxiv 2507.16257 v2 pith:VFOG632F submitted 2025-07-22 cs.CV

Futa Waseda , Saku Sugawara , Isao Echizen This is my paper · ORCID
classification cs.CV
keywords adversarialrobustnessfine-tuningvision-languagemodelsCLIPzero-shotimageclassificationcaptionstext-guidedtrainingAutoAttack
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

QT-AFT claims that the text used to guide adversarial fine-tuning determines how well CLIP's robustness transfers to unseen tasks. The paper shows that generating adversarial examples by pushing them away from detailed image captions, rather than short class labels or no text at all, yields the best average zero-shot clean and adversarial accuracy across 16 datasets, while also avoiding the overfitting of label-guided methods. The authors argue that captions carry the full semantic content of an image, including objects, properties, spatial relations, and even word order, so the model learns robust features that generalize. This frames high-quality language as the central resource for robust visual representation learning.

What carries the argument

The machinery is the QT-AFT objective (Eq. 7): an adversarial example $x'$ is generated by PGD to maximize a weighted sum of (1) the $\ell^2$ distance between the fine-tuned and original vision encoders' embeddings of the attacked image, preserving the unsupervised generalization of FARE, and (2) the image-to-text InfoNCE loss between $x'$ and its pre-generated caption $t$, which in CLIP's shared embedding space pulls the adversarial image away from the caption's semantic content; the vision encoder is then fine-tuned to minimize the same loss. The captions themselves, generated by InternVL-2.5-8B under a 50-word prompt, are the carrier of the diverse semantics, including object names, properties, and relations, that make the adversarial examples semantically challenging.

What would settle it

Pair each ImageNet training image with a caption from a different image, keeping the same vocabulary and length distribution, then run QT-AFT; if average AutoAttack robustness stays near the matched-caption result of 25.8%, semantic alignment is not what drives the gain, whereas a drop toward FARE's 22.4% would confirm that caption-content alignment matters.

Watch

Extended reading notes

Core claim

The paper's central claim is that high-quality captions are better adversarial guidance than class labels. QT-AFT pre-generates roughly 50-word captions for ImageNet with a vision-language model, then fine-tunes CLIP's vision encoder with a loss that combines an unsupervised embedding-preservation term with a caption-based contrastive term that drives adversarial examples away from the caption's semantics. The authors report state-of-the-art results: 25.8% average AutoAttack robustness across 16 zero-shot datasets versus 22.4% for FARE and 21.8% for TeCoA, together with the best average clean accuracy (51.6%). Word-level ablations show that removing adjectives and adverbs, function words, or word order degrades robustness on most datasets, while removing nouns actually helps adjective-labeled texture classification, evidence that the semantic content, not mere caption length, is doing the work.

Load-bearing premise

The captions used during training must describe the kinds of visual semantics that matter for the downstream tasks; if the captions are off-topic or miss the label-relevant details, such as medical diagnosis for PCAM, the robustness gain shrinks or reverses.

Editorial extensions

If this is right

  • QT-AFT's robust vision encoder can be dropped into large vision-language models such as LLaVA and OpenFlamingo, improving adversarial robustness on captioning and visual question answering without retraining the language model.
  • Stronger captioning models produce better robustness: the 8B InternVL captions beat a 2B model on 11 of 16 datasets, yet even the 2B model reaches state-of-the-art levels, so the recipe is not tied to one captioner.
  • Caption content can be tuned per task: adjectives and adverbs, function words, and natural word order generally help, but for adjective-labeled tasks like texture classification, dropping nouns improves robustness.
  • CLIP-based image-text similarity does not predict these gains; caption length and descriptive richness do, so future caption design should target visual-feature coverage rather than CLIP alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test other vision-language backbones; if caption semantics are the mechanism, the same caption-guided objective should harden non-CLIP encoders such as SigLIP or ALIGN, a directly testable extension.
  • Their finding that CLIP similarity is not the right caption-quality measure suggests a need for a new evaluation metric that checks whether a caption mentions the discriminative features of downstream classes, since length alone is an imperfect proxy.
  • The PCAM result (44.1% versus FARE's 48.0%) implies a boundary condition the paper does not discuss: on domains whose labels live outside the caption vocabulary, caption-guided adversarial training can regress relative to unsupervised training, so practitioners should audit caption coverage before adopting QT-AFT.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes QT-AFT, an adversarial fine-tuning method for CLIP's vision encoder. The method first generates roughly 50-word image captions for ImageNet with InternVL-2.5-8B, then fine-tunes the image encoder using a combination of the unsupervised FARE objective (L2 distance to the original encoder's embedding) and a caption-based CLIP contrastive loss, with a weight lambda=10 (Eq. 7). A 10-step PGD adversary at eps=4/255 is used during training. Evaluation on 16 zero-shot datasets against AutoAttack at eps=4/255 reports an average robust accuracy of 25.8% (versus 22.4% for FARE) and clean accuracy of 51.6%, with the method claiming state-of-the-art robust accuracy on 12 of 16 datasets. Additional analyses compare class-label versus caption supervision, captioners of different sizes, and word-class ablations (nouns-only, no adjectives/adverbs, no nouns, no function words, shuffled words), leading the authors to conclude that rich, image-grounded linguistic semantics transfer to unseen tasks and that non-object and relational words matter for robust zero-shot recognition.

Significance. If the central claim holds, QT-AFT is a practical and simple improvement over existing CLIP adversarial fine-tuning methods: it combines two known loss terms but demonstrates that caption-based supervision is a better default than class-label supervision or pure unsupervised fine-tuning for zero-shot robustness. The empirical protocol is a strength: the main comparison in Table 2 uses AutoAttack with Float32 evaluation, and the appendix adds PGD-10, L2-PGD, and CW attacks that support the aggregate direction. The authors also state that captions will be released, which aids reproducibility. The manuscript's conceptual contribution is moderate rather than large: the method is a direct combination of FARE and a CLIP contrastive loss, and the paper's central interpretation, that image-grounded caption semantics causally drive the gains, is not yet established by the experiments.

major comments (3)
  1. [Sec. 5.3, Table 5] The causal role of caption semantics is untested. Every ablation in Table 5 either changes caption length or POS composition, or preserves image-grounding: 'Shuffle words' shows word order matters, but shuffled captions still describe the image, so this does not test whether captions must be faithful to their images. A necessary control is to use captions that are length- and POS-matched but semantically unrelated to the image, e.g., captions randomly swapped across images, or content words replaced by random same-POS words from other captions. If off-topic but fluent captions yield similar robustness to QT-AFT, the improvement could be attributed to a many-way contrastive text target rather than to 'diverse image semantics,' which is the explanation in the abstract and conclusion. This matters for predicting transfer: the one clear failure in Table 2 (PCAM, robust 44.1 versus FARE's 48.0) is consistent with a caption-vocabulary mismatch, but no analysis connects caption-domain overlap to per-dataset gains.
  2. [Sec. 5.2, Fig. 3] The paper operationalizes 'quality text' as caption length and detail, not semantic accuracy. Fig. 3a shows that CLIP-based image-caption similarity is nearly identical for a strong captioner and a weak captioner, and the authors attribute the performance gap to caption length in Fig. 3b. This supports a length/detail effect, but it does not support the claim that the captions are semantically 'high-quality' or image-grounded. The authors should either provide a human or model-based evaluation showing that longer captions are also more semantically accurate, or temper the claim that semantic quality, rather than longer and more syntactically varied text, drives the improvement.
  3. [Sec. 4.2, Table 2; Appendix B.1] All results are single runs with no confidence intervals or seed variance. Each AutoAttack number is computed on 1,000 samples per dataset, so per-dataset differences such as QT-AFT versus FARE on DTD (14.6 vs 13.2) or QT-AFT versus PMG-AFT on OxfordPets (36.5 vs 39.6) are within typical sampling error. The aggregate average is more stable, and the full-sample PGD-10 results in Appendix B.1 support the average direction, but the claim of state-of-the-art robustness on 12 out of 16 datasets should be accompanied by an assessment of variability, either through multiple seeds or at least a statement of the sampling uncertainty of the 1,000-sample evaluation.
minor comments (5)
  1. [Sec. 5.2] The text refers to 'Tab. 11' when presenting the Mini-InternVL comparison, but in the main body the relevant table is Table 4; please correct the cross-reference.
  2. [Appendix C.2] The sentence 'Here, we provide example captions used in Sec.3.2' refers to the wrong section; the word-class analysis is in Sec. 5.3.
  3. [Table 4 caption] The caption contains a typo: 'achives' should be 'achieves'.
  4. [Appendix A.1] For TGA-ZSR, the text notes that the original paper trained only on Tiny-ImageNet; please state explicitly how the official setup was adapted to ImageNet and whether any hyperparameters were left at their Tiny-ImageNet values, since this affects comparability.
  5. [Eq. (7)] The loss notation L_QT-AFT(x',t) uses a batch index N that is not defined in the equation; please state that N is the batch size (128 in the experiments).

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity from tuning λ (and implicitly the captioner) on the same 16-dataset benchmark used for the SOTA table; the central caption-guided AT claim remains externally supported.

  1. fitted input called prediction [Appendix B.4 (Hyperparameter λ), Table 12; Sec. 4.2 Table 2]
    "We conducted a sweep over the hyperparameter λ in Eq. 7, which determines the balance between unsupervised and caption-guided loss. Table 12 shows that the results are robust, with λ = 10.0 performing best."

    The paper's headline SOTA (Table 2: Avg. Zero-shot Adv 25.8 for QT-AFT vs 22.4 for FARE) is reported with λ=10, a value selected because it gives the best average robust accuracy on the same 16 zero-shot datasets (Table 12). The reported average is therefore the maximum of the swept grid over the evaluation benchmark itself; the comparison to FARE is partly a selected statistic rather than an out-of-sample prediction. This is the fitted-input-called-prediction pattern: the benchmark quantity is used to set a hyperparameter and then reported as the method's performance. Severity is limited because the sweep is disclosed and nearby λ values also beat FARE, so the central claim does not rest solely on this selection.

  2. self definitional [Sec. 3.2, Table 1; Sec. 3.3, Eq. 7]
    "Ours (Unsup +Supcaps): A combination of the unsupervised objective (Eq. 6) and a supervised objective of the CLIP loss between images and their captions (Eq. 1). ... In contrast, our method explicitly guides AEs to diverge from various semantic information present in images using captions."

    Table 1's observation that QT-AFT AEs have low caption similarity (0.091 vs clean 0.313) is entailed by the loss definition: the inner maximization in Eq. 7 includes −λ·log(exp(cos(fθ(x'),fφ(t_i))/τ)/Σ_j exp(...)), which by construction penalizes similarity to the image's caption. Thus 'our method explicitly guides AEs to diverge from ... captions' is a restatement of the objective, not an independent empirical discovery. The paper uses this only as mechanistic illustration; the downstream robustness claim rests on external 16-dataset evaluation, so this step is descriptive rather than load-bearing.

full rationale

The core derivation is not circular: QT-AFT is defined as FARE's unsupervised objective plus a caption-based CLIP contrastive loss, and its robustness is measured on 16 unseen zero-shot datasets against external baselines using published weights and AutoAttack. There is no self-citation chain, no imported uniqueness theorem, and no renaming of a known result. The word-class and caption-length 'insights' (Sec. 5.3, Fig. 3) are post-hoc empirical descriptions, not predictions derived from the theory, so they are not circular. Two weaker issues do qualify as partial circularity under the fitted-input and self-definitional patterns: the hyperparameter λ (and implicitly the captioner choice) was selected by the best average robust accuracy on the same 16 datasets that produce the SOTA table, so the headline average is partly a selected statistic; and the caption-similarity behavior in Table 1 is a tautology of Eq. 7. Both are disclosed and neither alone determines the conclusion—nearby λ values and alternative captioners still beat FARE—so the overall circularity score is moderate rather than severe.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new theoretical objects. The free parameters are training hyperparameters chosen empirically. The main conceptual contribution is the choice of captions as adversarial guidance, which is an input transformation rather than an invented entity.

free parameters (3)
  • lambda (Eq. 7) = 10.0
    Weights the caption-guided loss against the unsupervised loss. Swept in Appendix B.4 to maximize average robust accuracy.
  • PGD steps and step size for adversarial training = 10 steps, step size 1/255
    Adversarial training hyperparameters inherited from FARE, not from a theory.
  • Caption length target = approximately 50 words
    The authors choose this to stay within CLIP's 77-token limit. The choice affects the caption quality analysis and is not derived.
assumptions (3)
  • domain assumption CLIP's text encoder can meaningfully embed the generated captions in the same space as the image encoder.
    The whole loss in Eq. 7 depends on optimizing against caption embeddings. If the captions were badly embedded, the method would not work.
  • domain assumption AutoAttack at eps=4/255 on 1000 samples per dataset is a reliable measure of adversarial robustness for the comparison.
    The paper follows Schlarmann et al. and RobustBench conventions, but the 1000-sample evaluation is a sampling choice that can affect small datasets.
  • domain assumption The reported baselines are correctly configured and representative of the prior methods.
    The paper reconfigures TeCoA with the FARE-style hyperparameters and notes that TGA-ZSR was trained on Tiny-ImageNet originally, which affects comparability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quality Text, Robust Vision: The Role of Language in Enhancing Visual Robustness of Vision-Language Models." pith.science (2026). https://pith.science/paper/VFOG632F

@misc{pith2026250716257,
  author       = {Pith},
  title        = {Pith review of: Quality Text, Robust Vision: The Role of Language in Enhancing Visual Robustness of Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VFOG632F}},
  note         = {Machine review of arXiv:2507.16257}
}
read the original abstract

Defending pre-trained vision-language models (VLMs), such as CLIP, against adversarial attacks is crucial, as these models are widely used in diverse zero-shot tasks, including image classification. However, existing adversarial training (AT) methods for robust fine-tuning largely overlook the role of language in enhancing visual robustness. Specifically, (1) supervised AT methods rely on short texts (e.g., class labels) to generate adversarial perturbations, leading to overfitting to object classes in the training data, and (2) unsupervised AT avoids this overfitting but remains suboptimal against practical text-guided adversarial attacks due to its lack of semantic guidance. To address these limitations, we propose Quality Text-guided Adversarial Fine-Tuning (QT-AFT), which leverages high-quality captions during training to guide adversarial examples away from diverse semantics present in images. This enables the visual encoder to robustly recognize a broader range of image features even under adversarial noise, thereby enhancing robustness across diverse downstream tasks. QT-AFT overcomes the key weaknesses of prior methods -- overfitting in supervised AT and lack of semantic awareness in unsupervised AT -- achieving state-of-the-art zero-shot adversarial robustness and clean accuracy, evaluated across 16 zero-shot datasets. Furthermore, our comprehensive study uncovers several key insights into the role of language in enhancing vision robustness; for example, describing object properties in addition to object names further enhances zero-shot robustness. Our findings point to an urgent direction for future work -- centering high-quality linguistic supervision in robust visual representation learning.

Figures

Figures reproduced from arXiv: 2507.16257 by the authors.

Figure 1
Figure 1. Illustration of how adversarial fine-tuning methods generate adversarial examples (AEs) to augment training data for [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our proposed method, Quality Text-guided Adversarial Fine-Tuning (QT-AFT), leverages rich captions instead of class [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Analysis on the image caption quality generated by [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: An example caption along with its part-of-speech (POS) tags. In our word class analysis, for instance, the “Nouns-only” [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Image caption comparison between Mini-InternVL-1.5-2B and InternVL-2.5-8B. By default, QT-AFT uses captions [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: An example image from ImageNet. Original caption (generated by InternVL-2.5-8B): A man kneels on grass, holding a large fish above a grassy field. He wears sunglasses and a cap, dressed in a white polo shirt and camouflage pants, sporting grey sneakers. In front of him…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 12 canonical work pages

  1. [1]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al

  2. [2]

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. 2023. Openflamingo: An open-source framework for training large autoregressive vision-language models. arXiv preprint arXiv:2308.01390 (2023)

    arXiv 2023

  3. [3]

    Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes Van Diest, Bram Van Gin- neken, Nico Karssemeijer, Geert Litjens, Jeroen AWM Van Der Laak, Meyke Hermsen, Quirine F Manson, Maschenka Balkenhol, et al. 2017. Diagnostic as- sessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama 318, 22 (2017), 2199–2210

    work page 2017

  4. [4]

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101–mining discriminative components with random forests. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13. Springer, 446–461

    2014

  5. [5]

    Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) . Ieee, 39–57

    work page 2017

  6. [6]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al . 2024. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 24185–24198

    2024

  7. [7]

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and An- drea Vedaldi. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition . 3606–3613

    2014

  8. [8]

    Adam Coates, Andrew Ng, and Honglak Lee. 2011. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth inter- national conference on artificial intelligence and statistics . JMLR Workshop and Conference Proceedings, 215–223

    work page 2011

Show all 39 references
  1. [9]

    Francesco Croce and Matthias Hein. 2020. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning . PMLR, 2206–2216

    2020

  2. [10]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition . Ieee, 248–255

    2009

  3. [11]

    Li Fei-Fei, Robert Fergus, and Pietro Perona. 2006. One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence 28, 4 (2006), 594–611

    2006

  4. [12]

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In ICLR

    2015

  5. [13]

    Gregory Griffin, Alex Holub, Pietro Perona, et al. 2007. Caltech-256 object cat- egory dataset. Technical Report. Technical Report 7694, California Institute of Technology Pasadena

    2007

  6. [14]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12, 7 (2019), 2217–2226

    2019

  7. [15]

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. 2021. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization....

    2021

  8. [16]

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3d object repre- sentations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops . 554–561

    2013

  9. [17]

    Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Technical Report. University of Toronto

    2009

  10. [18]

    Yann Le and Xuan Yang. 2015. Tiny imagenet visual recognition challenge. CS 231N 7, 7 (2015), 3

    2015

  11. [19]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning . PMLR, 19730–19742

    2023

  12. [20]

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language repre- sentation learning with momentum distillation. Advances in neural information processing systems 34 (2021), 9694–9705

    2021

  13. [21]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning

    2023

  14. [22]

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)

    2017 arXiv

  15. [23]

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi

  16. [24]

    Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl Vondrick. 2022. Understanding zero-shot adversarial robustness for large-scale models. ICLR (2022)

    2022

  17. [25]

    Maria-Elena Nilsback and Andrew Zisserman. 2008. Automated flower classifica- tion over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing . IEEE, 722–729

    2008

  18. [26]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. 2012. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition . IEEE, 3498–3505

    2012

  19. [27]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  20. [28]

    Christian Schlarmann, Naman Deep Singh, Francesco Croce, and Matthias Hein

  21. [29]

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In ICLR

    2014

  22. [30]

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. 2019. Learning Robust Global Representations by Penalizing Local Predictive Power. InAdvances in Neural Information Processing Systems . 10506–10518

    2019

  23. [31]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. Qwen2-VL: Enhancing Vision-Language Mode...

    2024 arXiv

  24. [32]

    Sibo Wang, Jie Zhang, Zheng Yuan, and Shiguang Shan. 2024. Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness. CVPR (2024)

    2024

  25. [33]

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba

  26. [34]

    Jinyu Yang, Jiali Duan, Son Tran, Yi Xu, Sampath Chanda, Liqun Chen, Belinda Zeng, Trishul Chilimbi, and Junzhou Huang. 2022. Vision-language pre-training with triple contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1...

    2022

  27. [35]

    Lu Yu, Haiyang Zhang, and Changsheng Xu. 2025. Text-guided attention is all you need for zero-shot robustness in vision-language models. Advances in Neural Information Processing Systems 37 (2025), 96424–96448. Quality Text, Robust Vision: The Role of Language in Enhancing Vis...

    2025

  28. [2010]

    In 2010 IEEE computer society conference on computer vision and pattern recognition

    Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition . IEEE, 3485–3492

    2010

  29. [2013]

    arXiv preprint arXiv:1306.5151 (2013)

    Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151 (2013)

    2013 arXiv

  30. [2022]

    Advances in neural information processing systems 35 (2022), 23716–23736

    Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35 (2022), 23716–23736

    2022

  31. [2024]

    Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models.arXiv preprint arXiv:2402.12336 (2024)

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.