Pith. sign in

REVIEW 4 cited by

Reading Isn't Believing: Adversarial Attacks On Multi-Modal Neurons

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2103.10480 v1 pith:GGTXQJ6L submitted 2021-03-18 cs.LG cs.CLcs.CV

David A. Noever , Samantha E. Miller Noever This is my paper · ORCID
classification cs.LGcs.CLcs.CV
keywords modelreadingvisualadversarialattacksbelievingclipfalse
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

With Open AI's publishing of their CLIP model (Contrastive Language-Image Pre-training), multi-modal neural networks now provide accessible models that combine reading with visual recognition. Their network offers novel ways to probe its dual abilities to read text while classifying visual objects. This paper demonstrates several new categories of adversarial attacks, spanning basic typographical, conceptual, and iconographic inputs generated to fool the model into making false or absurd classifications. We demonstrate that contradictory text and image signals can confuse the model into choosing false (visual) options. Like previous authors, we show by example that the CLIP model tends to read first, look later, a phenomenon we describe as reading isn't believing.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Attacking Attention of Foundation Models Disrupts Downstream Tasks

    cs.CR 2025-06 conditional novelty 5.0 of 10

    A task-agnostic attack that perturbs attention and embeddings of CLIP/ViT backbones degrades classification, retrieval, captioning, segmentation, and depth estimation without using labels or text.

  2. Typographic Attacks in a Multi-Image Setting

    cs.CR 2025-02 conditional novelty 5.0 of 10

    In a non-repeating multi-image setting, choosing typographic attack words by CLIP text-image similarity beats random matching on ImageNet and is claimed to transfer to InstructBLIP.

  3. Contrastive Spectral Rectification: Test-Time Defense towards Zero-shot Adversarial Robustness of CLIP

    cs.CV 2026-01 conditional novelty 4.0 of 10

    CSR detects and repairs adversarial CLIP inputs by comparing features with a low-pass filtered copy and applying a small contrastive PGD correction, claiming SOTA robust accuracy on 16 benchmarks.

  4. `Do as I say not as I do': A Semi-Automated Approach for Jailbreak Prompt Attack against Multimodal LLMs

    cs.CR 2025-02 reject novelty 4.0 of 10

    A voice jailbreak that buries a forbidden question between benign prompts reportedly succeeds against Gemini 67 to 93 percent of the time, but the metric comes from the target model judging itself and is not reliable.

Pith tools