Pith. sign in

REVIEW 8 cited by

An LLM can Fool Itself: A Prompt-Based Adversarial Attack

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.13345 v1 pith:ZU7WD7IF submitted 2023-10-20 cs.CR

Xilie Xu , Keyi Kong , Ning Liu , Lizhen Cui , Di Wang , Jingfeng Zhang , Mohan Kankanhalli This is my paper · ORCID
classification cs.CR
keywords adversarialattackpromptattackoriginalsamplefoolitselfadvglue
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The wide-ranging applications of large language models (LLMs), especially in safety-critical domains, necessitate the proper evaluation of the LLM's adversarial robustness. This paper proposes an efficient tool to audit the LLM's adversarial robustness via a prompt-based adversarial attack (PromptAttack). PromptAttack converts adversarial textual attacks into an attack prompt that can cause the victim LLM to output the adversarial sample to fool itself. The attack prompt is composed of three important components: (1) original input (OI) including the original sample and its ground-truth label, (2) attack objective (AO) illustrating a task description of generating a new sample that can fool itself without changing the semantic meaning, and (3) attack guidance (AG) containing the perturbation instructions to guide the LLM on how to complete the task by perturbing the original sample at character, word, and sentence levels, respectively. Besides, we use a fidelity filter to ensure that PromptAttack maintains the original semantic meanings of the adversarial examples. Further, we enhance the attack power of PromptAttack by ensembling adversarial examples at different perturbation levels. Comprehensive empirical results using Llama2 and GPT-3.5 validate that PromptAttack consistently yields a much higher attack success rate compared to AdvGLUE and AdvGLUE++. Interesting findings include that a simple emoji can easily mislead GPT-3.5 to make wrong predictions.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 13 citations worldwide. Full citation record

  1. CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    A black-box poisoning framework, CPA-RAG, generates fluent fake documents that steer retrieval-augmented language models toward attacker-chosen wrong answers, achieving over 90% success in the reported experiments.

  2. SMAB: MAB based word Sensitivity Estimation Framework and its Applications in Adversarial Text Generation

    cs.CL 2025-02 conditional novelty 6.0 of 10

    SMAB uses multi-armed bandit sampling and masked-language-model replacements to estimate word-level sensitivity of text classifiers, and applies it to accuracy prediction and adversarial text generation.

  3. SATORI: Static Test Oracle Generation for REST APIs

    cs.SE 2025-08 unverdicted novelty 5.0 of 10

    SATORI statically infers REST API test oracles from OpenAPI specs via LLMs, reporting F1 74.3%, above AGORA+'s 69.3%, with 18 confirmed bugs; the supplied full text, however, is a different paper.

  4. Robustness of Prompting: Enhancing Robustness of Large Language Models Against Prompting Attacks

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A two-stage automatic prompting pipeline (error correction followed by guidance) reduces the accuracy loss large language models suffer when input questions contain typos, reordered words, or irrelevant extra information.

  5. Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images

    cs.AI 2025-06 reject novelty 4.0 of 10

    SHE lowers behavioral hallucination scores by about 10 percent by detecting low visual-textual similarity and projecting out the hallucinated direction in embedding space.

  6. Stable Vision Concept Transformers for Medical Diagnosis

    cs.CV 2025-06 reject novelty 4.0 of 10

    A vision transformer with a concept bottleneck and denoised diffusion smoothing is claimed to give stable concept explanations under input perturbations while keeping diagnostic accuracy.

  7. AI Agent Behavioral Science

    q-bio.NC 2025-06 conditional novelty 4.0 of 10

    AI agents should be studied as behavioral entities shaped by context and interaction, not only as trained models.

  8. A comprehensive taxonomy of hallucinations in Large Language Models

    cs.CL 2025-08 conditional novelty 2.0 of 10

    A survey that organizes LLM hallucination types, causes, benchmarks, and mitigations, and restates the theorem that hallucination is inevitable for computable LLMs.

Pith tools