Pith. sign in

REVIEW 16 cited by

PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.04528 v5 pith:DHA4TDJ7 submitted 2023-06-07 cs.CL cs.CRcs.LG

Kaijie Zhu , Jindong Wang , Jiaheng Zhou , Zichen Wang , Hao Chen , Yidong Wang , Linyi Yang , Wei Ye
show 3 more authors
Yue Zhang Neil Zhenqiang Gong Xing Xie
This is my paper · ORCID
classification cs.CLcs.CRcs.LG
keywords promptsadversarialrobustnessanalysislanguagellmsacrosscomprehensive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The increasing reliance on Large Language Models (LLMs) across academia and industry necessitates a comprehensive understanding of their robustness to prompts. In response to this vital need, we introduce PromptRobust, a robustness benchmark designed to measure LLMs' resilience to adversarial prompts. This study uses a plethora of adversarial textual attacks targeting prompts across multiple levels: character, word, sentence, and semantic. The adversarial prompts, crafted to mimic plausible user errors like typos or synonyms, aim to evaluate how slight deviations can affect LLM outcomes while maintaining semantic integrity. These prompts are then employed in diverse tasks including sentiment analysis, natural language inference, reading comprehension, machine translation, and math problem-solving. Our study generates 4,788 adversarial prompts, meticulously evaluated over 8 tasks and 13 datasets. Our findings demonstrate that contemporary LLMs are not robust to adversarial prompts. Furthermore, we present a comprehensive analysis to understand the mystery behind prompt robustness and its transferability. We then offer insightful robustness analysis and pragmatic recommendations for prompt composition, beneficial to both researchers and everyday users.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 50 citations worldwide. Full citation record

  1. Imprompt: A Language Framework for Prompt Programming

    cs.AI 2026-07 conditional novelty 7.0 of 10

    Prompt programs can be typed like programs and compiled in different ways; Imprompt formalizes two such compilers and tests them on tax-law reasoning and data-labeling tasks.

  2. Visual Grounding in Zero-Shot Vision-Language Control

    cs.RO 2026-08 conditional novelty 6.0 of 10

    Input-ablation tests show most current VLMs are not visually grounded controllers, though a small symmetry-consensus ensemble works as a hazard monitor.

  3. Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks

    cs.SE 2025-07 conditional novelty 6.0 of 10

    Open-source 7B LLMs frequently produce requested C vulnerabilities when explicitly prompted, but the reported rates exclude most model outputs and the claimed persona effects are inconsistent.

  4. A Representation Level Analysis of NMT Model Robustness to Grammatical Errors

    cs.CL 2025-05 conditional novelty 6.0 of 10

    NMT encoders detect grammatical errors in early layers and move the error's representation toward the clean form in later layers; fine-tuning on noisy text increases reliance on the attention heads that do this work.

  5. Format Sensitivity Index: Token-Controlled Prompt Wrapper Robustness and Schema Compliance in LLM Benchmarking

    cs.AI 2026-05 conditional novelty 5.5 of 10

    Across 140k generations, wrapper-only formatting changes accuracy by over 30× across models, largely via parseability failures, so single-wrapper scores are statistically fragile.

  6. Automated Framework to Evaluate and Harden LLM System Instructions against Encoding Attacks

    cs.CR 2026-04 conditional novelty 5.0 of 10

    Refusal-aligned LLMs leak system instructions under encoding/serialization prompts at high rates, and one-shot CoT instruction reshaping substantially reduces that leakage without retraining.

  7. Agent Identity Evals: Measuring Agentic Identity

    cs.AI 2025-07 conditional novelty 5.0 of 10

    Introduces Agent Identity Evals (AIE), five similarity-based metrics for LMA identity stability, with pilot experiments showing identifiability always at zero and no statistical support.

  8. Investigating the Robustness of Retrieval-Augmented Generation at the Query Level

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Retrieval-augmented generation performance drops noticeably under minor query perturbations, with end-to-end results often tracking retriever behavior.

  9. Evaluating LLMs Robustness in Less Resourced Languages with Proxy Models

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Important words chosen by a small proxy model, when perturbed with typos or spacing errors, push Bielik, Mistral-7B, and Llama-3.1-8B to wrong answers on Polish classification tasks more often than random edits.

  10. Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignment

    cs.CV 2025-05 conditional novelty 5.0 of 10

    FOA-Attack aligns global and clustered local features via optimal transport with dynamic ensemble weighting to create targeted adversarial images that transfer to closed-source multimodal LLMs.

  11. Find Before You Fine-Tune: A Diagnostic Study of Small LLMs for Cybersecurity QA

    cs.CL 2026-07 conditional novelty 4.0 of 10

    Pre-fine-tuning scores on a three-task diagnostic can predict the direction of post-fine-tuning change in small LLMs for cybersecurity QA, but not the magnitude or rank-preservation, which is regime-dependent.

  12. Look Within or Look Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning

    cs.LG 2025-05 reject novelty 4.0 of 10

    The paper claims PEFT is a strict, less robust, lower-capacity subset of full fine-tuning, but the mathematical proofs contain load-bearing errors and the experiments, while suggestive, cannot repair them.

  13. When Generic Prompt Improvements Hurt: Evaluation-Driven Iteration for LLM Applications

    cs.CL 2026-01 conditional novelty 3.0 of 10

    Adding generic prompt rules to task-specific LLM prompts is not monotonic: in 15-20 case local suites, Llama 3 and Qwen 2.5 sometimes pass fewer extraction and RAG checks, so prompt changes should be tested per task.

  14. Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey categorizing prompt-based attacks on LLMs into four classes and proposing aspirational goals of un-distillable, un-finetunable, and un-editable models.

  15. Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions

    cs.CL 2025-06 conditional novelty 3.0 of 10

    LLM robustness research is organized into adversarial robustness, out-of-distribution robustness, and evaluation, with an accompanying GitHub collection of papers.

  16. The Science of Evaluating Foundation Models

    cs.CL 2025-02 conditional novelty 3.0 of 10

    A survey-and-checklist proposal that organizes LLM evaluation into an ABCD framework (Algorithm, Big Data, Computation, Domain Expertise) for context-aware, documented assessment.

Pith tools