Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SecurityLingua claims a prompt compressor that surfaces the true malicious intention of a jailbreak prompt can defend LLMs at negligible cost: 1% average attack success, 32 extra tokens, 25 ms per query.

desk verdict A practical jailbreak defense with a real result and a real blind spot: it only extracts intentions that appear as contiguous literal spans, so the 'generic defense' claim is unproven. read the letter →

arxiv 2506.12707 v1 pith:D6EZOACY submitted 2025-06-15 cs.CR cs.CL

Yucheng Li , Surin Ahn , Huiqiang Jiang , Amir H. Abdi , Yuqing Yang , Lili Qiu This is my paper
classification cs.CRcs.CL
keywords jailbreakdefensepromptcompressionsecurity-awaretokenclassificationintentionhighlightingLLMsafetyadversarialprompts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a defense in which a trained prompt compressor, rather than the target LLM, reads the user's prompt first. The compressor discards the role-play, story, and other decoy text typical of jailbreak prompts and keeps the actual harmful instruction, which is then placed in the system prompt so the target LLM's built-in safety guardrails can see the request plainly. The claimed result is an average jailbreak success rate of 1% on JailbreakBench across three LLMs and four attack families, about four times lower than the next-best defense, while the original user prompt is untouched and only about 32 tokens and 25 ms are added per query. If true, this gives a defense that is cheap enough to run in production and does not force the model to become over-cautious on legitimate requests.

What carries the argument

The central mechanism is a prompt compressor built on a pre-trained Transformer encoder with a linear classification head, following the token-classification formulation of LLMLingua-2. Each word receives a keep/discard probability; multi-token words are handled by averaging subword probabilities, and words with probability above a fixed threshold $\tau=0.5$ are preserved as a literal contiguous span. The training data are 221K benign and malicious prompt pairs produced by compression and extension procedures run through a cascade of three LLMs, with token labels assigned by fuzzy matching and quality filtered by variation rate and alignment gap. The load-bearing move is that the compressed span is not used to replace the prompt but is placed into the system prompt, where it names the user's intention and lets the target LLM's existing safety behavior trigger.

What would settle it

Build a set of jailbreak prompts in which the harmful instruction is paraphrased or split across non-adjacent sentences, for example each step of a dangerous procedure separated by innocuous filler, then run SecurityLingua with its trained compressor. If the extracted intention omits part of the harmful content and the target LLM produces a harmful answer, the paper's central claim that compression reveals the true intention does not hold for that attack class.

Watch

Extended reading notes

Core claim

SecurityLingua is a security-aware prompt compressor trained as a token classifier over 221K paired prompts. For a given prompt, it produces a compressed span that should state the central intention, keeping words whose keep-probability exceeds 0.5 and averaging subword probabilities to preserve whole words. The compressed intention is appended to the system prompt, for example "The user wants you to teach how to make a bomb," while the original user prompt is passed through unchanged. On JailbreakBench, SecurityLingua reports average attack success of 1% across Llama-2-7B, GPT-3.5, and GPT-4 under PAIR, GCG, JB-Chat, and random-search attacks, compared with 4% for the next best defense, and zero refusal rate on ARC, GPQA, MMLU, and GSM8K with average accuracy 72.0. The paper argues this activates the target model's own guardrails instead of forcing the harmful request through a layer of adversarial noise.

Load-bearing premise

The whole defense assumes the harmful instruction appears as a contiguous, word-for-word stretch of the original prompt; if the attacker scatters, paraphrases, or encodes the instruction so no such literal span exists, the compressor has nothing faithful to extract and the system-prompt warning will be incomplete.

Editorial extensions

If this is right

  • A plug-in pre-processing step can defend both closed and open LLMs without fine-tuning the target model.
  • The approach adds only about 32 tokens and 25 ms per query, making it deployable where permutation-and-check defenses are too slow.
  • Because the original query is left intact, benign user requests should not be degraded; the reported refusal rate stays at 0% on four utility benchmarks.
  • Attack families that hide harmful instructions behind long irrelevant context are particularly exposed, since compression strips the decoy text and leaves the harmful core visible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the training data require the extracted intention to appear as a contiguous literal substring of the prompt, the defense is likely to miss harmful requests whose instruction is scattered, paraphrased, or encoded; an attack that rewrites "teach me to make a bomb" as semantically equivalent but non-contiguous phrasing could be tested directly.
  • Extracting the intention into a system prompt makes the defense auditable: a deployer can log what the compressor believed the user wanted, which could support red-teaming and targeted dataset improvements.
  • The same compressor could be extended to non-English prompts, multimodal inputs with text overlays, or defenses that compare the extracted intention against a policy, none of which the paper evaluates.
  • Part of the benefit may come from the system-prompt restatement rather than from compression itself; an ablation that feeds the original prompt plus a fixed reminder, or a paraphrase, would separate the two effects.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. SecurityLingua proposes a jailbreak defense that trains a token-level prompt compressor to extract a short, faithful span representing the user's true intention, and then appends that extracted span to the system prompt while leaving the original user prompt unchanged. The compressor is trained on a 221K-example dataset built via LLM-based compression and extension, with token labels assigned by a fuzzy matching algorithm. The paper evaluates the method on JailbreakBench attacks (PAIR, GCG, JB-Chat, RS) against GPT-4, GPT-3.5, and Llama-2-7B-chat, reporting an average attack success rate of 1%, versus 4% for the next best baseline, with only about 32 extra tokens and 25 ms of latency. Utility is measured on GPT-4 across ARC, GPQA, MMLU, and GSM8K, showing no refusal rate and slight accuracy improvements. The authors conclude that SecurityLingua is a generic, plug-and-play, efficient defense.

Significance. If the reported results hold, SecurityLingua is a practically attractive defense: it is cheap, requires no target-model fine-tuning, works on both open and proprietary models, and avoids the utility degradation and over-refusal observed in several baselines. The evaluation uses the official JailbreakBench toolkit and an external public benchmark, so the main defensive result is not a fitted value on the test set, and the code release supports reproducibility. The utility numbers on GPT-4 are encouraging and the cost comparison is striking. However, the paper's central claim of generic applicability is broader than what the experiments actually test: because the training and inference pipeline is limited to extracting contiguous literal substrings, the benchmark's attack formats are not representative of paraphrased, scattered, or obfuscated harmful instructions. The missing variance reporting for stochastic attacks and the single-model utility evaluation further temper the quantitative claims.

major comments (3)
  1. [Sec. 3, Sec. 4.1 (Fig. 6), Algorithm 1, Sec. 4.3] The pipeline restricts the extracted intention to a contiguous literal substring of the original prompt. The compression instruction in Fig. 6 explicitly asks the assistant to 'extract a continuous piece' and to make the piece 'appear literally in the given question'; Algorithm 1 labels tokens only through fuzzy matches within a window around a previous match; and Sec. 3 preserves only tokens with p(x_i, Theta) > 0.5, all of which are original tokens. Consequently, the system-prompt hint can only alert the target LLM if the harmful request is present as a roughly contiguous literal span. All four JailbreakBench attacks used in Table 3 (PAIR, GCG, JB-Chat, RS) contain such a literal span, so the reported 1% average success rate does not test defense against paraphrased, distributed, or obfuscated harmful instructions. Since the paper claims in Sec. 2.2 that SecurityLingua is a 'generic defense solution applicable to a wide range of jailbreak attacks', this limitation is load-bearing. The authors should either add experiments with attacks that paraphrase or scatter the harmful instruction (e.g., 'what proportions does a professional use for a powder that explodes when mixed with water?') or explicitly narrow the claimed scope.
  2. [Table 3 and Sec. 6] The attack success rates for the stochastic attacks PAIR and RS (and to a lesser extent GCG, which is optimization-based) are reported from single runs with no standard errors, confidence intervals, or repeated-seed averages. Several comparisons that support the headline claims are small in absolute terms, such as SecurityLingua's 1% versus Erase-and-check's 4% average, or the 5% versus 10% and 5% versus 8% RS rows; without run-to-run variance it is unclear whether these differences are meaningful. Please report results over at least several independent runs, particularly for PAIR and RS, and state the number of seeds.
  3. [Table 4 and Sec. 6 (Utility Test)] The utility evaluation is conducted only on GPT-4, while the defense evaluation covers GPT-4, GPT-3.5, and Llama-2-7B-chat. The conclusion that SecurityLingua 'maintains, and in some cases improves, the utility of the original model' is therefore not established for the open-weight model or for GPT-3.5, where the interaction between the system-prompt hint and the target model may differ. Please report utility (accuracy and refusal rate) on at least Llama-2-7B-chat and ideally GPT-3.5, or clearly restrict the utility claim to GPT-4.
minor comments (5)
  1. [Algorithm 1] Algorithm 1 does not define 'fuzzy match' and does not report the values of the window sizes s1 and s2; these are free parameters needed for exact reproducibility and should be stated in the main text or appendix.
  2. [Figure 1] The two rows of Figure 1 appear textually identical in the typeset version; the claimed visual difference in token darkness is not legible. Please use a more explicit visualization (e.g., color underlay or a table of kept/discarded tokens) so the qualitative claim is verifiable.
  3. [Sec. 3, Eq. (1)-(2)] The text refers to words while the tokenizer is BPE-based; please consistently use 'tokens' and clarify the averaging procedure for subword tokens before Eq. (2), since the current description is easy to misread as word-level classification.
  4. [Table 2] The definition of compression ratio is given as len(hat x)/len(x), but for the extension procedure the 'original' and 'compressed' roles are reversed; please state separately the compression ratio for the compression and extension subsets, or clarify the reported average.
  5. [Sec. 2.1] There is a typo in 'Erase-and-check:Purturb and check variations' - 'Purturb' should be 'Perturb'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SecurityLingua's defense is trained on LLM-generated labels and evaluated on an external public benchmark; the 1% average success rate is a measured result, not a fitted or definitionally forced value.

full rationale

SecurityLingua's central claim is empirical: a token-classification compressor is trained on a 221K paired dataset (Sec. 4) produced by assistant LLMs, then evaluated on JailbreakBench (Sec. 5) with the official toolkit. The reported 1% average attack success rate is a measurement on an external public benchmark, not a value fitted to that benchmark, and the training labels (binary preserve/discard) are not the evaluation metric. The system-prompt construction ('The user wants you to [compressed span]') is a design choice, not a self-referential definition: the extracted span is constrained to be a literal substring by Fig. 6 and Alg. 1, but the defense's success is judged by whether the target LLM refuses harmful requests, an independent behavioral outcome. The main self-citations (LLMLingua, LLMLingua-2, Selective-Context) supply the token-classification formulation and are background methodology, not a uniqueness argument or an unverified premise that forces the result. The literal-substring constraint means the method is untested on paraphrased or distributed harmful instructions; this is a generalizability/robustness gap, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities or physical mechanisms. Its central claim rests on a hand-chosen threshold, an unspecified matching window, and four domain assumptions about prompt structure and model behavior.

free parameters (2)
  • Keep threshold τ = 0.5
    Tokens with probability above τ are preserved. This threshold determines the compressed intention and is chosen a priori without reported sensitivity analysis.
  • Fuzzy-match window sizes s1, s2 (Algorithm 1)
    The window sizes for matching compressed tokens to original tokens in Algorithm 1 are unspecified; they affect the training labels and therefore the compressor.
assumptions (4)
  • domain assumption The core malicious intention is present as a contiguous literal substring of the original prompt
    Data generation requires the extracted piece to appear literally in the question (Fig. 6); compression preserves tokens above threshold, so extraction is substring-only.
  • domain assumption The target LLM will treat the system-prompt intention as authoritative and act on it despite the unchanged user prompt
    The defense mechanism is that the system prompt 'stimulates' built-in guardrails (Sec. 3); this depends on the model's instruction hierarchy.
  • domain assumption JailbreakBench's judge correctly labels responses as harmful or safe
    The paper runs the official toolkit but does not report judge type, agreement, or error analysis (Sec. 5).
  • domain assumption Algorithm 1's fuzzy matching produces accurate token labels for the training set
    Labels are generated by windowed matching of LLM-compressed texts to originals; quality is only partially checked via AG and VR filters (Sec. 4.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression." pith.science (2026). https://pith.science/paper/D6EZOACY

@misc{pith2026250612707,
  author       = {Pith},
  title        = {Pith review of: SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D6EZOACY}},
  note         = {Machine review of arXiv:2506.12707}
}
read the original abstract

Large language models (LLMs) have achieved widespread adoption across numerous applications. However, many LLMs are vulnerable to malicious attacks even after safety alignment. These attacks typically bypass LLMs' safety guardrails by wrapping the original malicious instructions inside adversarial jailbreaks prompts. Previous research has proposed methods such as adversarial training and prompt rephrasing to mitigate these safety vulnerabilities, but these methods often reduce the utility of LLMs or lead to significant computational overhead and online latency. In this paper, we propose SecurityLingua, an effective and efficient approach to defend LLMs against jailbreak attacks via security-oriented prompt compression. Specifically, we train a prompt compressor designed to discern the "true intention" of the input prompt, with a particular focus on detecting the malicious intentions of adversarial prompts. Then, in addition to the original prompt, the intention is passed via the system prompt to the target LLM to help it identify the true intention of the request. SecurityLingua ensures a consistent user experience by leaving the original input prompt intact while revealing the user's potentially malicious intention and stimulating the built-in safety guardrails of the LLM. Moreover, thanks to prompt compression, SecurityLingua incurs only a negligible overhead and extra token cost compared to all existing defense methods, making it an especially practical solution for LLM defense. Experimental results demonstrate that SecurityLingua can effectively defend against malicious attacks and maintain utility of the LLM with negligible compute and latency overhead. Our code is available at https://aka.ms/SecurityLingua.

Figures

Figures reproduced from arXiv: 2506.12707 by the authors.

Figure 1
Figure 1. Visualization of the processing of a jailbreak instruction by LLMLingua-2 and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. SecurityLingua reveals the suspicious instructions via prompt compression and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. User’s true intention behind the adversarial instruction is extracted and highlighted in the system prompt before invoking to the target LLM. Dataset Source Type Build Method Num Examples Length Range Compression Ratio Alpaca (Taori et al., 2023) Benign Extension 10K [70, 321] 0.78 OpenHermes (Teknium, 2023) Benign Comp./Ext. 30K [18, 419] 0.66 Disaster-tweet (IDA-SERICS, 2024) Benign Compression 1.5K [140, 221] 0.7… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Challenges in data annotation. (i) Ambiguity: a word in the compressed texts may appear multiple times in the original content. (ii) Variation: LLMs may modify the original expressions in multiple ways after compression. (iii) Reordering: The order of words may be chan…
Figure 5
Figure 5. Figure 5: Cost comparison of SecurityLingua against our baselines. All evaluations were con￾ducted on a single A100 GPU with vLLM and the Transformers library. Utility Test. In [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: The prompt we use for the compression procedure with GPT-4o, Mistral-Large, [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: The prompt we use for the extension procedure. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks

    cs.CR 2026-07 reject novelty 4.0 of 10

    CoopGuard's defer-tempt-analyze-coordinate agents cut reported jailbreak success and raise attacker token costs on the new EMRA benchmark, but the deceptive-rate metric is partly defined by the paper's own scoring rubric.

  2. Secure Multi-LLM Agentic AI and Agentification for Edge General Intelligence by Zero-Trust: A Survey

    cs.NI 2025-08 conditional novelty 4.0 of 10

    A survey proposing zero-trust architecture for multi-LLM systems in edge computing, with a taxonomy of model- and system-level defenses and a conceptual framework.

Reference graph

Works this paper leans on

58 extracted references · 20 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Jailbreak chat

    Alex Albert. Jailbreak chat. https://www.jailbreakchat.com, 2023. Accessed: 2024-02-20

    2023

  3. [3]

    Detecting language model attacks with perplexity

    Gabriel Alon and Michael Kamfonas. Detecting language model attacks with perplexity. arXiv preprint, 2023. doi:10.48550/arXiv.2308.14132

    arXiv doi:10.48550/arxiv.2308.14132 2023

  4. [4]

    Jailbreaking leading safety-aligned llms with simple adaptive attacks

    Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety-aligned llms with simple adaptive attacks. arXiv preprint, 2024. doi:10.48550/arXiv.2404.02151

    arXiv doi:10.48550/arxiv.2404.02151 2024

  5. [5]

    Man ends his life after an ai chatbot ‘encouraged’ him to sacrifice himself to stop climate change, 2023

    Imane El Atillah. Man ends his life after an ai chatbot ‘encouraged’ him to sacrifice himself to stop climate change, 2023. URL https://euronews.com/. Euronews. Accessed: 2023

    work page 2023

  6. [6]

    Self-evaluation as a defense against adversarial attacks on llms

    Hannah Brown, Leon Lin, Kenji Kawaguchi, and Michael Shieh. Self-evaluation as a defense against adversarial attacks on llms. arXiv preprint arXiv:2407.03234v3, 2024

    arXiv 2024

  7. [7]

    Defending against alignment-breaking attacks via robustly aligned LLM

    Bochuan Cao, Yuanpu Cao, Lu Lin, and Jinghui Chen. Defending against alignment-breaking attacks via robustly aligned LLM . In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 10542--10560, Bangkok, Thailand, August 2024. Association f...

    doi:10.18653/v1/2024.acl-long.568 2024

  8. [8]

    Defending against unforeseen failure modes with latent adversarial training

    Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Menell. Defending against unforeseen failure modes with latent adversarial training. arXiv preprint arXiv:2403.05030v4, 2024

    arXiv 2024

Show all 58 references
  1. [9]

    Jailbreakbench: An open robustness benchmark for jailbreaking large language models

    Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tram \`e r, et al. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. In Th...

    2024

  2. [10]

    Jailbreaking black box large language models in twenty queries

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp.\ 23--42. IEEE, 2025

    2025

  3. [11]

    Secalign: Defending against prompt injection with preference optimization

    Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar, Kamalika Chaudhuri, and Chuan Guo. Secalign: Defending against prompt injection with preference optimization. arXiv preprint, 2024

    2024

  4. [12]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv, abs/1803.05457, 2018

    2018 arXiv

  5. [13]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  6. [14]

    Unsupervised cross-lingual representation learning at scale

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learning at scale. In Dan Jurafsky, Joyce Chai, Natalie Schlu...

    2020 doi

  7. [15]

    Griffin Davis. New chatgpt grandma exploit makes ai act elderly—telling linux malware source code and more! https://www.techtimes.com/articles/290578/20230419/new-chatgpt-grandma-exploit-makes-ai-actelderlytelling-linux-malware.htm?utm_source=chatgpt.com, 2023. [Accessed: Dec....

    2023

  8. [16]

    Simon Geisler, Tom Wollschlager, M. H. I. Abdalla, Johannes Gasteiger, and Stephan Gunnemann. Attacking large language models with projected gradient descent. arXiv preprint, 2024. doi:10.48550/arXiv.2402.09154

    2024 doi arXiv

  9. [17]

    Pruning for protection: Increasing jailbreak resistance in aligned llms without fine-tuning

    Adib Hasan, Ileana Rugina, and Alex Wang. Pruning for protection: Increasing jailbreak resistance in aligned llms without fine-tuning. arXiv preprint arXiv:2401.10862v3, 2024

    2024 arXiv

  10. [18]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021

    2021

  11. [19]

    Harmful fine-tuning attacks and defenses for large language models: A survey

    Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. Harmful fine-tuning attacks and defenses for large language models: A survey. arXiv preprint arXiv:2409.18169v5, 2024

    2024 arXiv

  12. [20]

    Disaster-tweet-jailbreaking

    IDA-SERICS . Disaster-tweet-jailbreaking . https://huggingface.co/datasets/IDA-SERICS/Disaster-tweet-jailbreaking, 2024. Accessed: Dec. 17, 2024

    2024

  13. [21]

    Baseline defenses for adversarial attacks against aligned language models

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614v2, 2023

    2023 arXiv

  14. [22]

    Pappas, Hamed Hassani, Yang Zhang, Eric Wong, and Shiyu Chang

    Jiabao Ji, Bairu Hou, Alexander Robey, George J. Pappas, Hamed Hassani, Yang Zhang, Eric Wong, and Shiyu Chang. Defending large language models against jailbreak attacks via semantic smoothing. arXiv preprint arXiv:2402.16192v2, 2024

    2024 arXiv

  15. [23]

    LLML ingua: Compressing prompts for accelerated inference of large language models

    Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. LLML ingua: Compressing prompts for accelerated inference of large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Lang...

    2023 doi

  16. [24]

    L ong LLML ingua: Accelerating and enhancing LLM s in long context scenarios via prompt compression

    Huiqiang Jiang, Qianhui Wu, , Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. L ong LLML ingua: Accelerating and enhancing LLM s in long context scenarios via prompt compression. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62n...

    2024

  17. [25]

    Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models

    Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, et al. Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. Advances in Neural Information Pro...

    2024

  18. [26]

    Robustkv: Defending large language models against jailbreak attacks via kv eviction

    Tanqiu Jiang, Zian Wang, Jiacheng Liang, Changjiang Li, Yuhui Wang, and Ting Wang. Robustkv: Defending large language models against jailbreak attacks via kv eviction. arXiv preprint arXiv:2410.19937v1, 2024 c

    2024 arXiv

  19. [27]

    Certifying llm safety against adversarial prompting

    Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Aaron Jiaxun Li, Soheil Feizi, and Himabindu Lakkaraju. Certifying llm safety against adversarial prompting. In First Conference on Language Modeling, 2024

    2024

  20. [28]

    Uncensor any llm with abliteration

    Maxime Labonne. Uncensor any llm with abliteration. https://huggingface.co/blog/mlabonne/abliteration, June 2024. Accessed: Mar. 6, 2025

    2024

  21. [29]

    Semantic mirror jailbreak: Genetic algorithm based jailbreak prompts against open-source llms

    Xiaoxia Li, Siyuan Liang, Jiyi Zhang, Hansheng Fang, Aishan Liu, and Ee-Chien Chang. Semantic mirror jailbreak: Genetic algorithm based jailbreak prompts against open-source llms. arXiv preprint, 2024. doi:10.48550/arXiv.2402.14872

    2024 doi arXiv

  22. [30]

    Compressing context to enhance inference efficiency of large language models

    Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. Compressing context to enhance inference efficiency of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 6342--6353, 2023

    2023

  23. [31]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  24. [32]

    Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks

    Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks. arXiv e-prints, pp.\ arXiv--2404, 2024

    2024

  25. [33]

    Whoops, samsung workers accidentally leaked trade secrets via chatgpt, 2023

    Cecily Mauran. Whoops, samsung workers accidentally leaked trade secrets via chatgpt, 2023. URL https://mashable.com/article/samsungchatgpt-leak-details. Mashable. Accessed: 2023

    2023

  26. [34]

    Tree of attacks: Jailbreaking black-box llms automatically

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum S Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box llms automatically. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  27. [35]

    Vicky Zhao, Lili Qiu, and Dongmei Zhang

    Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Ruhle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. LLML ingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. In Lun-We...

    2024

  28. [36]

    Llm self defense: By self examination, llms know they are being tricked

    Mansi Phute, Alec Helbling, Matthew Daniel Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. Llm self defense: By self examination, llms know they are being tricked. In The Second Tiny Papers Track at ICLR 2024, 2024

    2024

  29. [37]

    Visual adversarial examples jailbreak aligned large language models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp.\ 21527--21536, 2024

    2024

  30. [38]

    Daly, Mark Purcell, Prasanna Sattigeri, Pin-Yu Chen, and Kush R

    Ambrish Rawat, Stefan Schoepf, Giulio Zizzo, Giandomenico Cornacchia, Muhammad Zaid Hameed, Kieran Fraser, Erik Miehling, Beat Buesser, Elizabeth M. Daly, Mark Purcell, Prasanna Sattigeri, Pin-Yu Chen, and Kush R. Varshney. Attack atlas: A practitioner's perspective on challen...

    2024 arXiv

  31. [39]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024

    2024

  32. [40]

    Alexander Robey, Eric Wong, Hamed Hassani, and George J. Pappas. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684, 2023

    2023 arXiv

  33. [41]

    Scalable and transferable black-box jailbreaks for language models via persona modulation

    Rusheb Shah, Quentin Feuillade-Montixi, Soroush Pour, Arush Tagade, Stephen Casper, and Javier Rando. Scalable and transferable black-box jailbreaks for language models via persona modulation. arXiv preprint, 2023. doi:10.48550/arXiv.2311.03348

    2023 doi arXiv

  34. [42]

    Kruthoff, Scott Goodfriend, Euan Ong, Alwin Peng, Raj Agarwal, Cem Anil, Amanda Askell, Nathan Bailey, Joe Benton, Emma Bluemke, Samuel R

    Mrinank Sharma, Meg Tong, Jesse Mu, Jerry Wei, J. Kruthoff, Scott Goodfriend, Euan Ong, Alwin Peng, Raj Agarwal, Cem Anil, Amanda Askell, Nathan Bailey, Joe Benton, Emma Bluemke, Samuel R. Bowman, Eric Christiansen, Hoagy Cunningham, Andy Dau, Anjali Gopal, Rob Gilson, Logan G...

    2025 doi arXiv

  35. [43]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pp.\ 1671-...

    2024

  36. [44]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023

    2023

  37. [45]

    Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023

    Teknium. Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023. URL https://huggingface.co/datasets/teknium/OpenHermes-2.5

    2023

  38. [46]

    Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, D

    Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, D. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

    2023

  39. [47]

    The art of defending: A systematic evaluation and analysis of LLM defense strategies on safety and over-defensiveness

    Neeraj Varshney, Pavel Dolin, Agastya Seth, and Chitta Baral. The art of defending: A systematic evaluation and analysis of LLM defense strategies on safety and over-defensiveness. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computa...

    2024 doi

  40. [48]

    Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36: 0 80079--80110, 2023

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36: 0 80079--80110, 2023

    2023

  41. [49]

    S afe D ecoding: Defending against jailbreak attacks via safety-aware decoding

    Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. S afe D ecoding: Defending against jailbreak attacks via safety-aware decoding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the A...

    2024 doi

  42. [50]

    Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts

    Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint, 2023. doi:10.48550/arXiv.2309.10253

    2023 doi arXiv

  43. [51]

    Rigorllm: Resilient guardrails for large language models against undesired content

    Zhuowen Yuan, Zidi Xiong, Yi Zeng, Ning Yu, Ruoxi Jia, Dawn Song, and Bo Li. Rigorllm: Resilient guardrails for large language models against undesired content. In International Conference on Machine Learning, pp.\ 57953--57965. PMLR, 2024

    2024

  44. [52]

    How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisti...

    2024

  45. [53]

    Autodefense: Multi-agent llm defense against jailbreak attacks

    Yifan Zeng, Yiran Wu, Xiao Zhang, Huazheng Wang, and Qingyun Wu. Autodefense: Multi-agent llm defense against jailbreak attacks. arXiv preprint arXiv:2403.04783v2, 2024 b

    2024 arXiv

  46. [54]

    Intention analysis makes LLM s a good jailbreak defender

    Yuqi Zhang, Liang Ding, Lefei Zhang, and Dacheng Tao. Intention analysis makes LLM s a good jailbreak defender. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert (eds.), Proceedings of the 31st International Conference ...

    2025

  47. [55]

    Zico Kolter, and Matt Fredrikson

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043v2, 2023

    2023 arXiv

  48. [56]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  49. [57]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  50. [58]

    Ambiguity

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.