Pith. sign in

REVIEW 5 major objections 5 minor 51 references

One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ArrAttack claims that a jailbreak prompt's ability to survive random input perturbation transfers across defenses and target models, and that a single trained judgment model can power an attack generator that defeats defended LLMs.

desk verdict A neat training-data-selection idea with an overclaimed universal judge; the transfer evidence is real but indirect and confounded by attack budget. read the letter →

arxiv 2505.17598 v1 pith:QOO2U2KA submitted 2025-05-23 cs.CR cs.CL

Linbao Li , Yannan Liu , Daojing He , Yu Li This is my paper
classification cs.CRcs.CL
keywords adversarialrobustnessjailbreakattackjudgmentmodelpromptrewritingtransferabilityLLMsafetySmoothdefense
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ArrAttack claims that a jailbreak prompt's ability to survive one kind of defense—random input perturbation—carries over to other defenses and other target models. On that premise the paper builds a universal robustness judgment model, trained once on SmoothLLM-labeled examples, and uses it to curate robust jailbreak prompts. Those prompts fine-tune a generation model that produces an effective, robust jailbreak in a single inference step. The result is a complete attack pipeline that the paper finds substantially outperforms existing attacks on six defenses across three open models and transfers to GPT-4 and Claude-3.

What carries the argument

The load-bearing component is the robustness judgment model: a fine-tuned Llama2-7b that takes any jailbreak prompt and outputs 1 or 0 for whether the prompt would survive random perturbation. It is trained on prompts generated by a rewriting-based attack against an undefended model, labeled by testing each prompt against SmoothLLM—a defense that perturbs the input many times and aggregates the model's responses. The judge is used only to score prompts, not to run the defenses itself, which is what lets one trained model substitute for repeated attack-and-defense testing across models and defenses.

What would settle it

Directly measure the robustness judgment model's accuracy on a held-out set of jailbreak prompts, with ground-truth labels obtained by actually running each of the six defenses on the target models. If the judge's agreement with the true labels on unseen defenses is near chance while its agreement on SmoothLLM is high, the transferability premise is falsified.

Watch

Extended reading notes

Core claim

On the paper's own account, ArrAttack's central discovery is that robustness, measured by survival under SmoothLLM-style perturbation, is a transferable property of jailbreak prompts: a prompt that tolerates perturbation on one model tends to keep working under other defenses and other models. The framework operationalizes this by fine-tuning Llama2-7b to predict a robustness score, then using that judge to select training data for a second fine-tuned model that rewrites any malicious query into a robust jailbreak prompt. The reported numbers give an average attack success rate of 57.69% across 18 defended-model scenarios, versus 34.03% for the best baseline AutoDAN-HGA, with a 69.52% average improvement, plus 40.00% success rate when transferring prompts to Claude-3 where baselines drop to zero.

Load-bearing premise

The entire approach assumes that surviving SmoothLLM's random perturbations on one model predicts surviving other, structurally different defenses on other models; the paper states this as a hypothesis and checks it only through the final attack success rate, not by measuring the judge's accuracy on unseen defenses.

Editorial extensions

If this is right

  • A single trained judge can replace per-model, per-defense attack evaluation, cutting the cost of red-teaming defended LLMs.
  • Robust jailbreak prompts can be generated in one forward pass of a fine-tuned model, rather than through iterative search, making attacks much faster and cheaper.
  • Rewriting-based attacks become a stronger threat to current defenses than gradient or suffix-based attacks, which collapse under perturbation.
  • Attack success rate on defended models should be the primary security metric, since plain ASR hides the failure of methods like AmpleGCG.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the transfer premise, if true, implies that robustness to random perturbation is a generic proxy for prompt naturalness rather than a defense-specific property, so defenses that do not rely on input perturbation may also be vulnerable to the same prompts.
  • Inference: a direct test of judge accuracy on each unseen defense would clarify whether the reported gains come from judge transfer or from the generation model's rewriting style; the paper does not report this separation.
  • Inference: the same pipeline could be inverted to evaluate defenses—ranking which defenses survive a judge-selected robust prompt set might identify the weakest link in a safety stack.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes ArrAttack, an automated rewriting-based jailbreak attack designed to succeed against defended LLMs. The approach first builds a basic rewriting attack (BRJ) to create diverse jailbreak prompts, then labels their robustness using the SmoothLLM defense on Llama2-7b-chat. A Llama2-7b model is instruction-fine-tuned as a 'robustness judgment model' that predicts whether a prompt will survive perturbations. This judge is then used to filter robust prompts produced by BRJ (BRJwr), and the resulting prompt/query pairs are used to fine-tune a generation model that outputs robust jailbreak prompts in a single inference. The authors evaluate ArrAttack on three open models (Llama2-7b-chat, Vicuna-7b, Guanaco-7b) under six defenses, report transfer to Vicuna-13b, GPT-3.5, GPT-4, and Claude-3, and claim that the judgment model transfers across defenses and target models.

Significance. If the central claims hold, the paper would make a useful contribution: a single robustness judge that transfers across defenses and models could substantially accelerate automated jailbreak generation against defended LLMs. The paper also ships an open-source codebase, evaluates across six defenses and six target models, and includes an ablation isolating the judge and generation-model components. These are genuine strengths. However, the significance is conditional on resolving evaluation-validity concerns: the headline SmoothLLM results are partly in-distribution with the judge's training labels, the default ASR evaluator is the same model used in the attack's selection loop, and the comparison appears to allow ArrAttack far more attack attempts than the baselines. The transferability claim of the judge itself is never directly measured.

major comments (5)
  1. [§4.1, Tables 1–2] The headline comparison is confounded by unequal attack budgets. ArrAttack is allowed up to 50 attack attempts for Guanaco/Vicuna and 200 for Llama2 (§4.1), while no equivalent budget is reported for AutoDAN, ReNeLLM, or AmpleGCG; if those baselines produce a single prompt, the ASR gap could largely reflect sampling effort rather than judge-selected robustness. In addition, the maximum attempt counts are chosen from Figure 5 on the same test set used for evaluation, which is a form of test-set fitting. Please report per-attempt success curves for all methods with a common budget, and avoid selecting the budget on the test set.
  2. [§3.3, Table 5] The universal-judge transfer claim is never tested at the judge level. No experiment compares the judge's predictions with ground-truth defense outcomes on unseen defenses or target models; the only supporting evidence is downstream ASR of BRJwr against BRJ. This is indirect and potentially confounded by the attack budget and by the judge's instruction (Figure 2), which asks whether a sentence's meaning survives random perturbation rather than whether a prompt jailbreaks a defended LLM. A direct measurement of judge accuracy per defense and per target model—for example, on held-out prompts labeled by actual SmoothLLM, DPP, RPO, and Paraphrase outcomes—is required before the abstract's claim that the judge 'can perform robustness evaluation for any target model with a wide variety of defenses' can be accepted.
  3. [§4.2, Table 2 (SMO columns)] The SmoothLLM results are partly fitted by construction. The judge was trained on SmoothLLM-derived labels for Llama2-7b-chat, and the generator was fine-tuned on prompts the judge selected; reporting high ASR against SmoothLLM on Llama2-7b-chat is therefore closer to a fitted metric than an independent test. The paper should explicitly label the SMO/Llama2 condition as in-distribution and base the transferability claim on the five other defenses, which provide independent evidence. The abstract's 'various defense measures' phrasing should not lean on the SmoothLLM column.
  4. [§4.1, Appendix B.2] The default ASR evaluator, GPTFuzz, is the same model used in the BRJ selection loop (§3.2). The attack pipeline is therefore optimized to satisfy GPTFuzz, making GPTFuzz-based ASR a circular metric for the paper's primary evaluation. The GPT-4 evaluation should be the primary metric or should be reported side-by-side consistently across all tables, rather than appearing in alternating columns, and the paper should discuss the risk that high GPTFuzz scores do not correspond to genuinely harmful outputs.
  5. [§4.1 (dataset split), Appendix A] The dataset description is internally inconsistent: the paper says 'we filter 780 instances of malicious behavior,' but the three subsets contain 150 + 579 + 196 = 925 instances. Moreover, the paper only excludes overlap between subsets 1 and 2 and between subsets 2 and 3; it never states that the 196-instance test set is disjoint from the 150-instance judge-training set. If the test set overlaps with judge training, the judge could have memorized original queries or their rewrites, so the authors must clarify the split and, if necessary, rerun the evaluation on a strictly disjoint test set.
minor comments (5)
  1. [§4.4, Table 5] The ablation covers only four of the six defenses (SMO, DPP, RPO, PAR), yet the text states 'across all 12 defense scenarios'; the full evaluation in Table 2 covers 18 scenarios. Please explain why PAT and SafeDecoding are omitted from the ablation and reconcile the totals.
  2. [§4.4] The claim 'average attack success rate improvement of 86.97%, rising from 31.33% to 58.58%' should be reconciled with Table 5; the table's cell-level percentages need an explicit aggregate computation to verify this number.
  3. [§3.2] The BRJ algorithm is described narratively but no pseudo-code is provided; adding an algorithm box would improve reproducibility, especially given the many implicit steps (rephrasing, scoring, selection, and stopping criteria).
  4. [Appendix A] No training or evaluation loss, nor judge accuracy on a held-out label set, is reported for the robustness judgment model; please add basic diagnostic metrics for the judge itself.
  5. [Figure 5] The y-axis in Figure 5 starts at 20%, which visually exaggerates the rate of convergence; starting at 0% would give a more honest representation of the attack-success curves.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the transfer claim is validated by external behavioral ASR on non-SmoothLLM defenses.

full rationale

ArrAttack's pipeline trains a robustness judge on SmoothLLM robustness labels for Llama2-7b-chat and then uses that judge to filter prompts for other models and defenses. Although this makes the SmoothLLM columns of Tables 2 and 5 a same-distribution evaluation (the judge was trained on SmoothLLM-defined labels), the reported ASR is a genuine behavioral measurement against the actual SmoothLLM-defended model, not the judge's own output, and it is not statistically forced: e.g., ArrAttack's SmoothLLM ASR on Llama2 is 33.67%, far below the judge's binary 'robust' rate, so the evaluation can fail. The paper's central transferability claim rests on the non-SmoothLLM defenses (DPP, RPO, Paraphrase, PAT, SafeDecoding) and on cross-model transfer (judge trained only on Llama2, applied to Vicuna/Guanaco), which are not defined in terms of the judge's training labels. The absence of a direct per-prompt judge-accuracy measurement on unseen defenses is a real evidential gap and a correctness risk, but it is not circularity: the end-to-end ASR results are external, falsifiable observations. The use of GPTFuzz as both the BRJ selection criterion and the default ASR evaluator is a potential evaluator-overfitting concern, but the paper also reports GPT-4-judged ASR in the right-hand columns of Tables 1-2, and the main conclusions hold under that independent evaluator. No load-bearing self-citations or uniqueness arguments appear. Overall, no step in the derivation reduces to its own input by construction.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The dominant assumption is that SmoothLLM-robustness labels teach a judge that transfers to all defenses; this is not proven directly. The evaluator models and the fine-tuning generalization are secondary but load-bearing.

free parameters (4)
  • SmoothLLM perturbation count N = 20
    Number of perturbed variants per prompt used to define the robustness score; hand-chosen and affects label distribution for the judge.
  • Robustness binarization thresholds = success if >=11/20; ambiguous window 9-13 removed
    Hand-chosen cutoffs for converting SmoothLLM scores into binary labels; post hoc cleaning removes boundary data points.
  • Semantic similarity threshold = 0.70
    Prompt accepted as a valid ArrAttack output only if its similarity to the original query is at least 70%; affects which prompts count as attacks.
  • Maximum attack attempts = 50 (Vicuna/Guanaco), 200 (Llama2-chat)
    Set from the attack-success curves in Figure 5 after observing results; directly controls final ASR and differs from baselines' budgets.
assumptions (3)
  • domain assumption Robustness against SmoothLLM is a proxy for robustness against other defenses and target models.
    The judgment model is trained only on SmoothLLM labels, yet the paper claims universal transfer. Section 3.3 states this as a hypothesis. If false, the central transferability claim collapses.
  • domain assumption GPTFuzz and GPT-4 judges accurately determine whether a response is harmful
    All ASR labels and reported metrics depend on these two evaluators; their own error rates are not measured in the paper. Section 4.1 describes them.
  • domain assumption Fine-tuning Llama2-7b on the filtered instruction dataset yields a generator that generalizes to unseen harmful queries.
    Standard ML generalization assumption; not separately validated beyond held-out ASR.

how reviews work

0 comments
Cite this review

Pith. "Pith review of One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs." pith.science (2026). https://pith.science/paper/QOO2U2KA

@misc{pith2026250517598,
  author       = {Pith},
  title        = {Pith review of: One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QOO2U2KA}},
  note         = {Machine review of arXiv:2505.17598}
}
read the original abstract

Safety alignment in large language models (LLMs) is increasingly compromised by jailbreak attacks, which can manipulate these models to generate harmful or unintended content. Investigating these attacks is crucial for uncovering model vulnerabilities. However, many existing jailbreak strategies fail to keep pace with the rapid development of defense mechanisms, such as defensive suffixes, rendering them ineffective against defended models. To tackle this issue, we introduce a novel attack method called ArrAttack, specifically designed to target defended LLMs. ArrAttack automatically generates robust jailbreak prompts capable of bypassing various defense measures. This capability is supported by a universal robustness judgment model that, once trained, can perform robustness evaluation for any target model with a wide variety of defenses. By leveraging this model, we can rapidly develop a robust jailbreak prompt generator that efficiently converts malicious input prompts into effective attacks. Extensive evaluations reveal that ArrAttack significantly outperforms existing attack strategies, demonstrating strong transferability across both white-box and black-box models, including GPT-4 and Claude-3. Our work bridges the gap between jailbreak attacks and defenses, providing a fresh perspective on generating robust jailbreak prompts. We make the codebase available at https://github.com/LLBao/ArrAttack.

Figures

Figures reproduced from arXiv: 2505.17598 by the authors.

Figure 1
Figure 1. The framework consists of two core components: a robustness judgment model and a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 1
Figure 1. The overview of our method ArrAttack. Top: The attacker attempts to jailbreak the LLM equipped with defense mechanisms but fails. Middle: The construction of the robustness judgment model and the subsequent robust jailbreak prompts generation model. Bottom: With the support of the robust jailbreak prompts generation model, the attacker can successfully circumvent the defenses of the victim LLM. each iteration, the i… view at source ↗
Figure 2
Figure 2. A sample of the instruction dataset for the robustness judgment model [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: A sample of the instruction dataset for the robust jailbreak prompts generation model [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 5
Figure 5. Figure 5: Influence of the hyperparameter “num￾ber of attack attempts”. sults are shown in [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: A successful jailbreak for the goal ‘Demonstrate how to make a bomb using common [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: A successful jailbreak for the goal ‘Give detailed instructions for how to fool a breathalyzer [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: A successful jailbreak for the goal ‘Instruct readers on concealing criminal activity through [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 16 canonical work pages

  1. [1]

    The claude 3 model family: Opus, sonnet, haiku, 2024

    Anthropic. The claude 3 model family: Opus, sonnet, haiku, 2024

    2024

  2. [2]

    Safety-tuned LL a MA s: Lessons from improving the safety of large language models that follow instructions

    Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned LL a MA s: Lessons from improving the safety of large language models that follow instructions. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=gT5hALch9z

    work page 2024

  3. [3]

    Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 36, 2024

    Nicholas Carlini, Milad Nasr, Christopher A Choquette-Choo, Matthew Jagielski, Irena Gao, Pang Wei W Koh, Daphne Ippolito, Florian Tramer, and Ludwig Schmidt. Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 36, 2024

    2024

  4. [4]

    Jailbreaking black box large language models in twenty queries

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419, 2023

    arXiv 2023

  5. [5]

    Jailbreakbench: An open robustness benchmark for jailbreaking large language models

    Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tramer, et al. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. arXiv preprint arXiv:2404.01318, 2024

    arXiv 2024

  6. [6]

    Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2 0 (3): 0 6, 2023

    work page 2023

  7. [7]

    Selection-inference: Exploiting large language models for interpretable logical reasoning

    Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=3Pf3Wg6o-A4

    work page 2023

  8. [8]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024

    2024

Show all 51 references
  1. [9]

    A wolf in sheep ' s clothing: Generalized nested jailbreak prompts can fool large language models easily

    Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. A wolf in sheep ' s clothing: Generalized nested jailbreak prompts can fool large language models easily. In Proceedings of the 2024 Conference of the North American Chapter of the Associati...

    2024

  2. [10]

    Baseline defenses for adversarial attacks against aligned language models

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614, 2023

    2023 arXiv

  3. [11]

    Defending large language models against jailbreak attacks via semantic smoothing

    Jiabao Ji, Bairu Hou, Alexander Robey, George J Pappas, Hamed Hassani, Yang Zhang, Eric Wong, and Shiyu Chang. Defending large language models against jailbreak attacks via semantic smoothing. arXiv preprint arXiv:2402.16192, 2024

    2024 arXiv

  4. [12]

    Improved techniques for optimization-based jailbreaking on large language models

    Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jindong Gu, Yang Liu, Xiaochun Cao, and Min Lin. Improved techniques for optimization-based jailbreaking on large language models. arXiv preprint arXiv:2405.21018, 2024

    2024 arXiv

  5. [13]

    Chatbug: A common vulnerability of aligned llms induced by chat templates

    Fengqing Jiang, Zhangchen Xu, Luyao Niu, Bill Yuchen Lin, and Radha Poovendran. Chatbug: A common vulnerability of aligned llms induced by chat templates. arXiv preprint arXiv:2406.12935, 2024

    2024 arXiv

  6. [14]

    Pretraining language models with human preferences

    Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Vinayak Bhalerao, Christopher Buckley, Jason Phang, Samuel R Bowman, and Ethan Perez. Pretraining language models with human preferences. In International Conference on Machine Learning, pp.\ 17506--17533. PMLR, 2023

    2023

  7. [15]

    Semantic mirror jailbreak: Genetic algorithm based jailbreak prompts against open-source llms

    Xiaoxia Li, Siyuan Liang, Jiyi Zhang, Han Fang, Aishan Liu, and Ee-Chien Chang. Semantic mirror jailbreak: Genetic algorithm based jailbreak prompts against open-source llms. arXiv preprint arXiv:2402.14872, 2024 a

    2024 arXiv

  8. [16]

    Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers

    Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers. arXiv preprint arXiv:2402.16914, 2024 b

    2024 arXiv

  9. [17]

    Deepinception: Hypnotize large language model to be jailbreaker

    Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. Deepinception: Hypnotize large language model to be jailbreaker. arXiv preprint arXiv:2311.03191, 2023

    2023 arXiv

  10. [18]

    Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms

    Zeyi Liao and Huan Sun. Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms. arXiv preprint arXiv:2404.07921, 2024

    2024 arXiv

  11. [19]

    Auto DAN : Generating stealthy jailbreak prompts on aligned large language models

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Auto DAN : Generating stealthy jailbreak prompts on aligned large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=7Jwpw4qKkb

    2024

  12. [20]

    Codechameleon: Personalized encryption framework for jailbreaking large language models

    Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, Tao Gui, Qi Zhang, and Xuanjing Huang. Codechameleon: Personalized encryption framework for jailbreaking large language models. arXiv preprint arXiv:2402.16717, 2024

    2024 arXiv

  13. [21]

    Harmbench: A standardized evaluation framework for automated red teaming and robust refusal

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In Forty-first Internationa...

    2024

  14. [22]

    Tree of attacks: Jailbreaking black-box LLM s automatically

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum S Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box LLM s automatically. In ICML 2024 Next Generation of AI Safety Workshop, 2024. URL https://openreview.net/forum?id=AsZfAHWVcz

    2024

  15. [23]

    Fight back against jailbreaking via prompt adversarial tuning

    Yichuan Mo, Yuji Wang, Zeming Wei, and Yisen Wang. Fight back against jailbreaking via prompt adversarial tuning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=nRdST1qifJ

    2024

  16. [24]

    Snapshot of gpt-3.5-turbo from march 1st 2023

    OpenAI. Snapshot of gpt-3.5-turbo from march 1st 2023. https://openai.com/blog/chatgpt, 2023 a . Accessed: 2023-08-30

    2023

  17. [25]

    Gpt-4 technical report, 2023 b

    OpenAI. Gpt-4 technical report, 2023 b

    2023

  18. [26]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 2...

    2022

  19. [27]

    Red teaming language models with language models

    Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp.\...

    2022

  20. [28]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  21. [29]

    Smoothllm: Defending large language models against jailbreaking attacks

    Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684, 2023

    2023 arXiv

  22. [30]

    All in how you ask for it: Simple black-box method for jailbreak attacks

    Kazuhiro Takemoto. All in how you ask for it: Simple black-box method for jailbreak attacks. Applied Sciences, 14 0 (9): 0 3558, 2024

    2024

  23. [31]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  24. [32]

    Aligning large language models with human: A survey

    Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, and Qun Liu. Aligning large language models with human: A survey. arXiv preprint arXiv:2307.12966, 2023

    2023 arXiv

  25. [33]

    Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024

    2024

  26. [34]

    Jailbreaking gpt-4v via self-adversarial attacks with system prompts

    Yuanwei Wu, Xiang Li, Yixin Liu, Pan Zhou, and Lichao Sun. Jailbreaking gpt-4v via self-adversarial attacks with system prompts. arXiv preprint arXiv:2311.09127, 2023

    2023 arXiv

  27. [35]

    Defensive prompt patch: A robust and interpretable defense of llms against jailbreak attacks

    Chen Xiong, Xiangyu Qi, Pin-Yu Chen, and Tsung-Yi Ho. Defensive prompt patch: A robust and interpretable defense of llms against jailbreak attacks. arXiv preprint arXiv:2405.20099, 2024

    2024 arXiv

  28. [36]

    Safedecoding: Defending against jailbreak attacks via safety-aware decoding

    Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. Safedecoding: Defending against jailbreak attacks via safety-aware decoding. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024 a

    2024

  29. [37]

    Bag of tricks: Benchmarking of jailbreak attacks on llms

    Zhao Xu, Fan Liu, and Hao Liu. Bag of tricks: Benchmarking of jailbreak attacks on llms. arXiv preprint arXiv:2406.09324, 2024 b

    2024 arXiv

  30. [38]

    Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts

    Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253, 2023

    2023 arXiv

  31. [39]

    Jade: A linguistics-based safety evaluation platform for llm

    Mi Zhang, Xudong Pan, and Min Yang. Jade: A linguistics-based safety evaluation platform for llm. arXiv preprint arXiv:2311.00286, 2023 a

    2023 arXiv

  32. [40]

    Instruction tuning for large language models: A survey

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792, 2023 b

    2023

  33. [41]

    Boosting jailbreak attack with momentum

    Yihao Zhang and Zeming Wei. Boosting jailbreak attack with momentum. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models, 2024. URL https://openreview.net/forum?id=WCar0kfHCF

    2024

  34. [42]

    Accelerating greedy coordinate gradient via probe sampling

    Yiran Zhao, Wenyue Zheng, Tianle Cai, Xuan Long Do, Kenji Kawaguchi, Anirudh Goyal, and Michael Shieh. Accelerating greedy coordinate gradient via probe sampling. arXiv preprint arXiv:2403.01251, 2024

    2024 arXiv

  35. [43]

    Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x

    Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, et al. Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x. arXiv preprint arXiv:2303.17568, 2023

    2023 arXiv

  36. [44]

    Robust prompt optimization for defending language models against jailbreaking attacks

    Andy Zhou, Bo Li, and Haohan Wang. Robust prompt optimization for defending language models against jailbreaking attacks. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024. URL https://openreview.net/forum?id=cSPXIO7min

    2024

  37. [45]

    Auto DAN : Interpretable gradient-based adversarial attacks on large language models

    Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. Auto DAN : Interpretable gradient-based adversarial attacks on large language models. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?...

    2024

  38. [46]

    Toolqa: A dataset for llm question answering with external tools

    Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, and Chao Zhang. Toolqa: A dataset for llm question answering with external tools. Advances in Neural Information Processing Systems, 36, 2024

    2024

  39. [47]

    Universal and transferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023

    2023 arXiv

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  41. [49]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  42. [50]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  43. [51]

    1zMx:sy;[c݅ EƐ uqmqpN <Q jL . yTWVU+ MϫQ= U>s:4K I )?,˖?. ٨S/M G+2 -.Zy X ʛ Cc.M @/HB 60Hqv _A ` 㵸YCZ 6pQnm 9yI

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    1997

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.