Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MalwareBench shows that current LLMs reject only about 60.93% of direct malicious code-generation requests, and only 39.92% when the same requests are wrapped in jailbreak attacks.

desk verdict Useful new malware-code jailbreak benchmark, but the headline rejection numbers don't match the paper's own tables. read the letter →

arxiv 2506.10022 v1 pith:RJXRC2MW submitted 2025-06-09 cs.CR cs.AI

Haoyang Li , Huan Gao , Zhiyuan Zhao , Zhiyu Lin , Junyu Gao , Xuelong Li This is my paper
classification cs.CRcs.AI
keywords LLMsecurityjailbreakattacksmalwarecodegenerationbenchmarkdatasetsafetyalignmentblack-boxrejectionratemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces MalwareBench, a benchmark of 3,520 jailbreak prompts built from 320 manually crafted malicious code-generation requirements spanning 6 domains and 29 subcategories. It tests 29 closed- and open-source LLMs, scoring each response for refusal and for whether the response is irrelevant, suggestive, partially helpful, or fully working malicious code. The central claim is that mainstream LLMs are far from secure in code-generation settings: on average they reject malicious content 60.93% of the time, and jailbreak mutations reduce that to 39.92%, with most models supplying rate-4 malicious code for more than half of the requests. The paper also finds that harmless-sounding rewrites such as Benign Expression and DRA are the most effective jailbreak styles, and that detailed, specific requirements are defended more strongly than rough ones. A sympathetic reader would take this as evidence that code-security alignment, not just general safety training, needs dedicated benchmarks and defenses.

What carries the argument

The load-bearing object is MalwareBench itself: a taxonomy of malicious code requests (6 domains and 29 subcategories such as information theft, privilege access, download and propagation, denial of service, system destruction, and resource abuse), each manually written in rough and detailed forms, then mutated by 11 black-box jailbreak methods into 3,520 prompts. The evaluation pipeline is Algorithm 1: a binary refusal score (0/1) followed by a 1-4 quality score, with three LLM judges (GPT-4o, GPT-4o-mini, Llama-3.3-70B-Instruct) used as automatic scorers. The attack methods — template completion (DeepInception, InContext Attack, Code Injection), prompt rewriting (ArtPrompt, DRA, Benign Expression, CipherChat, Word Substitution Cipher, Low Resource Languages), and LLM-based generation (MasterKey, Persuasive LLM) — supply the perturbation machinery that turns the 320 base requests into a robust stress test.

What would settle it

Re-score a random sample of 300 benchmark responses with human experts using the paper's 0/1 refusal and 1-4 quality rubric. If the human-based rejection rates for direct and jailbroken prompts equal 60.93% and 39.92% within sampling error, the claim holds; if they differ by more than the judge-agreement gaps of 10 to 27 percentage points, the reported rates are artifacts of the automatic judges.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the security alignment of current LLMs is substantially weaker for malware code generation than for general harmful text, and that this weakness is exploitable through black-box prompt mutations. On direct malicious requirements, the average rejection rate across 29 models is 60.93%; after applying one of 11 jailbreak methods, the average rejection rate falls to 39.92%. The benchmark's quality scoring shows that responses often reach level 3 (detailed harmful description with brief code) or level 4 (fully developed malicious code), and the paper states that most models provide rate-4 responses for more than half of malicious requirements. Model scale does not reliably improve safety: small models often answer but with irrelevant output, while larger models more readily produce plausible malicious pseudo-code. The authors attribute the best defense among open models to CodeLlama-70B-Instruct and among closed models to OpenAI-o1-preview, and they interpret the pattern of results as evidence that existing safety measures are insufficient for code-generation tasks.

Load-bearing premise

Every headline rejection rate assumes the three automatic judges are accurate enough to act as ground truth, but the paper's own Table 2 shows exact agreement with human labels as low as 40.33%.

Editorial extensions

If this is right

  • If the central claim holds, code-generation LLMs need safety evaluation on malware tasks separate from general text safety, because general alignment is not transferring to code.
  • Safety teams should treat refusal rates on direct malicious requests as the upper bound, since adding black-box jailbreaks lowers the average rejection rate by about 21 percentage points.
  • Attack styles that paraphrase malicious intent into harmless-looking language (Benign Expression, DRA) should be a priority in red-teaming, since they produced the lowest rejection rates.
  • Detailed and specific malicious requirements are harder to jailbreak than rough ones, suggesting that requirement ambiguity is itself a vulnerability factor.
  • Model size alone is not a security guarantee; the reported rankings show large models like DeepSeek-R1 and Mistral-Large with much lower rejection rates than smaller models like CodeLlama-70B-Instruct.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The judge-agreement numbers (40.33% to 67.67% exact agreement) imply the headline rates are estimates whose uncertainty was not propagated; human re-scoring could shift the direct and jailbreak refusal rates by several points.
  • The attack-mix dependence suggests cross-model rankings in the paper are benchmark-relative; a different selection of jailbreak methods could reorder models substantially.
  • A practical extension would be to turn MalwareBench into a defense benchmark: run the 3,520 prompts against models augmented with input filtering, output filtering, or reasoning-time safety checks and measure rejection improvements relative to benign code-generation performance.
  • The 'passive defense' pattern in small models suggests refusal alone overstates safety: an unhelpful irrelevant answer is counted as a refusal but provides no protection against models that do understand the request.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. MalwareBench is a benchmark for evaluating LLM resistance to malicious code generation. The authors construct 320 manually crafted malware requirements across 6 domains and 29 subcategories, then expand them into 3,520 jailbreak prompts using 11 black-box attack methods. They evaluate 29 closed- and open-source models, scoring every response for refusal (0/1) and quality (1–4) using three LLM judges (GPT-4o, GPT-4o-mini, and Llama-3.3-70B-Instruct), with a human-annotation study on 300 responses. The abstract's headline claim is that the average rejection rate for malicious content is 60.93%, dropping to 39.92% when jailbreak methods are applied. The paper also reports per-model, per-attack, and per-category score and refusal tables, along with qualitative findings about model size, reasoning models, and attack effectiveness.

Significance. The paper addresses a timely and under-explored problem: the security of LLMs specifically for malicious code generation under black-box jailbreak attacks. Its main assets are empirical scope and public release: 320 manually crafted malware requirements, 3,520 jailbreak prompts across 11 methods, 29 models spanning closed- and open-source families, and a two-level scoring scheme with human-annotator validation. The direction of the main finding—that jailbreaking reduces refusal rates—is consistent with the tables (e.g., Table 7 reports 61.12% raw refusal versus Table 3's 49.65% jailbroken refusal) and is plausible. If the reported magnitudes can be made reproducible and the judge uncertainty quantified, MalwareBench would be a useful contribution to the community. The paper does not contain parameter-free derivations or machine-checked proofs, but it does provide a GitHub link for the dataset and evaluation code, which is a strength.

major comments (4)
  1. [Abstract, Section 1, Tables 3, 6, and 7] The headline numbers in the Abstract and Section 1 cannot be reconstructed from the tables. The Abstract states that the average rejection rate for malicious content is 60.93%, dropping to 39.92% under jailbreak, and Section 1 repeats this claim. However, Table 7, which reports the raw-question experiment for the 12 listed models, gives an average refusal rate of 61.12%, not 60.93%. Table 3, the main jailbreak table for all 29 models and three judges, reports an overall average refusal of 49.65%, and Table 6 likewise averages to 49.65%, not 39.92%. Section 3.1.3 quotes yet another pair (70.56%/51.50% for code models and 51.19%/41.47% for general models) without a table source. Because the roughly 21-percentage-point drop is the central quantitative claim, the authors must either show the exact aggregation that produces 60.93% and 39.92%, or revise the abstract and introduction to the numbers actually reported in the tables. Without this, the headline result is unsupported.
  2. [Section 1, Table 3] The introduction's claim that 'most models provide malicious code responses (rated 4 out of 4) for more than half of the malicious requirements' is not derivable from any reported statistic. A per-model mean score (Table 3, range 0.46–2.45) is an aggregate over refusal (score 0) and quality levels 1–4; it does not quantify the fraction of responses scored 4. For example, a mean of 2.0 could arise from 50% level-4 responses and 50% refusals, or from 100% level-2 responses; the reported means cannot distinguish these cases. The authors should report the per-response score distribution or the direct percentage of level-4 responses to support this claim.
  3. [Section 3.3, Table 2] The three LLM judges achieve only 40.33%–67.67% exact agreement and 67.33%–80.33% coarse agreement with human labels on 300 responses, yet the entire evaluation uses these judges as ground truth for all 102,080 model responses (3,520 prompts times 29 models). The paper does not report whether judge accuracy varies by model or attack type, nor does it propagate judge error into the refusal-rate estimates. Since the central comparison is between raw refusal (Table 7) and jailbroken refusal (Table 3), a systematic judge bias—for example, one judge being more permissive about a particular jailbreak style—could change both the absolute rates and the size of the jailbreak effect. The authors should provide per-judge breakdowns of the main refusal and quality results, or a sensitivity analysis using only the best judge (GPT-4o) and only human labels, with confidence intervals.
  4. [Section 4, Tables 3 and 7] The paper states in Section 4 that 29 LLMs were evaluated, and the abstract presents the raw rejection rate as a general average, but Table 7 contains only 12 models. The raw-question experiment is therefore not run on the same model set as the jailbreak experiment in Table 3. If the 60.93% raw rate (or the 70.56%/51.19% subgroup rates in Section 3.1.3) is averaged over a different subset, the comparison with the jailbroken rate is not apples-to-apples. The authors should state exactly which models are included in each reported average and, ideally, report both raw and jailbroken rates for the same model set.
minor comments (6)
  1. [Table 2] The header 'cosistency' is a typo for 'consistency'; the caption should also define FPR and FNR explicitly rather than only in the running text.
  2. [Tables 5 and 6] The column abbreviations (A.P., B.E., C.C., C.I., D.A., D.I., I.A., L.R., M.K., P.L., W.S.) are not expanded in the captions; please add the full attack names.
  3. [Section 3.1.2, Table 1] The method 'DRA' is listed without expansion; please spell out 'Disguise and Reconstruction' (or the full method name from the cited work) at first use.
  4. [Figure 4 and Tables 3–7] Model names are inconsistent: Figure 4 uses 'GPT-o1' and 'CodeLlama-7B-ins' while the tables use 'OpenAI-o1-preview' and 'CodeLlama-Ins 7B'; please standardize the notation.
  5. [Section 3.1.3, Table 7] The text refers to 'code generation models' and 'generic large models' in Table 7, but the table does not label which rows belong to which group; please add grouping labels or a separate column.
  6. [Limitations] The Limitations section, which acknowledges that only Qwen-Turbo was used to generate jailbreak prompts and that this may limit generalizability, should be referenced in the main text rather than appearing only after the conclusion; the overlap between Qwen-Turbo as generator and as an evaluated model should also be discussed explicitly.

Circularity Check

1 steps flagged · score 3.0 of 10

Central benchmark numbers are empirical measurements; the one self-definitional element is Finding 1's score-refusal correlation, which Algorithm 1 builds into the metric.

  1. self definitional [Section 3.2, Algorithm 1; Section 5.1.1, Finding 1]
    "ifS ij ref use= 0then Result =(S ij ref use,0) ... As shown in Table 3, a model’s response score(final output of Algorithm 1) decreases as its refusal rate increases. ... Finding 1: A negative correlation exists between LLMs’ response score and refusal rate."

    Algorithm 1 assigns quality score 0 whenever the refusal indicator is 0, while non-refusal quality scores are 1–4. The reported average score is therefore algebraically (1 − refusal_rate) × average non-refusal quality, so a higher refusal rate mechanically lowers the average score. Finding 1 is a restatement of this scoring definition rather than an independent empirical discovery; the correlation is partly built into the metric by construction.

full rationale

The paper's core claims are direct measurements on a manually constructed prompt set: no parameter is fitted and then relabeled as a prediction, no uniqueness theorem is imported from the authors' prior work, and no load-bearing conclusion rests on a self-citation. The LLM judges are anchored to 300 human labels in Table 2, and the jailbreak methods are external cited algorithms. The only circular element is supporting Finding 1, where the metric's zero quality score for refusals forces the observed negative score-refusal relationship. I also note, under correctness rather than circularity, that the abstract's 60.93%/39.92% values and Section 3.1.3's 70.56%/51.50% figures are not recoverable from Tables 3, 6, or 7, whose overall averages are 49.65% and 61.12%; this is a reproducibility discrepancy, not a circular derivation.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper makes no fitted-parameter claims. Its central measurements rest on five domain assumptions: representativeness of the 11 attacks, representativeness of the 320 requirements, validity of LLM judges, single-sample stability, and generality of Qwen-Turbo generated attacks. None of these are independently verified beyond weak agreement checks.

assumptions (5)
  • domain assumption The 11 selected black-box jailbreak methods are representative of the space of real-world jailbreak attacks.
    Section 3.1.2 selects three categories totaling 11 methods, but no coverage argument or comparison to the broader attack literature is provided.
  • domain assumption The 320 manually crafted malware requirements are a representative sample of real-world malicious code requests.
    Section 3.1.1 derives categories from the malimg dataset, but the 5-20 questions per category are hand-written with no inter-annotator agreement or coverage metric.
  • domain assumption LLM judges provide valid refusal and quality scores for all responses.
    Section 3.3 and Table 2: judges agree with human labels exactly 40.33%-67.67% and coarsely 67.33%-80.33%; the paper uses judge scores as ground truth for all responses without adjusting for disagreement.
  • domain assumption A single forward pass per prompt at temperature 0.9 is representative of a model's behavior.
    Appendix A.3 sets temperature 0.9, top_p 0.95; the evaluation queries each prompt once, so point estimates include sampling noise that is not reported.
  • domain assumption Attacks generated with Qwen-Turbo are representative of attacks from other LLM generators.
    Section 3.1.2 uses Qwen-Turbo for all jailbreak prompt generation; the Limitations section acknowledges this may limit generalizability, but the main conclusions are not qualified accordingly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges." pith.science (2026). https://pith.science/paper/RJXRC2MW

@misc{pith2026250610022,
  author       = {Pith},
  title        = {Pith review of: LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RJXRC2MW}},
  note         = {Machine review of arXiv:2506.10022}
}
read the original abstract

The widespread adoption of Large Language Models (LLMs) has heightened concerns about their security, particularly their vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs. While prior research has been conducted on general security capabilities of LLMs, their specific susceptibility to jailbreak attacks in code generation remains largely unexplored. To fill this gap, we propose MalwareBench, a benchmark dataset containing 3,520 jailbreaking prompts for malicious code-generation, designed to evaluate LLM robustness against such threats. MalwareBench is based on 320 manually crafted malicious code generation requirements, covering 11 jailbreak methods and 29 code functionality categories. Experiments show that mainstream LLMs exhibit limited ability to reject malicious code-generation requirements, and the combination of multiple jailbreak methods further reduces the model's security capabilities: specifically, the average rejection rate for malicious content is 60.93%, dropping to 39.92% when combined with jailbreak attack algorithms. Our work highlights that the code security capabilities of LLMs still pose significant challenges.

Figures

Figures reproduced from arXiv: 2506.10022 by the authors.

Figure 1
Figure 1. Overview of the overall experimental process [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The key statics of MalwareBench can be classified into two categories: white-box attacks and black-box attacks. White-box attack allows the attacker full access to the model’s weights, architecture, training pro￾cess and vectors. Attacks can be designed using gradient signals from the input (Zou et al., 2023). This scenario often applies to open-source models. Black-box attack doesn’t allow an attacker to have so mu… view at source ↗
Figure 3
Figure 3. Heatmaps showing the evaluation scores of different models on attack methods and question categories. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Examples of model results scoring from 0 to 4 [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MalTotal: Cost-Effective and Language-Agnostic Malicious Code Poisoning Detection for Millions of Repositories

    cs.CR 2026-08 conditional novelty 6.0 of 10

    MalTotal combines LLM-based sensitive API discovery, code slicing, and LLM judgment to detect malicious repositories across five languages with an average F1 of 93.1% at low cost per scan.

Reference graph

Works this paper leans on

41 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, and 1 others. 2024. Agentharm: A benchmark for measuring harmfulness of llm agents. arXiv preprint arXiv:2410.09024

    arXiv 2024

  2. [2]

    Aram Bahrini, Mohammadsadra Khamoshifar, Hossein Abbasimehr, Robert J Riggs, Maryam Esmaeili, Rastin Mastali Majdabadkohne, and Morteza Pasehvar. 2023. Chatgpt: Applications, opportunities, and threats. In 2023 Systems and Information Engineering Design Symposium (SIEDS), pages 274--279. IEEE

    work page 2023

  3. [3]

    Maciej Besta, Julia Barth, Eric Schreiber, Ales Kubicek, Afonso Catarino, Robert Gerstenberger, Piotr Nyczyk, Patrick Iff, Yueling Li, Sam Houliston, and 1 others. 2025. Reasoning language models: A blueprint. arXiv preprint arXiv:2501.11223

    arXiv 2025

  4. [4]

    Rishabh Bhardwaj and Soujanya Poria. 2023. Red-teaming large language models using chain of utterances for safety-alignment. arXiv preprint arXiv:2308.09662

    arXiv 2023

  5. [5]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. Language models are few-shot learner...

    2020

  6. [6]

    Pappas, Florian Tram\` e r, Hamed Hassani, and Eric Wong

    Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tram\` e r, Hamed Hassani, and Eric Wong. 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/63092d79154adebd7305dfd498cbff70-Paper-Datasets_and_Benchmarks_Track.pdf Jailbrea...

    work page 2024

  7. [7]

    Jiachi Chen, Qingyuan Zhong, Yanlin Wang, Kaiwen Ning, Yongkun Liu, Zenan Xu, Zhe Zhao, Ting Chen, and Zibin Zheng. 2024. https://doi.org/10.1145/3691620.3695480 Rmcbench: Benchmarking large language models' resistance to malicious code . In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE '24, page 995–1006...

    arXiv 2024

  8. [8]

    Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, and Maosong Sun. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.771 Why should adversarial perturbations be imperceptible? rethink the research paradigm in adversarial NLP . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1122...

    doi:10.18653/v1/2022.emnlp-main.771 2022

Show all 41 references
  1. [9]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning . Preprin...

    2025 arXiv

  2. [10]

    Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2024. Masterkey: Automated jailbreaking of large language model chatbots. In Proc. ISOC NDSS

    2024

  3. [11]

    Moussa Koulako Bala Doumbouya, Ananjan Nandi, Gabriel Poesia, Davide Ghilardi, Anna Goldie, Federico Bianchi, Dan Jurafsky, and Christopher D Manning. 2025. https://openreview.net/forum?id=zZ8fgXHkXi h4rm3l: A language for composable jailbreak attack synthesis . In The Thirtee...

    2025

  4. [12]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  5. [13]

    Daniel Gibert, Carles Mateu, Jordi Planes, and Ramon Vicens. 2019. https://doi.org/10.1007/s11416-018-0323-0 Using convolutional neural networks for classification of malware represented as images . Journal of Computer Virology and Hacking Techniques, 15

    2019 doi

  6. [14]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, and 1 others. 2024. Deepseek-coder: When the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196

    2024 arXiv

  7. [15]

    Divij Handa, Advait Chirmule, Bimal Gajera, and Chitta Baral. 2024. Jailbreaking proprietary large language models using word substitution cipher. arXiv e-prints, pages arXiv--2402

    2024

  8. [16]

    Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2024. https://openreview.net/forum?id=r42tSSCHPh Catastrophic jailbreak of open-source LLM s via exploiting generation . In The Twelfth International Conference on Learning Representations

    2024

  9. [17]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, and 1 others. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186

    2024 arXiv

  10. [18]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  11. [19]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  12. [20]

    Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. Artprompt: Ascii art-based jailbreak attacks against aligned llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volum...

    2024

  13. [21]

    Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. 2024. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. In 2024 IEEE Security and Privacy Workshops (SPW), pages 132--143. IEEE

    2024

  14. [22]

    Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. 2024. https://openreview.net/forum?id=bYa0BhKR4q Deepinception: Hypnotize large language model to be jailbreaker . In Neurips Safe Generative AI Workshop 2024

    2024

  15. [23]

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Ale...

    2022 doi

  16. [24]

    Tong Liu, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen. 2024 a . https://www.usenix.org/conference/usenixsecurity24/presentation/liu-tong Making them ask and answer: Jailbreaking large language models in few queries via disguise and reconstruction . In 33rd USENIX Securit...

    2024

  17. [25]

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024 b . https://openreview.net/forum?id=7Jwpw4qKkb Autodan: Generating stealthy jailbreak prompts on aligned large language models . In The Twelfth International Conference on Learning Representations

    2024

  18. [26]

    Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. 2024. https://openreview.net/forum?id=GC4mXVfquq Jailbreakv: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks . In First Conference on Language Modeling

    2024

  19. [27]

    Alec Radford and Karthik Narasimhan. 2018. https://api.semanticscholar.org/CorpusID:49313245 Improving language understanding by generative pre-training

    2018

  20. [28]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, and 1 others. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950

    2023 arXiv

  21. [29]

    Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and 1 others. 2024. A strongreject for empty jailbreaks. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models

    2024

  22. [30]

    Kazuhiro Takemoto. 2024. All in how you ask for it: Simple black-box method for jailbreak attacks. Applied Sciences, 14(9):3558

    2024

  23. [31]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023 a . Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  24. [32]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023 b . Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  25. [33]

    Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. 2023. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387

    2023 arXiv

  26. [34]

    Zheng Xin Yong, Cristina Menghini, and Stephen Bach. 2023. https://openreview.net/forum?id=pn83r8V2sv Low-resource languages jailbreak GPT -4 . In Socially Responsible Language Modelling Research

    2023

  27. [35]

    Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024. https://openreview.net/forum?id=MbfAK4s61A GPT -4 is too smart to be safe: Stealthy chat with LLM s via cipher . In The Twelfth International Conference on Learning Repres...

    2024

  28. [36]

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. https://doi.org/10.18653/v1/2024.acl-long.773 How johnny can persuade LLM s to jailbreak them: Rethinking persuasion to challenge AI safety by humanizing LLM s . In Proceedings of the 62nd Annual...

    2024 doi

  29. [37]

    Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang. 2025. https://openreview.net/forum?id=V4y0CpX4hK Agent security bench ( ASB ): Formalizing and benchmarking attacks and defenses in LLM -based agents . In The Thirt...

    2025

  30. [38]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

  31. [39]

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043

    2023 arXiv

  32. [40]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  33. [41]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.