Pith. sign in

REVIEW 4 major objections 6 minor 71 references

MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that LLM alignment filters judge prompts in isolation, and that this compositional blindness lets a two-stage compiler assemble functional malware from benign-looking parts, beating jailbreaks and paid services.

desk verdict The MDIR compiler pipeline is a genuine and well-engineered contribution, but the headline correctness numbers rest on an LLM judge that never executes code, so the paper's quantitative claims outrun its evidence. read the letter →

arxiv 2507.02057 v1 pith:FBQAE62S submitted 2025-07-02 cs.CR cs.AI

Lu Yan , Zhuo Zhang , Xiangzhe Xu , Shengwei An , Guangyu Shen , Zhou Xuan , Xuan Chen , Xiangyu Zhang This is my paper
classification cs.CRcs.AI
keywords largelanguagemodelsmalwaregenerationcompositionalblindnessLLMalignmentbypassintermediaterepresentationjailbreakingredteamingdecompositionattack
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that current safety alignment for large language models has a structural blind spot: each prompt is judged on its own, so a set of individually benign requests can add up to a working malware program without any single request looking malicious. To turn that blind spot into a systematic attack, the authors build the Malware Generation Compiler (MGC), a two-stage pipeline in which a small, loosely aligned model decomposes a high-level malicious goal into benign-appearing sub-tasks, a powerful aligned model writes concrete code for each sub-task, and a formal intermediate representation named MDIR assembles the parts into a compilable program. Across three benchmark datasets the paper reports that MGC outperforms jailbreaking methods by +365.79% and underground paid services by +78.07% in judged correctness, and that it reproduces or enhances 16 real-world malware samples. The intended lesson is that per-prompt safety filters are insufficient and that composition-aware defenses are required.

What carries the argument

The load-bearing object is MDIR, the Malware Description Intermediate Representation, a deliberately small domain-specific language modeled on the intermediate representation of a classic compiler. It supports only three primitive types plus library structs, and it distinguishes abstract functions, which carry a natural-language description instead of a body, from concrete functions, which contain real statements; every valid MDIR program has exactly one main function. The static type rules (for example, pointer arithmetic only with integer offsets, function calls matched to declared signatures, a single well-typed main) let the frontend catch and reject the weak model's hallucinated programs before any code is committed. The translation semantics then map MDIR to a target language such as C, and each abstract function is handed to the strong model as an isolated request whose returned signature is checked against the interface MDIR declared. A second mechanism, the iterative feedback loop, handles refusals: the backend model's rejection of a function triggers keyword sanitization (showRansomMsg becomes showMsg) and, if needed, granular decomposition of that function into smaller sub-functions until the strong model produces an implementation.

What would settle it

Execute the generated samples from the three benchmark datasets in a sandbox and count how many compile and actually exhibit the claimed malicious behavior (encrypting files, beaconing to a command server, persisting across reboots, and so on). If the execution-verified success rate runs far below the judge-assigned correctness scores — for instance, if most highly scored samples fail to compile or do nothing when run — the central claim that MGC reliably generates functional malware collapses.

Watch

Extended reading notes

Core claim

The central claim is that decomposition bypasses alignment without any obfuscation: an aligned model refuses 'write ransomware' but will happily implement 'list files in a directory,' 'encrypt a file,' and 'display a payment message,' and those pieces, joined offline, are a ransomware program. The paper's discovery is that this decomposition can be made systematic and reliable by separating planning from implementation. The frontend uses a weak model to expand the attacker's vague request, turn it into a workflow, and express that workflow in MDIR, whose abstract functions carry innocuous natural-language descriptions and whose formal syntax and type rules reject malformed output. The backend then asks a strong aligned model to implement each abstract function in isolation, verifying that the returned signature matches the declared interface. Because the strong model never sees the whole program, no prompt trips its safety filter, and because MDIR fixes the interfaces, the final assembly is deterministic rather than left to a fallible model. The paper reports judged-correctness gains of +365.79% over jailbreaking baselines and +78.07% over the best underground service, with high scores across all ten MITRE ATT&CK categories and stability across weak-model choices and sampling runs.

Load-bearing premise

The reported correctness numbers are assigned by a judge LLM that reads the generated code — inside a 'software security course' role-play — rather than by compiling and executing it, so the claim that MGC reliably produces functional malware assumes the judge's scores reflect true functional behavior; only 16 of the many generated samples received actual behavioral validation.

Editorial extensions

If this is right

  • Per-prompt alignment as currently deployed does not stop malware generation: MGC composes working malicious programs while no individual prompt contains detectable malicious intent.
  • The skill barrier for malware creation drops sharply, because the frontend's decomposition step lets an attacker who can describe a goal in plain language drive the whole pipeline.
  • History-based defenses are not an easy remedy: the paper's adaptive-defense experiment, in which a defender inspects a window of 100 requests, detects the attack below 0.6% of the time even when every malicious step is present in the window.
  • The leading jailbreaking pipelines and underground paid services are far less reliable than structured decomposition at producing executable, behaviorally faithful code.
  • Robust defense requires composition-aware mechanisms, such as correlating sub-task requests and interfaces across sessions, rather than stronger filters on individual prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The decomposition principle is likely general: the same weak-plans/strong-implements split could assemble phishing infrastructure, fraud tooling, or other dual-use code, because it attacks the structure of alignment review rather than any particular model's guardrails.
  • A defense the paper leaves implicit is monitoring for MDIR-like structured decomposition itself — repeated requests that declare formal interfaces and abstract function descriptions — which could be flagged before assembly ever happens.
  • Because the headline metrics come from a judge model reading code rather than running it, the claim would be put on firmer ground by execution-based validation on far more than the 16 case-study samples.
  • A testable extension is to run the same pipeline against models whose safety training includes compositional examples (pairs of prompts whose joined intent is labeled), which would indicate whether the blind spot is fixable by training data alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces MGC, a two-stage framework that generates malware from natural-language descriptions by decomposing the malicious request into seemingly benign sub-tasks. A weak LLM (Mistral-7B) produces a workflow and translates it into MDIR, a small typed intermediate representation; a strong aligned LLM (GPT-4o-mini, Claude, or Hermes) implements the abstract functions in C/Python/Rust; and an iterative feedback loop handles refusals via keyword sanitization and granular decomposition. The evaluation compares MGC with direct queries, two jailbreak pipelines, and underground paid services on the MSC, Mitre, and Malla datasets, using GPT-4o-mini as a judge for correctness and quality, supplemented by syntax metrics and, for 16 MSC projects, sandbox-based behavioral validation. The paper claims large improvements over baselines (+365.79% over jailbreaks, +78.07% over underground services) and that MGC reproduces or enhances 16 real-world malware samples.

Significance. If the capability claims were fully supported, MGC would be a notable contribution: it identifies a plausible compositional blind spot in LLM alignment, proposes a concrete intermediate representation with formal syntax, typing, and translation rules, and provides behavioral evidence on a small set of real malware projects. The formalization of MDIR, the iterative refusal-handling design, and the 16 execution-validated case studies are credible strengths, as is the robustness check across weak and strong model choices. However, the headline quantitative claims rest on judge scores of unexecuted code, and the generality of the 'reliably generates functional malware' statement is not yet established by the reported evidence.

major comments (4)
  1. [§8.1, §A.3, Table 1] The headline claims ('reliably generates functional malware', '+365.79%', '+78.07%') are built on GPT-4o-mini judge scores of correctness and quality without executing the code. The judge is prompted in a 'software security course' role-play (A.2) and is asked to compare the generated workflow against its own understanding of the required components (A.3), which can inflate scores and makes the evaluation partly self-referential; for the GPT-4o-mini backend, the generator and judge are the same model. Only 16 of 125 MSC projects receive any sandbox or behavioral validation (Table 1), and even there the behavior matches are partial (e.g., Kaiten 42.86%, PunBB 33.33%). No execution-based validation is reported for the Mitre (1,000 prompts) or Malla (35 prompts) datasets. The finding that 'MGC reliably generates realistic and behaviorally faithful malware' (Finding 1) is therefore not supported for the full evaluation. Please add per-dataset execution-based pass rates on a representative sample, or restrict the claims to compilable code judged to implement the requested components.
  2. [§8.4, Table 4] The underground-service comparison mixes evaluation protocols. The correctness, quality, and compilation-rate columns for BadGPT, CodeGPT, EscapeGPT, Evil-GPT, FreedomGPT, MakerGPT, XXXGPT, DarkGPT, and WolfGPT appear to carry values over from the Malla study, while MGC is scored by the new GPT-4o-mini judge under the software-security-course prompt. The caption only states that compilation rates are taken from the original paper, but the same provenance issue applies to the correctness and quality scores. If the two sets of scores were produced by different judges or prompts, the reported '+78.07%' margin over underground services is not an apples-to-apples comparison. Please either rerun the underground baselines under the identical judge protocol or calibrate the Malla scores with a common held-out set.
  3. [Abstract/§1 vs §8.4, Figure 10, Table 3] The quoted improvements are not derivable from the reported numbers. From Table 4, MGC's Malla correctness (7.52) versus the best underground service XXXGPT (6.63) is +13.4%, not +78.07%. From Figure 10, MGC versus DRA and DeepInception on Malla is not +365.79% under any obvious formula. The manuscript never states the aggregation formula for the headline percentages. Please provide the exact per-dataset and aggregate computation, with confidence intervals. Additionally, DRA and DeepInception output 0 LOC for Claude and Hermes in Table 3 and score 0.00 in Figure 10, so the jailbreak comparison may be measuring total pipeline failure rather than a meaningful capability difference; please report refusal rates and the number of candidates generated per baseline.
  4. [Reproducibility] The manuscript provides no code, data, or full prompt templates for the MGC frontend, backend, judge, or the adaptive-defense experiment; the only artifact is a demonstration website. Since the main findings are empirical and depend on the exact wording of the decomposition and judge prompts, the claims cannot be checked from the text alone. Please release, at least in a controlled form for reviewers, the MDIR few-shot examples, the CoT prompts, the judge prompt, the dataset filters, and the sandbox configuration used for Table 1.
minor comments (6)
  1. [§3] There are typos: 'we introduces' should be 'we introduce', and 'poses nontrivial' should be 'is nontrivial'.
  2. [Table 1] The header row is garbled or misaligned, and several rows (e.g., BallPit) show 0 flagged behaviors but 4 enhanced behaviors, which is confusing; please clarify what '0 -' means and define 'enhanced' behavior.
  3. [§6.1] The phrase 'intricate self-correction mechanism' should be 'iterative self-correction mechanism'; the mechanism itself is not ablated or evaluated in isolation, so its contribution is unclear.
  4. [§A.3] The judge is asked to 'use the same standard' as the first generation, which can introduce order effects; please randomize evaluation order and report inter-rater or human agreement on a subset.
  5. [§A.1] Please specify the compiler, toolchain, and operating system used to determine the 125 'compilable' MSC projects, and state how many of the 16 Table 1 projects were actually compiled versus only analyzed by static or dynamic sandbox tools.
  6. [§8.6, Table 8] The adaptive-defense protocol is underspecified: it is not clear how the 100-request history is generated, what the classifier input is, or how the five random subsets are chosen; this is a useful preliminary result but should be labeled as such.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MDIR is a formal IR with independent type-checking and translation semantics; evaluation bias from LLM judge and limited sandbox validation is a validity concern, not a circular reduction.

full rationale

MGC's pipeline is not built from fitted parameters or from equations that reduce to their own outputs. MDIR is presented with an abstract syntax, static typing rules, and a translation semantics (Figs. 3-6) that map MDIR programs into C in a deterministic, syntax-directed way. The backend prompts a strong LLM to fill abstract function bodies, and the only feedback loop is refusal-driven sanitization or decomposition (Algorithm 1); this loop does not use the evaluation scores as inputs. The headline correctness/quality numbers come from GPT-4o-mini as an LLM judge that does not execute code, and only a subset of MSC projects received sandbox-based behavior validation; while this is a substantial threat to the paper's external validity, it is not circularity under the stated criteria because the judge's scores are not fed back into MGC's generation process. The two self-citations in the related work (RL-jack and Rapid Optimization) are contextual mentions and are not load-bearing for the paper's central claim. No specific step can be quoted where a prediction is equivalent by construction to its inputs, so the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no adjustable parameters; instead, it relies on three domain assumptions about LLM behavior and evaluation that are not empirically proven beyond the paper's own limited validation. These assumptions are load-bearing for the central claims.

assumptions (3)
  • domain assumption Strongly aligned LLMs evaluate each prompt in isolation and do not detect intent across separate sub-tasks.
    This is the foundational premise of the attack, introduced in the Abstract and Section 1, and is required for the sub-tasks to be generated without refusal.
  • domain assumption A weakly aligned LLM such as Mistral-7B can correctly decompose malware intents into benign-appearing components that cover all required functionality.
    The frontend relies on this for MDIR generation (Section 6); if the decomposition is incomplete, the final composed program would be non-functional.
  • domain assumption The LLM judge (GPT-4o-mini) provides reliable and unbiased scores for code correctness and quality without being misled by the security-course framing.
    Section A.2/A.3 use the judge for the main metrics; biased scoring would inflate all reported improvements.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation." pith.science (2026). https://pith.science/paper/FBQAE62S

@misc{pith2026250702057,
  author       = {Pith},
  title        = {Pith review of: MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FBQAE62S}},
  note         = {Machine review of arXiv:2507.02057}
}
read the original abstract

Large language models (LLMs) have democratized software development, reducing the expertise barrier for programming complex applications. This accessibility extends to malicious software development, raising significant security concerns. While LLM providers have implemented alignment mechanisms to prevent direct generation of overtly malicious code, these safeguards predominantly evaluate individual prompts in isolation, overlooking a critical vulnerability: malicious operations can be systematically decomposed into benign-appearing sub-tasks. In this paper, we introduce the Malware Generation Compiler (MGC), a novel framework that leverages this vulnerability through modular decomposition and alignment-evasive generation. MGC employs a specialized Malware Description Intermediate Representation (MDIR) to bridge high-level malicious intents and benign-appearing code snippets. Extensive evaluation demonstrates that our attack reliably generates functional malware across diverse task specifications and categories, outperforming jailbreaking methods by +365.79% and underground services by +78.07% in correctness on three benchmark datasets. Case studies further show that MGC can reproduce and even enhance 16 real-world malware samples. This work provides critical insights for security researchers by exposing the risks of compositional attacks against aligned AI systems. Demonstrations are available at https://sites.google.com/view/malware-generation-compiler.

Figures

Figures reproduced from arXiv: 2507.02057 by the authors.

Figure 1
Figure 1. Motivation for MGC. Insight 1: Decomposing malicious tasks into innocuous modular steps enables bypassing alignment. Recognizing the limitations of direct queries and jailbreak tech￾niques, attackers leverage a key insight: malicious behavior can be systematically decomposed into modular steps, each appearing benign in isolation. Instead of requesting ransomware directly, the attacker first turns to a weaker model t… view at source ↗
Figure 2
Figure 2. Workflow of MGC. The frontend of MGC leverages a smaller, weakly aligned LLM (denoted as M𝑤), such as Mistral. Its primary function is to de￾compose a high-level malicious objective into a set of potentially benign operations. Furthermore, the frontend employs MDIR to define how these benign operations should be composed to achieve the overarching malicious goal. To support this process, MDIR is intentionally design… view at source ↗
Figure 3
Figure 3. The abstract syntax of 𝜆MDIR. of allowing users to define custom data structures, MDIR only al￾lows the use of structures imported from external libraries. These structures are denoted using two textual strings, 𝑐LibraryName and 𝑐StructName. For example, dirent.h::dirent refers to the dirent data structure defined in dirent.h. MDIR also supports pointers and arrays. The simplicity of this type system is an intention… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Overview of frontend. To improve structural integrity, we introduce a intricate self￾correction mechanism [36]. In this step, M𝑤 reviews the interac￾tions between different components identified earlier, verifying that dependencies are correctly handled and logical con…
Figure 5
Figure 5. Figure 5: Static semantics (excerpt) of 𝜆MDIR. 6.3 MDIR Verification However, despite MDIR being intentionally designed for simplicity, the limited capability ofM𝑤 can still result in hallucinations and the generation of ill-formed MDIR programs. To address this, we define well-…
Figure 6
Figure 6. Figure 6: The translation semantics of 𝜆MDIR (to C language). to 𝑡. For data structures 𝑡𝑠 = 𝑐LibraryName:𝑐StructName, 𝑡 ↑ 𝑠 refers to the structure with the same name StructName, imported from the external library LibraryName. For other compound data types, such as arrays, 𝑡 ↑ …
Figure 7
Figure 7. Figure 7: MGC achieves higher correctness and quality scores compared to baseline approaches across all datasets. DQ(𝑥) denotes Direct Query to model 𝑥 without decomposition, where 𝑥 ∈ {M: Mistral, G: GPT-4o-mini, C: Claude, H: Hermes-Llama}. MGC(M→𝑥) represents MGC with Mistral…
Figure 8
Figure 8. Figure 8: MGC with strong models exhibits consistently high correctness across MITRE ATT&CK categories. and implementation, to evaluate if MGC pipeline can also enhance the performance of a weakly aligned model. Persistence Exfil Privilege Escalation Recon C2 ExecutionEvasion Co…
Figure 10
Figure 10. Figure 10: MGC outperforms jailbreaking techniques across datasets. to 4.07, demonstrating effective safety alignment. Even the less￾restricted weak model Mistral achieves only up to 2.78 correctness scores, suggesting its limited capability in generating functional code. From a…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 33 canonical work pages

  1. [1]

    [n. d.]. https://github.com/vxunderground/MalwareSourceCode

    work page

  2. [2]

    [n. d.]. https://huggingface.co/TheDrummer/Tiger-Gemma-9B-v2

    work page

  3. [3]

    [n. d.]. https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b

    work page

  4. [4]

    [n. d.]. https://www.virustotal.com/gui/home/upload

    work page

  5. [5]

    [n. d.]. https://hybrid-analysis.com/

    work page

  6. [6]

    [n. d.]. https://www.crowdstrike.com/en-us/

    work page

  7. [7]

    Jamal Al-Karaki, Muhammad Al-Zafar Khan, and Marwan Omar. 2024. Exploring llms for malware detection: Review, framework design, and countermeasure approaches. arXiv preprint arXiv:2409.07587 (2024)

    arXiv 2024

  8. [8]

    Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons. 2023. Image hijacks: Adversarial images can control generative models at runtime. arXiv preprint arXiv:2309.00236 (2023)

    arXiv 2023

Show all 71 references
  1. [9]

    Mika Beckerich, Laura Plein, and Sergio Coronado. 2023. Ratgpt: Turning online llms into proxies for malware attacks. arXiv preprint arXiv:2308.09183 (2023)

    2023 arXiv

  2. [10]

    Rishabh Bhardwaj and Soujanya Poria. 2023. Red-teaming large language models using chain of utterances for safety-alignment. arXiv preprint arXiv:2308.09662 (2023)

    2023 arXiv

  3. [11]

    Manish Bhatt, Sahana Chennabasappa, Cyrus Nikolaidis, Shengye Wan, Ivan Evti- mov, Dominik Gabi, Daniel Song, Faizan Ahmad, Cornelius Aschermann, Lorenzo Fontana, et al. 2023. Purple llama cyberseceval: A secure coding benchmark for language models. arXiv preprint arXiv:2312.0...

    2023 arXiv

  4. [12]

    Barry W Boehm, John R Brown, and Myron Lipow. 1976. Quantitative evaluation of software quality. In Proceedings of the 2nd international conference on Software engineering. 592–605

    1976

  5. [13]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  6. [14]

    Pin-Yu Chen, Han Shen, Payel Das, and Tianyi Chen. 2025. Fundamental Safety- Capability Trade-offs in Fine-tuning Large Language Models. arXiv preprint arXiv:2503.20807 (2025)

    2025 arXiv

  7. [15]

    Xuan Chen, Yuzhou Nie, Lu Yan, Yunshu Mao, Wenbo Guo, and Xiangyu Zhang

  8. [16]

    Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2025. Security and privacy challenges of large language models: A survey. Comput. Surveys (2025)

    2025

  9. [17]

    Yi Dong, Ronghui Mu, Gaojie Jin, Yi Qi, Jinwei Hu, Xingyu Zhao, Jie Meng, Wenjie Ruan, and Xiaowei Huang. 2024. Building guardrails for large language models. arXiv preprint arXiv:2402.01822 (2024)

    2024 arXiv

  10. [18]

    Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfi...

    2022 arXiv

  11. [19]

    Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. 2023. Mart: Improving llm safety with multi-round automatic red-teaming. arXiv preprint arXiv:2311.07689 (2023)

    2023 arXiv

  12. [20]

    Chengquan Guo, Xun Liu, Chulin Xie, Andy Zhou, Yi Zeng, Zinan Lin, Dawn Song, and Bo Li. 2024. RedCode: Risky Code Execution and Generation Bench- mark for Code Agents. In Advances in Neural Information Processing Systems , A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paqu...

    2024

  13. [21]

    Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al . 2023. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352 (2023)

    2023 arXiv

  14. [22]

    Md Imran Hossen, Jianyi Zhang, Yinzhi Cao, and Xiali Hei. 2024. Assessing cybersecurity vulnerabilities in code large language models. arXiv preprint arXiv:2404.18567 (2024)

    2024 arXiv

  15. [23]

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology (2024)

    2024

  16. [24]

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674 (2023)

    2023 arXiv

  17. [25]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515 (2024)

    2024 arXiv

  18. [26]

    Rudolf K Keller and Reinhard Schauer. 1998. Design components: Towards software composition at the design level. In ICSE

    1998

  19. [27]

    Chris Lattner and Vikram Adve. 2004. LLVM: A compilation framework for lifelong program analysis & transformation. In International symposium on code generation and optimization, 2004. CGO 2004

    2004

  20. [28]

    Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han

  21. [29]

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. 2024. Images are achilles’ heel of alignment: Exploiting visual vulnerabilities for jail- breaking multimodal large language models. InEuropean Conference on Computer Vision. Springer, 174–189

    2024

  22. [30]

    Lizhi Lin, Honglin Mu, Zenan Zhai, Minghan Wang, Yuxia Wang, Renxi Wang, Junjie Gao, Yixuan Zhang, Wanxiang Che, Timothy Baldwin, et al. 2025. Against The Achilles’ Heel: A Survey on Red Teaming for Generative Models. Journal of Artificial Intelligence Research 82 (2025), 687–775

    2025

  23. [31]

    Zilong Lin, Jian Cui, Xiaojing Liao, and XiaoFeng Wang. 2024. Malla: Demysti- fying Real-world Large Language Model Integrated Malicious Services. arXiv preprint arXiv:2401.03315 (2024)

    2024 arXiv

  24. [32]

    Fang Liu, Yang Liu, Lin Shi, Houkun Huang, Ruifeng Wang, Zhen Yang, Li Zhang, Zhongqi Li, and Yuchi Ma. 2024. Exploring and evaluating hallucinations in llm-powered code generation. arXiv preprint arXiv:2404.00971 (2024)

    2024

  25. [33]

    Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen

  26. [34]

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023. Autodan: Generat- ing stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451 (2023)

    2023 arXiv

  27. [35]

    Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, et al. 2023. Prompt Injec- tion attack against LLM-integrated Applications. arXiv preprint arXiv:2306.05499 (2023)

    2023 arXiv

  28. [36]

    In USENIX

    Making them ask and answer: Jailbreaking large language models in few queries via disguise and reconstruction. In USENIX

  29. [37]

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2023. Tree of attacks: Jailbreaking black-box llms automatically. arXiv preprint arXiv:2312.02119 (2023)

    2023 arXiv

  30. [38]

    Manish Nagireddy, Bernat Guillén Pegueroles, and Ioana Baldini. 2024. DARE to Diversify: DAta Driven and Diverse LLM REd Teaming. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 6420–6421

    2024

  31. [39]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self. Feedback (2023)

    2023

  32. [40]

    Yin Minn Pa Pa, Shunsuke Tanizaki, Tetsui Kou, Michel Van Eeten, Katsunari Yoshioka, and Tsutomu Matsumoto. 2023. An attacker’s dream? exploring the capabilities of chatgpt for developing malware. In Proceedings of the 16th Cyber Security Experimentation and Test Workshop. 10–18

    2023

  33. [41]

    Sudipta Paria, Aritra Dasgupta, and Swarup Bhunia. 2023. Divas: An llm-based end-to-end framework for soc security analysis and policy-based protection. arXiv preprint arXiv:2308.06932 (2023)

    2023 arXiv

  34. [42]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. NeurIPS (2022)

    2022

  35. [43]

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. 2023. Chatdev: Communicative agents for software development. arXiv preprint arXiv:2307.07924 (2023)

    2023 arXiv

  36. [44]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. NeurIPS (2023)

    2023

  37. [45]

    Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red teaming language models with language models. arXiv preprint arXiv:2202.03286 (2022). Preprint, July 2025, Virtual Lu Yan et al

    2022 arXiv

  38. [46]

    Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2023. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. arXiv preprint arXiv:2307.14539 (2023)

    2023 arXiv

  39. [47]

    Guangyu Shen, Siyuan Cheng, Kaiyuan Zhang, Guanhong Tao, Shengwei An, Lu Yan, Zhuo Zhang, Shiqing Ma, and Xiangyu Zhang. 2024. Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia. arXiv preprint arXiv:2402.05467 (2024)

    2024 arXiv

  40. [48]

    Rusheb Shah, Quentin Feuillade Montixi, Soroush Pour, Arush Tagade, and Javier Rando. 2023. Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation. In NeurIPS workshop SoLaR

    2023

  41. [49]

    Martin Shepperd. 1988. A critique of cyclomatic complexity as a software metric. Software Engineering Journal 3, 2 (1988), 30–36

    1988

  42. [50]

    Jiawen Shi, Zenghui Yuan, Yinuo Liu, Yue Huang, Pan Zhou, Lichao Sun, and Neil Zhenqiang Gong. 2024. Optimization-based prompt injection attack to llm- as-a-judge. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 660–674

    2024

  43. [51]

    Do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023. "Do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.03825 (2023)

    2023 arXiv

  44. [52]

    Megh Thakkar, Yash More, Quentin Fournier, Matthew Riemer, Pin-Yu Chen, Amal Zouaq, Payel Das, and Sarath Chandar. 2024. Combining domain and alignment vectors to achieve better knowledge-safety trade-offs in llms. arXiv preprint arXiv:2411.06824 (2024)

    2024 arXiv

  45. [53]

    Lingzhi Wang, Jiahui Wang, Kyle Jung, Kedar Thiagarajan, Emily Wei, Xiangmin Shen, Yan Chen, and Zhenyuan Li. 2024. From Sands to Mansions: Enabling Automatic Full-Life-Cycle Cyberattack Construction with LLM. arXiv preprint arXiv:2407.16928 (2024)

    2024

  46. [54]

    Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang, and Cho- Jui Hsieh. 2024. Red teaming language model detectors with language models. Transactions of the Association for Computational Linguistics 12 (2024), 174–189

    2024

  47. [55]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How Does LLM Safety Training Fail?. In NeurIPS

    2023

  48. [56]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  49. [57]

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024. Executable code actions elicit better llm agents. arXiv preprint arXiv:2402.01030 (2024)

    2024 arXiv

  50. [58]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing (2024)

    2024

  51. [59]

    Jiahao Yu, Xingwei Lin, and Xinyu Xing. 2023. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253 (2023)

    2023 arXiv

  52. [60]

    Jiacen Xu, Jack W Stokes, Geoff McDonald, Xuesong Bai, David Marshall, Siyue Wang, Adith Swaminathan, and Zhou Li. 2024. Autoattacker: A large language model guided system to implement automatic cyber-attacks. arXiv preprint arXiv:2403.01038 (2024)

    2024 arXiv

  53. [61]

    Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents. arXiv preprint arXiv:2403.02691 (2024)

    2024 arXiv

  54. [62]

    Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B Tenenbaum, and Chuang Gan. 2023. Planning with large language models for code generation. arXiv preprint arXiv:2303.05510 (2023)

    2023 arXiv

  55. [63]

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi

  56. [64]

    arXiv preprint arXiv:2401.06373 (2024)

    How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. arXiv preprint arXiv:2401.06373 (2024)

    2024 arXiv

  57. [65]

    de- composition correctness,

    Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 (2023). MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generatio...

    2023 arXiv

  58. [67]

    Ying Zhang, Xiaoyan Zhou, Hui Wen, Wenjia Niu, Jiqiang Liu, Haining Wang, and Qiang Li. 2024. Tactics, Techniques, and Procedures (TTPs) in Interpreted Malware: A Zero-Shot Generation with Large Language Models. arXiv preprint arXiv:2407.08532 (2024)

    2024 arXiv

  59. [68]

    Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, and Rui Wang. 2023. Unifying the perspectives of nlp and software engineering: A survey on language models for code. arXiv preprint arXiv:2311.07989 (2023)

    2023 arXiv

  60. [70]

    CDN behavior analysis:

  61. [71]

    "" Generates a randomized subdomain for evasion

    Traffic anomaly monitoring [Omitted: detailed discussion of anti-domain-fronting defenses] """ def monitor_domain_patterns(traffic_log): # [Displays how to detect domain fronting anomalies rather than implement them] patterns = { 'mismatched_headers': 0, 'cdn_anomalies': 0} fo...

  62. [2023]

    arXiv preprint arXiv:2311.03191 (2023)

    Deepinception: Hypnotize large language model to be jailbreaker. arXiv preprint arXiv:2311.03191 (2023)

    2023 arXiv

  63. [2024]

    arXiv preprint arXiv:2406.08725 (2024)

    Rl-jack: Reinforcement learning-powered black-box jailbreaking attack against llms. arXiv preprint arXiv:2406.08725 (2024)

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.