Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RedCoder, an automated multi-turn red-teaming agent, induces vulnerable code in up to 65% of conversations with code LLMs.

desk verdict A useful, genuinely new red-teaming pipeline with strong reported gains, but the load-bearing evaluation oracle (CodeGuru) needs stronger independent validation before I'd trust the absolute rates. read the letter →

arxiv 2507.22063 v2 pith:E2N27A5R submitted 2025-06-25 cs.SE cs.AI

Wenjie Jacky Mo , Qin Liu , Xiaofei Wen , Dongwon Jung , Hadi Askari , Wenxuan Zhou , Zhe Zhao , Muhao Chen This is my paper
classification cs.SEcs.AI
keywords redteamingcodeLLMsmulti-turnattacksvulnerabilityinductionCWEguardrailsretrieval-augmentedgenerationsecurityevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that red-teaming of code LLMs should model the interactive, multi-turn nature of real AI-assisted programming, and that a fully automated agent can do this better than any previous single-turn or multi-turn attack. RedCoder is built by running a four-agent gaming loop that produces successful 'prototype conversations' and a library of reusable attack tactics; a code LLM fine-tuned on those conversations steers live conversations while retrieving tactics from the library. Across four victim models, RedCoder achieves vulnerability rates of 39.41% to 65.29%, roughly 2–10 times higher than the compared baselines, including attacks on a proprietary model. The authors also report that single-turn guardrails do not meaningfully reduce the attack, whereas a multi-turn guardrail trained on the same prototype conversations cuts the rates substantially. If these results hold, security evaluation of AI coding assistants can be automated at scale, and safety work needs to move to conversation-level defenses.

What carries the argument

The load-bearing object is the gaming loop: an attacker LLM (GPT-4o) carries up to 20 five-turn conversations against a defender made of a coding agent (Llama3-8B-Instruct) plus a guardrail, while Amazon CodeGuru labels each conversation for CWE vulnerabilities and a strategy analyst writes a short summary of the behavioral change that turned a failed attempt into a successful one. The successful conversations (2,098 in all) are decomposed into (history → next utterance) pairs for supervised fine-tuning of the RedCoder backbone. The strategy summaries form a key–value arsenal whose keys are embeddings of individual (query, response) turns; at each turn beyond the first, RedCoder embeds the previous interaction and injects the nearest strategy summary into the system prompt. That retrieval step is what lets the agent adapt its wording to the victim's responses rather than replaying a fixed attack.

What would settle it

Have several independent security engineers manually review a few hundred conversations that CodeGuru flagged as vulnerable and an equal number it cleared, applying a concrete exploitability test (e.g., whether the code can be triggered with a realistic input to produce a security impact). If expert agreement with CodeGuru falls well below the 90% agreement reported in the paper's 20-conversation pilot, the central claim that RedCoder induces genuine vulnerabilities at the reported rates is not established.

Watch

Extended reading notes

Core claim

RedCoder's central claim is that the search over adversarial prompts for code LLMs can be automated and made multi-turn without human prompt engineering. The approach treats successful red-teaming as a learnable behavior: a multi-agent game generates conversations in which an attacker defeats a guardrail-equipped defender, the successful conversations are used as supervised fine-tuning data, and a strategy analyst distills what changed between failed and successful attempts into a retrievable tactic arsenal. At test time, RedCoder treats the victim's latest response as feedback and retrieves the most relevant tactic to shape its next utterance. The reported result is that this method induces at least one CWE-listed vulnerability in 39.41% (CodeLlama-7B), 61.18% (CodeGemma-7B), 65.29% (Qwen2.5-Coder-7B), and 40.00% (DeepSeek-R1-Distill-8B) of conversations, outperforming GCG, AutoDAN, CoA-Feedback, and ActorAttack by large margins, and that the learned tactics transfer to unseen CWE families and to Claude 3.5 Sonnet.

Load-bearing premise

The entire measured success of RedCoder rests on Amazon CodeGuru being an accurate oracle for whether generated code is genuinely vulnerable; if CodeGuru's flags are noisy or biased, RedCoder's reported vulnerability rates overstate real-world security risk.

Editorial extensions

If this is right

  • Automated multi-turn red teaming of code LLMs is feasible without human prompt engineering, enabling large-scale security audits of coding assistants.
  • Single-turn guardrails miss multi-turn attacks: the paper finds they leave RedCoder's vulnerability rate essentially unchanged, so deployed defenses need conversation-level context.
  • Vulnerability-inducing behavior transfers across CWE families: on 20 held-out vulnerability types RedCoder still reaches 65.00% on Qwen2.5-Coder-7B, close to the 65.29% on the original 43-task set.
  • Proprietary, safety-tuned models remain susceptible: RedCoder induces vulnerable code in 42.85% of conversations with Claude 3.5 Sonnet.
  • Reasoning-oriented training does not by itself harden code LLMs, since DeepSeek-R1-Distill-8B exhibits a 40.00% vulnerability rate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method's success is measured through Amazon CodeGuru; a plausible inference is that RedCoder is partly optimized to trigger this detector, so its 'vulnerability rate' should be read as a CodeGuru-flagged rate until independent, exploit-level validation is done.
  • The strategy-arsenal design is domain-agnostic: swapping the evaluator (e.g., a SQL-injection or privacy-policy checker) could yield comparable multi-turn red-team agents for other structured generation tasks.
  • A direct implication for safety research is that guardrail training should use conversation-level labels at the exact turn where vulnerable code first appears; the paper's Appendix A labels history up to that turn as 'safe' and the turn itself as 'unsafe,' a recipe that could generalize to other multi-turn safety benchmarks.
  • Because the gaming process uses GPT-4o as attacker and Llama3-8B as defender, the prototype conversations inherit those models' blind spots; different victim and attacker models may yield different tactic arsenals.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents RedCoder, a multi-turn red-teaming agent for Code LLMs. RedCoder is constructed by a multi-agent gaming process in which an attacker LLM interacts with a defender (a coding agent plus a custom multi-turn guardrail), an evaluator (Amazon CodeGuru) labels whether any response contains CWE vulnerabilities, and a strategy analyst distills reusable tactics from failed-to-successful conversation transitions. The successful prototype conversations are used to fine-tune a Llama-3-8B backbone, and at deployment a retrieval-augmented module retrieves strategy summaries to guide each turn. Experiments on CodeLlama-7B, CodeGemma-7B, Qwen2.5-Coder-7B, and DeepSeek-R1-Distill-Llama-8B report vulnerability rates between 39.41% and 65.29%, outperforming AutoDAN, GCG, CoA-Feedback, and ActorAttack. The paper also reports an out-of-distribution evaluation on 20 new CWE tasks, a small human validation of CodeGuru, and a defense study comparing single-turn and multi-turn guardrails.

Significance. If the reported numbers survive scrutiny, RedCoder is a timely and useful contribution: it addresses a realistic multi-turn setting, automates the pipeline without per-task human effort, and provides a reusable strategy arsenal with retrieval-augmented deployment. The OOD experiment in Appendix E.1 (65.00% on 20 held-out CWE types vs. 65.29% on the original set) is a genuine strength, as is the head-to-head comparison against four published baselines and the inclusion of Algorithm 1 for reproducibility. The paper does not currently provide code or per-task results, but the core idea is clear and falsifiable. The main risk is measurement validity: because the same detector (CodeGuru) serves as both training signal and evaluation oracle, the absolute rates and relative gains could partly reflect detector-specific optimization rather than genuine vulnerability induction.

major comments (3)
  1. [§2.2 and §3.1 (Evaluator and Evaluation Details)] The central evaluation metric, Vulnerability Rate, is computed by Amazon CodeGuru (§3.1), and CodeGuru is also the evaluator used during the gaming process to filter successful prototype conversations and to create the transition pairs used to distill attack strategies (§2.2). RedCoder is therefore trained and retrieval-augmented on CodeGuru-positive examples, while none of the baselines is optimized against this detector. Consequently, a systematic false-positive tendency of CodeGuru on the particular code patterns RedCoder learns to emit would inflate both the absolute rates in Table 1 and the relative gains over baselines. Appendix D validates CodeGuru on only 20 CodeGemma conversations with a single annotator; this sample is too small to establish precision on the adversarial distribution, and the two observed disagreements are CodeGuru false negatives, which do not directly address the false-positive concern. Please substantially expand the human-validated sample (across victim models and including conversations that CodeGuru flags), or replace the oracle with an independent detector plus human verification, and report Vulnerability Rates on human-confirmed snippets.
  2. [Table 1 and §3.1 (Main Results)] Table 1 reports each vulnerability rate as a point estimate with no confidence intervals, repeated runs, or significance tests. The headline comparisons (e.g., RedCoder 65.29% vs. ActorAttack 8.24% on Qwen2.5-Coder-7B) are large, but several smaller gaps (e.g., DeepSeek-R1-Distill-8B: RedCoder 40.00% vs. AutoDAN 22.49%) may be within sampling noise for 170 conversations. Please report bootstrap confidence intervals over the 170 tasks, or repeated runs with different random seeds, and a paired significance test (e.g., McNemar or permutation test) over the same task set for all methods.
  3. [§3.1 (Dataset) and Appendix E.1] The main benchmark includes the same 43 seed tasks used in the gaming process that generated RedCoder's fine-tuning conversations and strategy arsenal (§3.1). The authors state that the conversations are distinct because the defender differs from test-time victims, but the task instructions and target CWE types are shared, so RedCoder has an in-distribution advantage over the baselines on those tasks. The OOD experiment in Appendix E.1, which uses 20 disjoint CWE types and achieves 65.00% vs. 65.29%, is encouraging and largely answers this concern; however, it is confined to one victim model (Qwen2.5-Coder-7B) and reported only in the appendix. Please make the OOD results a primary result, or clearly separate in-distribution and OOD performance in the main table, and extend the OOD evaluation to at least one more victim model.
minor comments (6)
  1. [§3.4 and Table 2] The text says 'As shown in Tab. 8' when discussing the single-turn vs. multi-turn guardrail defense results, but Table 8 in Appendix G reports guardrail detection rates on 500 conversations, not the end-to-end defense vulnerability rates; the reference should point to Table 2.
  2. [§3.1 and Table 1] Model names are inconsistent: Table 1 uses 'Qwen-2.5-Coder-7B' and 'DeepSeek-R1-Distill-8B', while §3.1 uses 'Qwen-Coder-7B' and 'DeepSeek-R1-Distill-Llama-8B'; please unify the names across the paper.
  3. [§4, §B, and references] The related-work section cites AutoDAN as Liu et al. (2024), while the baseline implementation in Appendix B cites Liu et al. (2025) and links to AutoDAN-Turbo; please clarify which version is actually evaluated and cite it consistently.
  4. [Appendix D] Appendix D says 'A PhD student studying' without completing the sentence; also, the human-validation protocol should state whether the annotator saw the CodeGuru labels and how CWE-level disagreements were adjudicated.
  5. [§2.2 (Defender)] The guardrail decision is defined as arg max P(g|...), but the paper does not specify how P is obtained from the guardrail model; please provide the exact scoring function or logit extraction procedure.
  6. [Algorithm 1] Algorithm 1 line 14 says the evaluator detects 'CWE vulnerabilities or malicious cyberactivity', but malicious cyberactivity is not defined or used elsewhere in the paper; please remove or define this term.

Circularity Check

1 steps flagged · score 6.0 of 10

REDCODER's central vulnerability-induction metric is generated by the same Amazon CodeGuru detector used to select its training conversations; the only independent check is a 20-conversation human review.

  1. fitted input called prediction [Section 2.2 (Evaluator) + Algorithm 1 + Section 3.1 (Evaluation Details)]
    "For automated detection, we use Amazon CodeGuru, a production-grade vulnerability detector as our evaluator. Algorithm 1: 'Assign detection label d←1 if any vulnerability is detected; if d=1 then Save C as a prototype conversation.' Section 3.1: 'We use Amazon CodeGuru to detect security vulnerabilities in the generated code. Our primary evaluation metric is the Vulnerability Rate, defined as the proportion of conversations in which at least one response (r_i) contains code flagged with a CWE vulnerability.'"

    The same detector (CodeGuru) supplies both the training label and the test metric. Prototype conversations are retained only when CodeGuru assigns d=1, and REDCODER is fine-tuned on those conversations. The headline Vulnerability Rate then counts a conversation as successful exactly when CodeGuru flags a CWE. The model is therefore optimized to maximize CodeGuru flags, and the reported 'vulnerability induction' is CodeGuru-flag induction. The only external anchor, Appendix D, is a human review of 20 CodeGemma conversations (90% agreement), which is too small to establish that the optimized outputs are genuinely vulnerable rather than artifacts of CodeGuru's specific pattern.

full rationale

REDCODER's pipeline is otherwise self-contained: the multi-agent gaming process, prototype-conversation SFT, and RAG retrieval are described with concrete algorithms and external baselines. The citations to ThinkGuard (Wen et al., 2025) and Mo et al. (2025) are not load-bearing; ThinkGuard is retrained on the paper's own conversations, and the reasoning-robustness contrast is incidental. The central circularity concern is the CodeGuru feedback loop identified above: CodeGuru is used both to filter training prototypes and to score the primary metric, so the quantitative claim reduces to the detector's output. The paper attempts to break the loop with a 20-conversation human validation (Appendix D), which is a genuine external check, but the sample is far too small to certify the precision of CodeGuru on the full distribution of 20,000+ snippets. A larger independent human annotation or a second static/dynamic analyzer would be needed to establish that REDCODER's optimized prompts induce real vulnerabilities rather than detector-specific patterns. This is a partial circularity, not a total one: evaluation victims differ from the gaming defender, and the human agreement on 20 conversations provides some evidence against gross gaming. Score 6 reflects that the central claim's measurement, though externally anchored, is still dominated by the same oracle used in training.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim does not rest on a mathematical derivation, so the ledger captures the empirical and design assumptions. The main burden is the reliance on CodeGuru as both training signal and evaluation oracle, plus the reuse of seed tasks. The listed parameters (k, n, benchmark size, guardrail data) are manually chosen and define the evaluation setting. No new physical entities are introduced.

free parameters (4)
  • maximum conversation turns (k) = 5
    Manual choice in Section 2.2 and Section 3.1. Caps the length of multi-turn attacks and affects achievable success rates; results may shift with longer or shorter conversations.
  • gaming iterations per task (n) = 20
    Manual choice in Section 3.1. Determines the size of the prototype-conversation dataset (2,098 conversations) and the diversity of the strategy arsenal; the vulnerability rate is measured for this budget.
  • benchmark task set = 170 tasks (43 seed + augmented)
    Curated by the authors from 43 CWE seeds (Section 3.1). The main results use this set, and the 43 seed tasks also appear in the gaming process, a design choice that shapes both training and evaluation.
  • guardrail training conversations = 800
    Number of multi-turn conversations used to fine-tune the custom multi-turn guardrail (Appendix A). The guardrail is used as the defender during gaming, so its quality depends on this choice.
assumptions (4)
  • domain assumption Amazon CodeGuru's vulnerability detections are a valid proxy for CWE vulnerabilities in generated code.
    CodeGuru is used as both the evaluator in the gaming process and the success metric in evaluation; Appendix D provides only a 20-conversation human validation.
  • domain assumption Reusing the 43 seed tasks in the evaluation benchmark does not inflate RedCoder's scores.
    The paper argues the defender differs from test-time victims, so conversations are distinct (Section 3.1). However, the attacker has seen these task descriptions during gaming, and the out-of-distribution experiment only checks new CWEs for one model.
  • ad hoc to paper The custom multi-turn guardrail represents a realistic strong defense during gaming.
    The guardrail is trained on conversations from the same gaming framework (Appendix A), so it may overfit to RedCoder-style attacks, affecting the difficulty of the simulated defense.
  • domain assumption Any CWE flag from CodeGuru counts as a successful attack, regardless of severity or abstraction level.
    Described in Section C as a coarse-grained metric. This broad definition can inflate success rates by counting low-severity or ambiguous detections.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RedCoder: Automated Multi-Turn Red Teaming for Code LLMs." pith.science (2026). https://pith.science/paper/E2N27A5R

@misc{pith2026250722063,
  author       = {Pith},
  title        = {Pith review of: RedCoder: Automated Multi-Turn Red Teaming for Code LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E2N27A5R}},
  note         = {Machine review of arXiv:2507.22063}
}
read the original abstract

Large Language Models (LLMs) for code generation (i.e., Code LLMs) have demonstrated impressive capabilities in AI-assisted software development and testing. However, recent studies have shown that these models are prone to generating vulnerable or even malicious code under adversarial settings. Existing red-teaming approaches rely on extensive human effort, limiting their scalability and practicality, and generally overlook the interactive nature of real-world AI-assisted programming, which often unfolds over multiple turns. To bridge these gaps, we present RedCoder, a red-teaming agent that engages victim models in multi-turn conversation to elicit vulnerable code. The pipeline to construct RedCoder begins with a multi-agent gaming process that simulates adversarial interactions, yielding a set of prototype conversations and an arsenal of reusable attack strategies. We then fine-tune an LLM on these prototype conversations to serve as the backbone of RedCoder. Once deployed, RedCoder autonomously engages Code LLMs in multi-turn conversations, dynamically retrieving relevant strategies from the arsenal to steer the dialogue toward vulnerability-inducing outputs. Experiments across multiple Code LLMs show that our approach outperforms prior single-turn and multi-turn red-team methods in inducing vulnerabilities in code generation, offering a scalable and effective tool for evaluating the security boundaries of modern code-generation systems.

Figures

Figures reproduced from arXiv: 2507.22063 by the authors.

Figure 1
Figure 1. REDCODER begins with benign prompts and adaptively steers the conversation based on the victim’s responses (highlighted), ultimately inducing the model to generate vulnerable code. that of human developers (Roziere et al., 2023; Nam et al., 2024; Wang and Chen, 2023). As Code LLMs become integrated into critical stages of soft￾ware engineering pipelines, ensuring the reliability and safety of their outputs is essent… view at source ↗
Figure 2
Figure 2. To build REDCODER, we use a multi-agent gaming process to generate (1) prototype conversations and (2) a strategy arsenal. We fine-tune a red-team LLM on the prototype conversations to serve as the backbone of REDCODER. At deployment, a Retrieval-Augmented Generation (RAG) mechanism enhances attack effectiveness and adaptability by retrieving strategies from the arsenal. utterance based on the conversation history t… view at source ↗
Figure 3
Figure 3. When the defender declines to respond to [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: All retrieval variants yield positive improve [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SoK: Intent-Oriented Systematization of Multi-Turn LLM Jailbreaks

    cs.CR 2026-08 conditional novelty 7.0 of 10

    Multi-turn LLM jailbreaks succeed based on how harmful intent is organized across turns, not on interaction length, and detection should shift to session and cross-session scope.

  2. Code Monitor Red Teaming for Public-Test-Passing Code

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Weak LLM verifiers miss most hidden bugs in public-test-passing code, and adversarial pressure makes that monitoring worse.

Reference graph

Works this paper leans on

56 extracted references · 28 canonical work pages · cited by 2 Pith papers

  1. [1]

    Hojjat Aghakhani, Wei Dai, Andre Manoel, Xavier Fernandes, Anant Kharkar, Christopher Kruegel, Giovanni Vigna, David Evans, Ben Zorn, and Robert Sim. 2024. Trojanpuzzle: Covertly poisoning code-suggestion models. In 2024 IEEE Symposium on Security and Privacy (SP), pages 1122--1140. IEEE

    work page 2024

  2. [2]

    AWS . 2020. Coursera uses codeguru profiler to optimize performance. Available at AWS Blog, accessed 2025-06-27

    work page 2020

  3. [3]

    AWS . 2022. Atlassian uses codeguru profiler for production monitoring. Available at AWS Blog, accessed 2025-06-27

    work page 2022

  4. [4]

    AWS . 2023. How devfactory builds better applications with codeguru. Available at AWS Blog, accessed 2025-06-27

    work page 2023

  5. [5]

    AWS . 2025. Amazon CodeGuru Security—Reduce false-positive detections . https://aws.amazon.com/codeguru. Accessed: 2025-06-27

    work page 2025

  6. [6]

    Domenico Cotroneo, Cristina Improta, Pietro Liguori, and Roberto Natella. 2024. Vulnerabilities in ai code generators: Exploring targeted data poisoning attacks. In Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension, pages 280--292

    work page 2024

  7. [7]

    Arghavan Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Michel C Desmarais, and Zhen Ming Jack Jiang. 2023. Github copilot ai pair programmer: Asset or liability? Journal of Systems and Software, 203:111734

    work page 2023

  8. [8]

    Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2024. https://openreview.net/forum?id=vESNKdEMGp Multilingual jailbreak challenges in large language models . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    work page 2024

Show all 56 references
  1. [9]

    Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. 2023. Mart: Improving llm safety with multi-round automatic red-teaming. arXiv preprint arXiv:2311.07689

    2023 arXiv

  2. [10]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  3. [11]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  4. [12]

    Jingxuan He and Martin Vechev. 2023. Large language models for code: Security hardening and adversarial testing. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 1865--1879

    2023

  5. [13]

    John Heibel and Daniel Lowd. 2024. Mapping your model: Assessing the impact of adversarial attacks on llm-based programming assistants. arXiv preprint arXiv:2407.11072

    2024 arXiv

  6. [14]

    Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel. 2017. Adversarial attacks on neural network policies. arXiv preprint arXiv:1702.02284

    2017 arXiv

  7. [15]

    Yujin Huang, Terry Yue Zhuo, Qiongkai Xu, Han Hu, Xingliang Yuan, and Chunyang Chen. 2023. Training-free lexical backdoor attacks on language models. In Proceedings of the ACM Web Conference 2023, pages 2198--2208

    2023

  8. [16]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186

    2024 arXiv

  9. [17]

    Cristina Improta. 2023. Poisoning programs by un-repairing code: Security concerns of ai-generated code. In 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW), pages 128--131. IEEE

    2023

  10. [18]

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674

    2023 arXiv

  11. [19]

    Arnav Kumar Jain, Gonzalo Gonzalez-Pumariega, Wayne Chen, Alexander M Rush, Wenting Zhao, and Sanjiban Choudhury. 2025. Multi-turn code generation through single-step rewards. In ICLR 2025 Workshop: VerifAI: AI Verification in the Wild

    2025

  12. [20]

    Slobodan Jenko, Jingxuan He, Niels M \"u ndler, Mark Vero, and Martin T Vechev. 2024. Practical attacks against black-box code completion engines. CoRR

    2024

  13. [21]

    Slobodan Jenko, Niels M \"u ndler, Jingxuan He, Mark Vero, and Martin Vechev. 2025. Black-box adversarial attacks on llm-based code completion. In ICLR 2025 Workshop on Building Trust in Language Models and Applications

    2025

  14. [22]

    Akshita Jha and Chandan K Reddy. 2023. Codeattack: Code-based adversarial attacks for pre-trained programming language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14892--14900

    2023

  15. [23]

    Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.809 A rt P rompt: ASCII art-based jailbreak attacks against aligned LLM s . In Proceedings of the 62nd Annual Meeting o...

    2024 doi

  16. [24]

    Yifan Jiang, Kriti Aggarwal, Tanmay Laud, Kashif Munir, Jay Pujara, and Subhabrata Mukherjee. 2024 b . https://doi.org/10.48550/ARXIV.2409.17458 RED QUEEN: safeguarding large language models against concealed multi-turn jailbreaking . CoRR, abs/2409.17458

    2024 doi arXiv

  17. [25]

    Yanzhou Li, Shangqing Liu, Kangjie Chen, Xiaofei Xie, Tianwei Zhang, and Yang Liu. 2023. Multi-target backdoor attacks for code pre-trained models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7236--7254

    2023

  18. [26]

    Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei Xiao. 2025. Autodan-turbo: A lifelong agent for strategy self-exploration to jailbreak llms. ICLR

    2025

  19. [27]

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. Autodan: Generating stealthy jailbreak prompts on aligned large language models. ICLR

    2024

  20. [28]

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. 2024. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In ICML

    2024

  21. [29]

    Anderson, Yaron Singer, and Amin Karbasi

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum S. Anderson, Yaron Singer, and Amin Karbasi. 2024. http://papers.nips.cc/paper\_files/paper/2024/hash/70702e8cbb4890b4a467b984ae59828a-Abstract-Conference.html Tree of attacks: Jailbreaking black-box llms ...

    2024

  22. [30]

    The MITRE. 2025. Common weakness enumeration. https://cwe.mitre.org/

    2025

  23. [31]

    Wenjie Mo, Jiashu Xu, Qin Liu, Jiongxiao Wang, Jun Yan, Chaowei Xiao, and Muhao Chen. 2025. Test-time backdoor mitigation for black-box large language models with defensive demonstrations. Findings of NAACL

    2025

  24. [32]

    Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an llm to help with code understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pages 1--13

    2024

  25. [33]

    Thanh-Dat Nguyen, Yang Zhou, Xuan Bach D Le, Patanamon Thongtanunam, and David Lo. 2023. Adversarial attacks on code models with discriminative graph patterns. arXiv preprint arXiv:2308.11161

    2023 arXiv

  26. [34]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203.13474

    2022 arXiv

  27. [35]

    Mary Branscombe O'Neill. 2024. Ai-generated code can cause outages — and developers need better tools to prevent them. https://www.techrepublic.com/article/ai-generated-code-outages/. Accessed: 2025-04-29

    2024

  28. [36]

    OpenAI. 2024. Gpt-4o: Openai’s newest model. https://openai.com/index/gpt-4o

    2024

  29. [37]

    Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2025. Asleep at the keyboard? assessing the security of github copilot’s code contributions. Communications of the ACM, 68(2):96--105

    2025

  30. [38]

    Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590

    2023 arXiv

  31. [39]

    Qibing Ren, Chang Gao, Jing Shao, Junchi Yan, Xin Tan, Wai Lam, and Lizhuang Ma. 2024 a . Codeattack: Revealing safety generalization challenges of large language models via code completion. arXiv preprint arXiv:2403.07865

    2024 arXiv

  32. [40]

    Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao. 2024 b . Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues. arXiv preprint arXiv:2410.10700

    2024

  33. [41]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950

    2023 arXiv

  34. [42]

    Mark Russinovich, Ahmed Salem, and Ronen Eldan. 2024. https://doi.org/10.48550/ARXIV.2404.01833 Great, now write an article about that: The crescendo multi-turn LLM jailbreak attack . CoRR, abs/2404.01833

    2024 doi arXiv

  35. [43]

    CodeGemma Team, Heri Zhao, Jeffrey Hui, Joshua Howland, Nam Nguyen, Siqi Zuo, Andrea Hu, Christopher A Choquette-Choo, Jingyue Shen, Joe Kelley, et al. 2024. Codegemma: Open code models based on gemma. arXiv preprint arXiv:2406.11409

    2024 arXiv

  36. [44]

    Jianxun Wang and Yixiang Chen. 2023. A review on code generation with llms: Application and evaluation. In 2023 IEEE International Conference on Medical Artificial Intelligence (MedAI), pages 284--289. IEEE

    2023

  37. [45]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672

    2024 arXiv

  38. [46]

    Xiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo, and Muhao Chen. 2025. Thinkguard: Deliberative slow thinking leads to cautious guardrails. arXiv preprint arXiv:2502.13458

    2025 arXiv

  39. [47]

    Michel Wermelinger. 2023. Using github copilot to solve simple programming problems. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, pages 172--178

    2023

  40. [48]

    Fangzhou Wu, Xiaogeng Liu, and Chaowei Xiao. 2023. Deceptprompt: Exploiting llm-driven code generation via adversarial natural language instructions. arXiv preprint arXiv:2312.04730

    2023 arXiv

  41. [49]

    Kankanhalli

    Xilie Xu, Keyi Kong, Ning Liu, Lizhen Cui, Di Wang, Jingfeng Zhang, and Mohan S. Kankanhalli. 2024. https://openreview.net/forum?id=VVgGbB9TNV An LLM can fool itself: A prompt-based adversarial attack . In The Twelfth International Conference on Learning Representations, ICLR ...

    2024

  42. [50]

    Hao Yang, Lizhen Qu, Ehsan Shareghi, and Gholamreza Haffari. 2024 a . https://doi.org/10.48550/ARXIV.2410.11459 Jigsaw puzzles: Splitting harmful questions to jailbreak large language models . CoRR, abs/2410.11459

    2024 doi arXiv

  43. [51]

    Xikang Yang, Xuehai Tang, Songlin Hu, and Jizhong Han. 2024 b . Chain of attack: a semantic-driven contextual multi-turn attacker for llm. arXiv preprint arXiv:2405.05610

    2024 arXiv

  44. [52]

    Jinchuan Zhang, Yan Zhou, Yaxin Liu, Ziming Li, and Songlin Hu. 2024. Holistic automated red teaming for large language models through top-down test case generation and multi-turn interaction. arXiv preprint arXiv:2409.16783

    2024 arXiv

  45. [53]

    Kunhao Zheng, Juliette Decugis, Jonas Gehring, Taco Cohen, Benjamin Negrevergne, and Gabriel Synnaeve. 2024. What makes large language models reason in (multi-turn) code generation? arXiv preprint arXiv:2410.08105

    2024 arXiv

  46. [54]

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043

    2023 arXiv

  47. [55]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.