Pith. sign in

REVIEW 4 major objections 6 minor 58 references

Poisoned code manuals make AI coding assistants install fake packages

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 21:28 UTC pith:SURLPLRW

load-bearing objection A credible and novel attack that is oversold by its abstract; the same-retriever evaluation and weak transferability keep it from being a proven real-world threat, but it deserves a serious referee. the 4 major comments →

arxiv 2509.07941 v1 pith:SURLPLRW submitted 2025-09-09 cs.CR cs.AI

ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation

Kai Ye , Liangcai Su , Chenxiong Qian This is my paper
classification cs.CR cs.AI
keywords Retrieval-Augmented Code GenerationRAG poisoningdependency hijackingsupply chain attackcode generation securityposition-aware beam searchprompt injectionmalicious packages
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that Retrieval-Augmented Code Generation (RACG) systems, which let AI coding assistants pull in external code manuals, can be hijacked by planting poisoned documentation in the manual corpus. An attacker who controls a handful of pages (as few as 0.01% of the database) can make the assistant recommend a package name of the attacker's choosing, such as a 'safe' variant of matplotlib. The attack works without knowing the user's exact query, because the poisoned documents are engineered to rank high for any plausible query. It combines a ranking sequence of optimized gibberish tokens that boosts retrieval with a multilingual code comment that persuades the LLM to switch dependencies. If the results are correct, the trust developers place in AI code suggestions becomes a practical supply-chain entry point.

Core claim

The paper demonstrates a directed dependency-hijacking attack on RAG-based code generation. Its formal goal is that for every user query, at least one poisoned document appears in the top-k retrieval set, and that the LLM then recommends the attacker's package. ImportSnare constructs these documents by splicing two sequences into clean code snippets: a Ranking Sequence of gibberish tokens optimized line-by-line with gradient-guided beam search to raise embedding similarity to proxy queries, and an Inducing Sequence of multilingual comments, generated by LLMs and selected by teacher-forcing probability, that recommends the fake dependency in harmless-sounding language. On Python, Rust, and Ja

What carries the argument

Position-aware beam search (ImportSnare-R) and multilingual inductive suggestions (ImportSnare-G). The ranking sequence is a short token string inserted at line ends; a HotFlip-style gradient approximation over the embedding matrix chooses token replacements that maximize cosine similarity between poisoned document and proxy query embeddings, with beams initialized from each line position so insertion point and tokens optimize jointly. The inducing sequence is a comment in one of eight languages, assembled from 'switch to X' suggestions written by several LLMs and filtered by teacher-forced generation probability of the target package name. A shorter re-search round then repairs the ranking

Load-bearing premise

The attack depends on the gibberish ranking sequence computed with a local embedding model still moving poisoned documents into the top-k when the production RAG system uses a different retriever, possibly with reranking; the paper's own cross-retriever table shows sharp drops, and any pipeline that diverges enough from the proxy embedding would stop the attack before the LLM ever sees the poisoned text.

What would settle it

Run the paper's benchmark queries and target packages through a production RAG pipeline whose retriever is any embedding model plus a reranker not in the paper's proxy set (gte-base-en-v1.5, all-mpnet-base-v2, bge-base-en-v1.5, e5-base-v2), then measure Precision@k for poisoned documents and ASR. If Precision@k falls to the level of a clean corpus and ASR returns to zero, the transferability premise is refuted; a simpler probe is checking whether the optimized ranking sequences are low-frequency gibberish tokens that a token-frequency filter can strip away without degrading retrieval.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the results hold, any RAG-based code assistant drawing its corpus from user-contributed content (e.g., GitHub, Stack Overflow, forums) can be steered toward attacker-chosen dependencies without the attacker knowing the user's query.
  • A poisoning ratio of 0.01% means only a small number of injected pages may suffice in realistic corpora, and attackers can plant documents before even uploading the malicious package.
  • Because 'safe', 'v2', and 'robust' suffixes achieve high attack success while obvious typosquats are often corrected, the attack exploits LLM priors about trustworthy naming rather than user typing errors.
  • LLM-based detection of poisoned documents, prompts, and outputs is partial and costly, so the paper's own mitigation results indicate no reliable defense currently exists.
  • The same framework generalizes across Python, Rust, and JavaScript and across LLMs from different families, suggesting the vulnerability is not tied to one model or ecosystem.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If ranking sequences transfer through a reranker as they transfer across embedding models, the same beam-search poisoning could apply to non-code RAG domains such as API documentation or configuration lookup; testing ImportSnare-R against a production reranker is a direct next step.
  • The LLM self-paraphrase step hints at a general weakness: suggestions phrased by LLMs in their own idiom are more persuasive to other LLMs, which could be exploited to benchmark safety alignment outside code generation.
  • A cheap defense to test is filtering retrieved documents by embedding-neighborhood anomaly, since the optimized ranking sequence is deliberately gibberish and likely occupies an isolated region of embedding space; whether such gibberish survives rerankers remains open.
  • The paper's finding that typosquatting is a weakening vector suggests future package-registry defenses should focus on 'enhanced version' naming patterns (safe/v2/full) rather than only near-miss spellings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces ImportSnare, a framework for poisoning documentation in Retrieval-Augmented Code Generation (RACG) so that victim LLMs recommend attacker-chosen malicious dependency packages. The method has two components: ImportSnare-R, which uses position-aware beam search to create low-visibility 'ranking sequences' that increase the embedding similarity between poisoned documents and proxy queries, and ImportSnare-G, which crafts multilingual, security-flavored 'inducing sequences' inserted as code comments to steer the LLM's import suggestions. The evaluation covers Python, Rust, and JavaScript, multiple LLMs (GPT-4o, DeepSeek-v3/R1, Claude, etc.), several retrievers, baselines, ablations, poisoning-ratio sweeps, and code-quality comparisons. The authors report high ASR for some target packages (e.g., matplotlib_safe 0.677 on GPT-4o-mini), low required poisoning ratios (0.01% of the whole database), and release plans for a benchmark suite.

Significance. The attack scenario is realistic and important: if poisoned documentation can consistently redirect RAG-based code assistants to malicious packages, this is a genuine supply-chain risk for a growing user population. The paper has clear strengths: it requires no white-box access to the target models, holds out test queries from the proxy split, tests three languages and several LLMs, compares against baselines, includes ablations, and attempts transferability analysis. However, the strongest quantitative evidence is limited to the same embedding model (gte-base-en-v1.5) for both optimization and evaluation, and the cross-retriever data show sharp degradation. The abstract's 'over 50%' claim is also not representative of Table 1. If the retriever-transferability gap is addressed or the claims are appropriately narrowed, the work would be a valuable contribution to the security analysis of LLM-powered development tools.

major comments (4)
  1. [Abstract / Section 6.1 / Table 1] The abstract states that ImportSnare 'achieves significant attack success rates (over 50% for popular libraries such as matplotlib and seaborn) in general.' This overgeneralizes Table 1. For many popular targets the ASR is far below 50%: numpy max 0.209, requests 0.125, moment 0.186, uuid 0.103, sklearn 0.452, and most JavaScript targets are under 0.2. The 50%+ numbers are concentrated in specific package/LLM combinations (e.g., matplotlib_safe, scraper, regex_safe on some models). Please report aggregate statistics (mean/median over targets per model) and qualify the abstract claim so it reflects the actual distribution, not selected high-performing cells.
  2. [Section 4.3 / 5.4 / 6.6 / Table 6] The ranking sequences are optimized against gte-base-en-v1.5 and the headline Precision@k and ASR in Table 1 are computed with the same retriever. Table 6 shows that this does not transfer to alternative retrievers: scipy ASR drops from 0.235 to 0.000 on all-mpnet-base-v2 and e5-base-v2, sklearn ASR drops from 0.452 to 0.000 on bge-base-en-v1.5 and e5-base-v2, and sklearn Precision@k drops from 7.39 to 2.61. Real RAG deployments often use different embedding models and add rerankers, which Section 7.2 itself admits 'may degrade the transferability of adversarial perturbations optimized on simplified proxies.' The end-to-end ASR in Table 1 therefore does not establish practicality for production RAG. Please add an evaluation with a held-out retriever and at least one reranking pipeline, or explicitly restrict the threat model to configurations where the victim uses the same retriever as t
  3. [Section 6.1 / Table 4] There is an internal inconsistency between Table 1 and Table 4, both labeled GPT-4o-mini with k=10. For matplotlib_safe, Table 1 reports ASR 0.677; Table 4 reports 0.274. Similarly, pandas_v2 is 0.360 vs 0.163, and seaborn_safe is 0.333 vs 0.200. This suggests different experimental settings (e.g., different test-query splits, different poisoned-document sets, or a different prompt) that are not documented. The discrepancy undermines the reproducibility of the headline results; please clarify the exact protocol and, if appropriate, unify the two tables.
  4. [Section 2.3 / Section 5.3 / Section 6.3] The formal attack goal in Section 2.3 is universal over the query set: 'for all q in Q, there exists a' in A where a' is in TopK(q, Pn U A, k).' The empirical metric ASR, however, measures the proportion of test queries for which the LLM's response contains the target library. The paper does not demonstrate the universal guarantee, and the abstract's 'over 50%' is an average, not a coverage statement. Moreover, the 0.01% poisoning-ratio claim in Section 6.3 is demonstrated for a single target (matplotlib_safe) and a single LLM (DeepSeek-v3). Please clarify how the formal goal relates to the empirical metric, and either broaden the low-ratio experiment or narrow the corresponding claim.
minor comments (6)
  1. [References] References [25] and [26] are the same paper ('Backstabber's knife collection'), and references [28] and [29] are also the same ('Neural Exec'). Please merge the duplicates.
  2. [Table 7] The column header 'Franch' is a typo for 'French'.
  3. [Section 4.1 / 4.3] Typographical errors: 'Incuding Sequence' in Section 4.1 should be 'Inducing Sequence'; 'The gold of ImportSnare-R' in Section 4.3 should be 'The goal of ImportSnare-R'.
  4. [Section 5.5] The sentence 'For the local proxy LLM, we use LLama3.2-3B by default. For the local proxy LLM, we use GPT4o-mini by default.' is contradictory; the second instance should presumably refer to the model used to generate inductive suggestions or to the target LLM. Please correct.
  5. [Table 1] Several seaborn rows (seaborn_safe, robust_seaborn, malware_seaborn, seaborn_v2) are missing the poisoning-ratio and #Queries columns. Please fill in the values or add a footnote explaining why they are omitted.
  6. [Section 5.4] The heading 'Contriever' is used for the retrieval models, but the listed models are mGTE, bge-base-en-v1.5, all-mpnet-base-v2, and e5-base-v2. Since 'Contriever' is itself a specific retriever, the terminology is confusing. Also, reference [3] appears to be for M3-embedding, not for bge-base-en-v1.5; please verify the citation.

Circularity Check

0 steps flagged

No significant circularity: ImportSnare's derivation is an empirical optimization with held-out test queries and explicit cross-retriever transferability checks; the only self-citation is non-load-bearing.

full rationale

The paper's derivation chain is not circular. The core attack constructs poisoned documents by optimizing a ranking sequence against a proxy retriever (gte-base-en-v1.5) via gradient-guided token replacement (Eq. 1, Algorithm 1) and selects inducing suggestions by teacher-forcing probability on a proxy LLM (Eq. 4). The reported Precision@k and ASR are computed on a held-out 20% test-query split (§5.2) with the same retriever/LLM families; this is a same-model generalization test, not a definitional identity. The retrieval objective is average cosine similarity over proxy queries, while Precision@k is a top-k hit rate over disjoint test queries, so success is not forced by construction. The paper's own §6.6 reports cross-retriever transferability with degraded numbers (e.g., scipy ASR drops from 0.235 to 0.000 on all-mpnet-base-v2 and e5-base-v2), and §7.2 explicitly acknowledges that reranking mechanisms may degrade transferability of perturbations optimized on simplified proxies. This transparency shows the evaluation is not hiding a fit. The only self-citation is [48] in §2.3, used as a passing comparison to training-data perturbation; it is not load-bearing for any central claim. No uniqueness theorem, ansatz, or renamed known result is invoked. Therefore no circular step meets the evidentiary bar of the review rules; the same-retriever evaluation is a methodological limitation (overfitting risk) but not circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The attack rests on several domain assumptions about real-world RAG systems and developers, largely stated in the threat model: document injection, proxy-query fidelity, black-box access to the embedding model, and LLM susceptibility to multilingual code-comment suggestions. The only fitted numbers are the beam search hyperparameters, chosen by hand and shown to control attack strength. No new physical or theoretical entities are introduced.

free parameters (2)
  • Beam search hyperparameters = L=20, B=10, kb=15, N=50, N'=25
    Chosen by the authors, balancing performance and compute cost; larger values improve attack effectiveness (Section 6.4, Figure 6), so the reported ASR depends on this hand-tuned configuration.
  • Top-k retrieval depth = k=10 (default; ablated to 5 and 20)
    Set as a common RAG context size; Table 4 shows ASR varies with k, so the headline numbers depend on this choice.
axioms (5)
  • domain assumption Attacker can inject poisoned documents into the RAG corpus (e.g., via crawled GitHub/StackOverflow content).
    Stated in the threat model (Section 3); the entire attack requires the ability to add documents to the database without being removed.
  • domain assumption Proxy queries used for poisoning approximate the real user query distribution.
    Stated in Section 4.2; if real queries differ, poisoned documents will not rank high.
  • domain assumption The attacker can query the embedding model (black-box) used for ranking, and its embedding geometry is accessible for HotFlip-style gradient optimization.
    Algorithm 1 uses gradient information; the main evaluations use the same encoder (gte-base-en-v1.5) that was queried during construction, so transferability to a different production retriever is not guaranteed.
  • domain assumption LLMs follow multilingual code-comment suggestions to switch to the target package.
    Core premise of ImportSnare-G; empirically true for most targets but not for typo-squatted names like requstss, for which ASR is 0 on many models.
  • standard math First-order Taylor approximation of the embedding similarity (as in HotFlip) is valid for token substitution.
    Appendix A.1 uses a standard gradient approximation to select replacement tokens.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation." pith.science (2026). https://pith.science/paper/SURLPLRW

@misc{pith2026250907941,
  author       = {Pith},
  title        = {Pith review of: ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SURLPLRW}},
  note         = {Machine review of arXiv:2509.07941}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Code generation has emerged as a pivotal capability of Large Language Models(LLMs), revolutionizing development efficiency for programmers of all skill levels. However, the complexity of data structures and algorithmic logic often results in functional deficiencies and security vulnerabilities in generated code, reducing it to a prototype requiring extensive manual debugging. While Retrieval-Augmented Generation (RAG) can enhance correctness and security by leveraging external code manuals, it simultaneously introduces new attack surfaces. In this paper, we pioneer the exploration of attack surfaces in Retrieval-Augmented Code Generation (RACG), focusing on malicious dependency hijacking. We demonstrate how poisoned documentation containing hidden malicious dependencies (e.g., matplotlib_safe) can subvert RACG, exploiting dual trust chains: LLM reliance on RAG and developers' blind trust in LLM suggestions. To construct poisoned documents, we propose ImportSnare, a novel attack framework employing two synergistic strategies: 1)Position-aware beam search optimizes hidden ranking sequences to elevate poisoned documents in retrieval results, and 2)Multilingual inductive suggestions generate jailbreaking sequences to manipulate LLMs into recommending malicious dependencies. Through extensive experiments across Python, Rust, and JavaScript, ImportSnare achieves significant attack success rates (over 50% for popular libraries such as matplotlib and seaborn) in general, and is also able to succeed even when the poisoning ratio is as low as 0.01%, targeting both custom and real-world malicious packages. Our findings reveal critical supply chain risks in LLM-powered development, highlighting inadequate security alignment for code generation tasks. To support future research, we will release the multilingual benchmark suite and datasets. The project homepage is https://importsnare.github.io.

Figures

Figures reproduced from arXiv: 2509.07941 by Chenxiong Qian, Kai Ye, Liangcai Su.

Figure 1
Figure 1. Figure 1: Attack chain of our proposed directed dependency hijacking risk in RACG. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Workflow of our proposed ImportSnare. added to the corpus can be consistently retrieved by RAG systems, and (2) their ability to override the LLM’s internal knowledge (rep￾resented by model weights). We assume the model’s training data (encoded in weights) contains only clean library and package names. Drawing inspiration from [57], our approach involves inserting two distinct adversarial sequences: Rankin… view at source ↗
Figure 3
Figure 3. Figure 3: Pipeline of choosing to-be-poisoned documents. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance of different poisoning percentages of [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation performance of modules in our Import￾Snare. R denotes ImportSnare-R. G denotes ImportSnare-G. R+G and G+R mean different orders. R(N+N’) indicates the incorporation of additional reconstruction iterations from the second-round ranking sequence into the first round. 1 6 10 15 L 0.275 0.300 0.325 0.350 0.375 0.400 0.425 0.450 0.475 ASR 5 10 15 20 B 5 10 15 20 kb 0 2 4 6 8 10 Precision@k ASR Precisio… view at source ↗
Figure 6
Figure 6. Figure 6: Ablation of hyperparameters in our ImportSnare-R. 6.5 Influences on Code Generation Quality We evaluate the effects of poisoned RAG databases by comparing code outputs generated using both poisoned and clean documenta￾tion sources in [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Mitigation results. Documents, Prompt and Re [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: A real-world demo demonstrating how Copilot in VSCode could be misled into suggesting a hijacked package ( [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: A real-world demo demonstrating how Cursor Agent code suggestion includes hijacked package ( [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: A real-world demo demonstrating that Tencent Yuanbao web-chat LLM (DeepSeek-r1) provides help regarding debug [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 20 canonical work pages · 1 internal anchor

  1. [1]

    2023.{Fact-Saboteurs}: A taxonomy of evi- dence manipulation attacks against{Fact-Verification} systems

    Sahar Abdelnabi and Mario Fritz. 2023.{Fact-Saboteurs}: A taxonomy of evi- dence manipulation attacks against{Fact-Verification} systems. In32nd USENIX Security Symposium (USENIX Security 23). 6719–6736

    work page 2023

  2. [2]

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program Synthesis with Large Language Models.arXiv preprint arXiv:2108.07732(2021)

    Pith/arXiv arXiv 2021

  3. [3]

    Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

  4. [4]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374(2021)

    Pith/arXiv arXiv 2021

  5. [5]

    Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar, Kamalika Chaudhuri, and Chuan Guo. 2024. Aligning llms to be robust against prompt injection.arXiv preprint arXiv:2410.05451(2024)

    Pith/arXiv arXiv 2024

  6. [6]

    Pengzhou Cheng, Yidong Ding, Tianjie Ju, Zongru Wu, Wei Du, Ping Yi, Zhu- osheng Zhang, and Gongshen Liu. 2024. Trojanrag: Retrieval-augmented gen- eration can be backdoor driver in large language models.arXiv preprint arXiv:2405.13401(2024)

    Pith/arXiv arXiv 2024

  7. [7]

    Wen Cheng, Ke Sun, Xinyu Zhang, and Wei Wang. 2025. Security Attacks on LLM-based Code Completion Tools. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 23669–23677

    work page 2025

  8. [8]

    Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2023. Classeval: A manually- crafted benchmark for evaluating llms on class-level code generation.arXiv preprint arXiv:2308.01861(2023)

    Pith/arXiv arXiv 2023

  9. [9]

    Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. Hotflip: White- box adversarial examples for text classification.arXiv preprint arXiv:1712.06751 (2017)

    Pith/arXiv arXiv 2017

  10. [10]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models.arXiv preprint arXiv:2407.21783 (2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    Pith/arXiv arXiv 2025

  12. [12]

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring Coding Challenge Competence With APPS.NeurIPS (2021)

    2021

  13. [13]

    Yihao Huang, Chong Wang, Xiaojun Jia, Qing Guo, Felix Juefei-Xu, Jian Zhang, Geguang Pu, and Yang Liu. 2024. Efficient Universal Goal Hijacking with Semantics-guided Prompt Organization.arXiv preprint arXiv:2405.14189(2024)

    Pith/arXiv arXiv 2024

  14. [14]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186(2024)

    Pith/arXiv arXiv 2024

  15. [15]

    Fengqing Jiang, Zhangchen Xu, Luyao Niu, Boxin Wang, Jinyuan Jia, Bo Li, and Radha Poovendran. 2024. POSTER: identifying and mitigating vulnerabilities in llm-integrated applications. InProceedings of the 19th ACM Asia Conference on Computer and Communications Security. 1949–1951

    work page 2024

  16. [16]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation.arXiv preprint arXiv:2406.00515 (2024)

    Pith/arXiv arXiv 2024

  17. [17]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering.. InEMNLP (1). 6769–6781

    2020

  18. [18]

    Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettle- moyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. DS-1000: A natural and reliable benchmark for data science code generation. InInternational Conference on Machine Learning. PMLR, 18319–18345

    2023

  19. [19]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems33 (2020), 9459–9474

    2020

  20. [20]

    Genpei Liang, Xiangyu Zhou, Qingyu Wang, Yutong Du, and Cheng Huang. 2021. Malicious packages lurking in user-friendly python package index. In2021 IEEE 20th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom). IEEE, 606–613

    work page 2021

  21. [21]

    Bo Lin, Shangwen Wang, Liqian Chen, and Xiaoguang Mao. 2025. Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation.arXiv preprint arXiv:2502.03233(2025)

    Pith/arXiv arXiv 2025

  22. [22]

    Xiaogeng Liu, Zhiyuan Yu, Yizhe Zhang, Ning Zhang, and Chaowei Xiao. 2024. Automatic and universal prompt injection attacks against large language models. arXiv preprint arXiv:2403.04957(2024)

    Pith/arXiv arXiv 2024

  23. [23]

    Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, et al. 2023. Prompt injection attack against llm-integrated applications.arXiv preprint arXiv:2306.05499(2023)

    Pith/arXiv arXiv 2023

  24. [24]

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct.arXiv preprint arXiv:2306.08568(2023)

    Pith/arXiv arXiv 2023

  25. [26]

    Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. 2020. Backstab- ber’s knife collection: A review of open source software supply chain attacks. In Detection of Intrusions and Malware, and Vulnerability Assessment: 17th Interna- tional Conference, DIMV A 2020, Lisbon, Portugal, June 24–26, 2020, Proceedings 17. Springer, 23–43

    work page 2020

  26. [27]

    Soumen Pal, Manojit Bhattacharya, Sang-Soo Lee, and Chiranjib Chakraborty

    work page

  27. [29]

    A domain-specific next-generation large language model (LLM) or Chat- GPT is required for biomedical engineering and research.Annals of biomedical engineering52, 3 (2024), 451–454

    work page 2024

  28. [30]

    Anselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos, and Yuan- dong Tian. 2024. Advprompter: Fast adaptive adversarial prompting for llms. arXiv preprint arXiv:2404.16873(2024)

    Pith/arXiv arXiv 2024

  29. [31]

    Dario Pasquini, Martin Strohmeier, and Carmela Troncoso. 2024. Neural exec: Learning (and learning from) execution triggers for prompt injection attacks. In Proceedings of the 2024 Workshop on Artificial Intelligence and Security. 89–100

    work page 2024

  30. [32]

    Code Manual

    Piotr Przymus and Thomas Durieux. 2025. Wolves in the Repository: A Software Engineering Analysis of the XZ Utils Supply Chain Attack. In2025 IEEE/ACM 22nd International Conference on Mining Software Repositories (MSR). IEEE, 91–102. ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation Conference ’25, June 03–05, 2018, Wood...

    work page 2025

  31. [33]

    Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models.arXiv preprint arXiv:2211.09527(2022)

    Pith/arXiv arXiv 2022

  32. [34]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiao- qing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950 (2023)

    Pith/arXiv arXiv 2023

  33. [35]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks.arXiv preprint arXiv:1908.10084(2019)

    Pith/arXiv arXiv 2019

  34. [36]

    Avital Shafran, Roei Schuster, and Vitaly Shmatikov. 2024. Machine against the rag: Jamming retrieval-augmented generation with blocker documents.arXiv preprint arXiv:2406.05870(2024)

    Pith/arXiv arXiv 2024

  35. [37]

    Kunal Sankhe, Mauro Belgiovine, Fan Zhou, Shamnaz Riyaz, Stratis Ioannidis, and Kaushik Chowdhury. 2019. ORACLE: Optimized Radio clAssification through Convolutional neuraL nEtworks. InIEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 370–378

    work page 2019

  36. [38]

    Alexander Wan, Eric Wallace, and Dan Klein. 2024. What evidence do language models find convincing?arXiv preprint arXiv:2402.11782(2024)

    Pith/arXiv arXiv 2024

  37. [39]

    The Pragmatic Engineer. 2023. The Pulse 134: Engineering Leadership. https: //newsletter.pragmaticengineer.com/p/the-pulse-134?utm_campaign=post&ut m_medium=web. Accessed: 2025-07-09

    work page 2023

  38. [40]

    Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021. Codet5: Identifier- aware unified pre-trained encoder-decoder models for code understanding and generation.arXiv preprint arXiv:2109.00859(2021)

    Pith/arXiv arXiv 2021

  39. [41]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training.arXiv preprint arXiv:2212.03533(2022)

    Pith/arXiv arXiv 2022

  40. [42]

    Laurie Williams, Giacomo Benedetti, Sivana Hamer, Ranindya Paramitha, Imra- nur Rahman, Mahzabin Tamanna, Greg Tystahl, Nusrat Zahan, Patrick Morrison, Yasemin Acar, et al. 2025. Research directions in software supply chain security. ACM Transactions on Software Engineering and Methodology34, 5 (2025), 1–38

    work page 2025

  41. [43]

    Zora Zhiruo Wang, Akari Asai, Xinyan Velocity Yu, Frank F Xu, Yiqing Xie, Graham Neubig, and Daniel Fried. 2024. Coderag-bench: Can retrieval augment code generation?arXiv preprint arXiv:2406.14497(2024)

    Pith/arXiv arXiv 2024

  42. [44]

    Zhihui Xie, Jiahui Gao, Lei Li, Zhenguo Li, Qi Liu, and Lingpeng Kong. 2024. Jailbreaking as a reward misspecification problem.arXiv preprint arXiv:2406.14393 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  43. [45]

    Simon Willison. [n. d.]. Delimiters won’t save you from prompt injection, 2023. URL https://simonwillison. net/2023/May/11/delimiters-wont-save-you4 ([n. d.])

    work page 2023

  44. [46]

    Shenao Yan, Shen Wang, Yue Duan, Hanbin Hong, Kiho Lee, Doowon Kim, and Yuan Hong. 2024. An{LLM-Assisted}{ Easy-to-Trigger} Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection. In33rd USENIX Security Symposium (USENIX Security 24). 1795–1812

    work page 2024

  45. [47]

    Jiaqi Xue, Mengxin Zheng, Yebowen Hu, Fei Liu, Xun Chen, and Qian Lou. 2024. Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models.arXiv preprint arXiv:2406.00083(2024)

    Pith/arXiv arXiv 2024

  46. [48]

    Kai Ye, Liangcai Su, and Chenxiong Qian. 2025. How Far Are We from True Unlearnability?. InThe Thirteenth International Conference on Learning Represen- tations. https://openreview.net/forum?id=I4Lq2RJ0eJ

    work page 2025

  47. [49]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly.High-Confidence Computing(2024), 100211

    work page 2024

  48. [50]

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, et al. 2024. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval.arXiv preprint arXiv:2407.19669(2024)

    Pith/arXiv arXiv 2024

  49. [51]

    Junan Zhang, Kaifeng Huang, Bihuan Chen, Chong Wang, Zhenhao Tian, and Xin Peng. 2023. Malicious package detection in NPM and pypi using a single model of malicious behavior sequence.arXiv preprint arXiv:2309.02637(2023)

    Pith/arXiv arXiv 2023

  50. [52]

    Xinyi Zheng, Chen Wei, Shenao Wang, Yanjie Zhao, Peiming Gao, Yuanchao Zhang, Kailong Wang, and Haoyu Wang. 2024. Towards robust detection of open source software supply chain poisoning attacks in industry environments. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 1990–2001

    work page 2024

  51. [53]

    Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang. 2023. CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Bench- marking on HumanEval-X. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5673–5684

    work page 2023

  52. [54]

    Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al . 2024. Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence.arXiv preprint arXiv:2406.11931(2024)

    Pith/arXiv arXiv 2024

  53. [55]

    Zexuan Zhong, Ziqing Huang, Alexander Wettig, and Danqi Chen. 2023. Poi- soning retrieval corpora by injecting adversarial passages.arXiv preprint arXiv:2310.19156(2023)

    Pith/arXiv arXiv 2023

  54. [56]

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models.arXiv preprint arXiv:2307.15043(2023)

    Pith/arXiv arXiv 2023

  55. [57]

    Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al

  56. [58]

    Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions.arXiv preprint arXiv:2406.15877(2024)

    Pith/arXiv arXiv 2024

  57. [60]

    Code Manual

    Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. 2024. Poisonedrag: Knowledge poisoning attacks to retrieval-augmented generation of large lan- guage models.arXiv preprint arXiv:2402.07867(2024). Conference ’25, June 03–05, 2018, Woodstock, NY Kai Ye, Liangcai Su, and Chenxiong Qian Table 8: English inductive suggestions from several SOTA LLMs Model ...

    Pith/arXiv arXiv 2024

  58. [2024]

    InFindings of the Association for Computational Linguistics ACL 2024

    M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. InFindings of the Association for Computational Linguistics ACL 2024. 2318–2335

    work page 2024

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.