Trojan Detection Through Pattern Recognition for Large Language Models

Matthew Yudin; Rauf Izmailov; Razvan Stefanescu; Vedant Bhasin

REVIEW 4 major objections 5 minor 23 references

Trojan Detection Through Pattern Recognition for Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A logits-only, perturbation-checked pipeline detects LLM Trojan triggers, hitting AUC 1.0 on the TrojAI test set.

desk verdict A logits-only black-box Trojan detection pipeline with a clever verification idea, but the evaluation is thin and the verification premise is overclaimed in the text and contradicted by the paper's own false positive. read the letter →

arxiv 2501.11621 v1 pith:GYG2KGIR submitted 2025-01-20 cs.CL cs.LG

Vedant Bhasin , Matthew Yudin , Razvan Stefanescu , Rauf Izmailov This is my paper · ORCID

classification cs.CLcs.LG

keywords Trojandetectionbackdoorattackslargelanguagemodelstriggerinversionblack-boxlogits-onlyperturbationverificationRLHFpoisoning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that Trojan backdoors in large language models can be detected without model weights or gradients, using only output logits and the tokenizer. It proposes a three-stage pipeline: filter the vocabulary by comparing next-token probabilities against a clean guide model, decode candidate trigger sequences by greedy or beam search, and verify candidates by checking whether they survive small character-level perturbations but break under appended semantic-preserving phrases. On the TrojAI April 2024 test set the pipeline classifies all 12 models correctly with greedy decoding (ROC-AUC 1.0) and nearly all with beam search (0.97). The same verification idea separates true RLHF backdoors from other adversarial strings with AUC 0.90 for large perturbations and 0.84 for small ones. If this holds, Trojan detection becomes feasible in settings where only API logits are available.

What carries the argument

The framework's load-bearing pieces are the high confidence subsequence score and the two-stage verification procedure. Given a sequence of token probabilities $p = \{p_1, p_2, \dots, p_n\}$ and a threshold $\tau$, the high confidence subsequence is the longest consecutive run with $p_i \geq \tau$; this metric replaces vanilla joint probability because it is insensitive to unknown trigger length. Identification decodes candidate sequences from filtered tokens, either by greedy decoding from context-token pairs or by beam search from single tokens, and keeps candidates whose high confidence subsequence exceeds a length threshold. Verification then applies large semantic-preserving perturbations (appended phrases such as 'Reply in English') and small character-level perturbations (case changes and seven special characters), and scores a candidate by its activation fraction across contexts and perturbations, with DBSCAN clustering used in the beam-search variant to group near-identical outputs.

What would settle it

Run the beam-search identification and two-stage verification on a clean Llama-2 model with no injection; if any candidate reaches an activation fraction as high as the poisoned models after both perturbation stages, the verification premise fails. The paper already reports one such strong false positive in the TrojAI beam-search evaluation, so reproducing it under controlled clean-only conditions would settle the claim.

Watch

Extended reading notes

Core claim

The central claim is that Trojan trigger sequences leave a distinctive signature in next-token probability patterns: they decode as unusually long runs of high-confidence tokens, and they respond to input perturbations differently from benign high-likelihood sequences. True triggers are invariant to small character-level changes such as case modification and special-character insertion, but break when followed by a large semantic-preserving phrase such as 'Be concise'; benign sequences show the opposite pattern. The paper operationalizes this with a logits-only pipeline and reports ROC-AUC 1.0 for the greedy variant and 0.97 for the beam-search variant on the TrojAI test set, with all ground-truth triggers in the training set recovered. On the RLHF dataset, the perturbation-based verification separates the five true backdoors from contestant-discovered adversarial strings with AUC 0.90 for large perturbations and 0.84 for small perturbations.

Load-bearing premise

The whole method leans on the belief that real Trojan triggers survive small character tweaks and break under appended instructions, while harmless high-likelihood text breaks under small tweaks and survives appended instructions, even though the paper's own beam-search run found a harmless sequence that survived both.

Editorial extensions

If this is right

If the claim holds, Trojan detection no longer requires white-box access: an API that exposes logits is enough to screen a model for backdoors.
The high confidence subsequence metric, rather than plain joint probability, is the right score for trigger candidates because it is insensitive to unknown trigger length.
The verification stage is essential; without it the identification stage flags dozens of benign high-likelihood sequences, and the perturbation test removes most of them.
The beam-search variant finds all ground-truth triggers without hand-picked context tokens but is less specific, and the reported strong false positive shows the verification test is not yet sufficient by itself.
Perturbation robustness can serve as a general property for distinguishing alignment-breaking backdoors from ordinary jailbreak strings in RLHF-trained models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to replace the clean guide model with a self-referential baseline, such as the target model's own probabilities under multiple prompts or seeds; if that works, the filtration stage no longer depends on having a trusted clean model.
The paper's beam-search false positive suggests that perturbation invariance is necessary but not sufficient; combining it with trigger reconstruction fidelity, such as whether the candidate reproduces a known target response, could close the residual gap.
Because the RLHF verification uses reward-model scores rather than output text, the same perturbation logic could be applied at inference time as a defense: if a suspicious input's harmful effect vanishes when an innocuous instruction is appended, it may be a backdoor trigger.
The method depends on output logits, so it does not transfer directly to fully hidden black-box APIs; whether text-only outputs with semantic similarity would preserve the signal is a direct empirical question.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short take: this is a genuinely useful three-stage black-box trigger detection pipeline, but the evaluation is thinner than the headline numbers suggest and the verification premise in Section 3.3 is stated too strongly—the paper's own beam-search false positive is a concrete counterexample that never gets analyzed.

What's new: the combination of logits-only token filtration, greedy/beam search identification using a high-confidence-subsequence metric, and perturbation-based verification is not in the cited prior art. The verification transfer to the RLHF dataset, where it separates ground-truth backdoors from contestant-discovered adversarial strings with AUC 0.90/0.84, is a nice result that suggests the differential-robustness idea has some generality. The paper also earns credit for reporting the beam-search false positive honestly instead of hiding it.

Soft spots: the evidence base is narrow—12 TrojAI test models and 5 RLHF models, no baseline comparisons, no error bars or confidence intervals, and no released code. K=600 and the greedy context set are set empirically on training data, with no sensitivity analysis; the text says the context choice was 'crucial,' which is exactly the kind of manual knob that needs a robustness check. The biggest conceptual issue is Section 3.3's universal claim: true Trojans are invariant to small perturbations and benign sequences are not. The beam-search results contradict that directly: a benign sequence survives the perturbations and inflates a clean model's Trojan probability, dropping AUC to 0.97. The paper acknowledges the false positive but never shows the sequence, its activation values, or whether a different threshold or DBSCAN setup could exclude it. So the verification stage's failure mode is unquantified, and the mechanism behind the 0.97 AUC is not fully explained. I'd stop short of saying the mechanism is invalid—the RLHF transfer and the training-set separation suggest the differential holds on average—but the claim in the methods section needs to be softened and the counterexample needs a real analysis.

Who this is for: anyone working on LLM Trojan detection or model auditing. The pipeline is implementable from the description, but you'd want code before building on it. It deserves a serious referee: the idea is sound enough, the task is important, and the identified weaknesses are addressable in revision. I'd send it out, with a request for baselines, sensitivity analysis, code, and an honest treatment of the false positive.

Referee Report

4 major / 5 minor

Summary. The paper proposes a multistage black-box Trojan detection framework for LLMs that operates on output logits and a tokenizer only. The pipeline filters the vocabulary by comparing next-token probabilities against a clean guide model, decodes candidate trigger sequences via greedy or beam search using a 'high confidence subsequence' metric, and then applies a two-stage verification step that combines large semantic-preserving perturbations with small character-level perturbations. Activation fractions are used to assign each model a Trojan probability. The authors report ROC-AUC 1.0 for greedy decoding and 0.97 for beam search on the TrojAI April 2024 test set, and AUCs of 0.90 and 0.84 for the verification stage alone on the RLHF poisoned-model dataset.

Significance. If the reported results hold, this is a valuable contribution to black-box Trojan detection: it is logits-only, does not assume access to gradients or model weights, and it introduces a concrete verification mechanism based on differential robustness to perturbation. The evaluation uses held-out test models and thresholds that were set before the test evaluation, so the headline numbers are not circular. The high-confidence-subsequence metric and the perturbation-based verification idea are worth developing further. However, the evidence base is thin: there are no error bars, no baseline comparisons, manual selection of context tokens is described as crucial, and the paper itself reports a strong benign false positive that survives the verification stage. These issues limit the strength of the central claim and need to be addressed before the paper can be accepted.

major comments (4)

[Section 3.3 and Section 5.3] The verification premise is contradicted by the paper's own beam-search result. Section 3.3 assumes that true Trojan triggers are invariant to small character-level perturbations and brittle to large semantic-preserving prompts, while benign high-likelihood sequences are not. Section 5.3 and Figure 7 report that a strong false positive, a benign sequence, survives perturbation and remains as a robust candidate. This is a concrete counterexample to the premise as stated, and the paper does not disclose the sequence, its activation values over the 10 perturbations and 5 contexts, or a threshold/cluster rule that would exclude it while retaining the ground-truth triggers. Since the verification stage is described as critical, the unexplained counterexample blocks the conclusion that the differential-robustness test separates triggers from false positives.
[Section 5.2, Table 4, Section 5.1] The greedy variant's reported 1.0 ROC-AUC depends on the manual choice of 3-5 context tokens, which Section 5.2 calls 'crucial,' and on K=600 set empirically in Section 5.1. No sensitivity analysis, no confidence intervals, and no systematic procedure for selecting contexts are reported for the 12 test models. Because the context-selection step is not specified beyond 'choose a very small subset of filtered tokens,' the claim that this is a general black-box pipeline is not yet supported; the manual tuning could substantially overstate the achievable performance.
[Section 5.3 and Section 3.4] The beam-search variant computes Trojan probabilities via DBSCAN clustering of sequence embeddings, but the embedding model, distance metric, and DBSCAN parameters (eps, min_samples) are not specified. This is load-bearing because different clustering settings can change activation fractions and hence the reported 0.97 AUC, and the result cannot be reproduced or independently assessed without these details.
[Section 5.4 and Table 6] The RLHF evaluation covers only the verification stage applied to contestant-submitted adversarial strings, not the full identification-plus-verification pipeline, as Section 3.5 acknowledges. Table 6 reports per-model percent changes without confidence intervals, and the separation between ground-truth triggers and other triggers varies widely (e.g., Model 1 character perturbations: 4% vs 44%; Model 4 large perturbations: 73% vs 141%). The reported AUCs of 0.90 and 0.84 rest on a small number of strings and no baseline comparison, so the paper's claim of generalizability to the RLHF setting is not yet established.

minor comments (5)

[Section 1] The sentence 'In the black-box paradigm, model weights, gradients, or Trojan responses are available' appears to mean 'are not available'; the current wording contradicts the black-box setting described throughout the paper.
[Section 5.1] The filtering results are stated inconsistently: the text says the process 'can reduce the token space by around 90%' and later says it filters '98% of the vocabulary.' Clarify which figure is meant and how it is computed.
[Throughout] There are several typos and formatting issues, including 'T rigger' in the Section 2.2 heading, 'T able' in table captions, and 'acheive' in the Figure 1 caption.
[Table 4 and Section 5.2] Table 4 lists 'Contexts 4' for greedy decoding, while Section 5.2 says '3-5 filtered tokens' are chosen; the exact number and the criterion for choosing them should be stated precisely.
[Section 5.3] The acknowledgement that kernel density estimates can produce values outside [0,1] is helpful, but the affected figures should perhaps plot the raw distributions to avoid possible misinterpretation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is evaluated against externally labeled held-out models, with no parameter fitted to the test labels.

full rationale

The paper's central claim is that a logits-only pipeline (token filtration, trigger identification, verification) produces Trojan probabilities that separate poisoned from clean models. The ground-truth labels come from external TrojAI and RLHF competition datasets (references 15 and 18), not from the method itself. The verification score is a hand-defined activation fraction based on output persistence under perturbations, and the AUC is computed by comparing that score with the externally supplied labels; no classifier weight or threshold is fitted to those labels. Hyperparameters such as K, tau, and the high-confidence threshold are set on the training models and applied to the held-out test models, which is an overfitting concern but not circularity. The paper openly reports that beam search produces a strong benign false positive that survives verification (Sections 5.3 and Figure 7), which undermines the robustness premise as a universal rule, but this is a correctness limitation rather than a derivation that reduces to its own inputs. There are no load-bearing self-citations, imported uniqueness theorems, or ansatz-smuggling citations; the only self-referential element is the paper's own empirical claim that its metric is robust, which is tested against external data. Therefore no circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

No new physical or ontological entities are posited; the method uses standard model outputs and derived scores. The load-bearing assumptions are domain assumptions about trigger statistics, the availability of a clean guide model, and perturbation robustness.

free parameters (10)

K, filtered token count = 600
Number of tokens kept after token filtration; called 'empirically determined' in Section 5.1 and directly controls which triggers can be found.
high confidence threshold tau (greedy) = 0.9
Token probability threshold for the high-confidence subsequence metric in greedy decoding; hand-set and shown in Table 4.
high confidence threshold tau (beam search) = 0.975
Token probability threshold for the high-confidence subsequence metric in beam search; hand-set and shown in Table 4.
high confidence subsequence length threshold = 5
Minimum length of a high-confidence run for a sequence to be flagged for verification; hand-set in Table 4.
decoded sequence length = 16
Length of token sequences generated during trigger identification; hand-set in Table 4.
beam width = 32
Beam width for the beam search variant; hand-set in Table 4.
greedy context tokens = 3-5 filtered tokens, chosen manually
Context set used in greedy decoding; the paper states the choice was 'crucial' in Section 5.2, so results depend on manual selection.
special character perturbation set = "* . ? > ) / @" (7 characters)
Small perturbation set used in verification, listed in Section 3.3; no analysis of why these seven characters are sufficient or representative.
DBSCAN parameters = not reported
DBSCAN clustering of sequence embeddings is used to compute activation fractions in the beam search variant (Section 5.3), but eps and min_samples are unstated.
verification similarity and activation thresholds = not reported
Thresholds for deciding whether a candidate is a true trigger based on similarity or activation fraction are mentioned in Sections 3.3 and 3.4, but their numerical values are not given.

assumptions (5)

domain assumption A clean, unpoisoned guide model is available and next-token probability differences at the SOS token surface trigger tokens.
Used in Algorithm 4 and Section 3.1; if the guide model is not clean or the trigger token does not stand out at SOS, the trigger is filtered away before identification.
domain assumption Trojan triggers are composed of tokens with very high next-token probabilities, forming long runs above threshold tau.
Motivates the high-confidence subsequence metric in Section 3.1 and Equation 1, and underlies the identification algorithms.
domain assumption True triggers are robust to stated character-level perturbations and disrupted by appended semantic-preserving prompts, while benign high-likelihood strings are not.
Central verification premise in Section 3.3; contradicted by the beam-search false positive discussed in Section 5.3.
domain assumption The RLHF reward model provided by competition hosts correctly scores conversation harmfulness.
Used to replace the high-confidence subsequence metric for RLHF verification in Sections 3.5 and 4.2.
standard math Softmax of output logits gives valid next-token probabilities; beam search and greedy decoding are standard decoding procedures.
Background computational assumptions used throughout Section 3.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Trojan Detection Through Pattern Recognition for Large Language Models." pith.science (2026). https://pith.science/paper/GYG2KGIR

@misc{pith2026250111621,
  author       = {Pith},
  title        = {Pith review of: Trojan Detection Through Pattern Recognition for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GYG2KGIR}},
  note         = {Machine review of arXiv:2501.11621}
}

read the original abstract

Trojan backdoors can be injected into large language models at various stages, including pretraining, fine-tuning, and in-context learning, posing a significant threat to the model's alignment. Due to the nature of causal language modeling, detecting these triggers is challenging given the vast search space. In this study, we propose a multistage framework for detecting Trojan triggers in large language models consisting of token filtration, trigger identification, and trigger verification. We discuss existing trigger identification methods and propose two variants of a black-box trigger inversion method that rely on output logits, utilizing beam search and greedy decoding respectively. We show that the verification stage is critical in the process and propose semantic-preserving prompts and special perturbations to differentiate between actual Trojan triggers and other adversarial strings that display similar characteristics. The evaluation of our approach on the TrojAI and RLHF poisoned model datasets demonstrates promising results.

Figures

Figures reproduced from arXiv: 2501.11621 by the authors.

**Figure 1.** Challenges associated with trigger identification. The presence of strong false positives makes it difficult to acheive high recall and precision simultaneously. 5.1 Filtering Both variants of our method scale poorly with the number of candidate tokens. Given the large vocabulary of LLM tokenizers, restricting the subspace is imperative to make our approach feasible. Our filtering process is highly effective in re… view at source ↗

**Figure 2.** [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

**Figure 3.** Trigger candidates for the TrojAI Rev1 train models ranked by their activations. The experiments were run with 10 perturbations and 5 contexts leading to 50 candidates per trigger. The top six are all ground truth triggers, with false positives showing a consistently low activation frequency. Our experiments with beam search also demonstrate high performance in trigger identification, with all ground truth triggers… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Trojan probabilities for models in the TrojAI dataset calculated by the autoregressive greedy decoding; the Trojan probability is the maximum activation fraction. Hyperparameter Greedy Beam Search Beam Width 1 32 Decoded Sequence Length 16 16 Contexts 4 0 high confide…

**Figure 5.** Figure 5: Distribution of Trojan probabilities for clean & Trojan models calculated with the autoregressive greedy decoding. Clean models have a very low average activation fraction and a low variance, being tightly clustered with a maximum Trojan probability of less than 15% …

**Figure 7.** Figure 7: Trojan probabilities calculated by the beam search algorithm. A strong false positive is observed, likely because beam search identifies strong high-likelihood sequences that are robust to perturbation [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

**Figure 9.** Figure 9: Candidates clustered by DBSCAN and projected onto three principal components. Each trigger forms it’s own distinct cluster. Reward Changes by Perturbation Method Large Perturbations Character Perturbations Model Ground Truth (%) Other Triggers (%) Ground Truth (%) Oth…

**Figure 10.** Figure 10: ROC curve for detecting ground truth backdoors from the other adversarial strings by large, semantic-preserving perturbations on the RLHF dataset [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 6 canonical work pages

[1]

In: 2017 ieee symposium on security and privacy (sp), pp

Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 ieee symposium on security and privacy (sp), pp. 39–57. Ieee (2017)

work page 2017
[2]

In: 2018 IEEE security and privacy workshops (SPW), pp

Carlini, N., Wagner, D.: Audio adversarial examples: Targeted attacks on speech- to-text. In: 2018 IEEE security and privacy workshops (SPW), pp. 1–7. IEEE (2018)

work page 2018
[3]

URL https://arxiv.org/ abs/1706.03741

Christiano, P., Leike, J., Brown, T.B., Martic, M., Legg, S., Amodei, D.: Deep reinforcement learning from human preferences (2023). URL https://arxiv.org/ abs/1706.03741

arXiv 2023
[4]

In: Proceedings of the First Workshop on Neural Machine Translation

Freitag, M., Al-Onaizan, Y.: Beam search strategies for neural machine translation. In: Proceedings of the First Workshop on Neural Machine Translation. Association for Computational Linguistics (2017). DOI 10.18653/v1/w17-3207. URL http: //dx.doi.org/10.18653/v1/W17-3207

doi:10.18653/v1/w17-3207 2017
[5]

arXiv preprint arXiv:2402.09154 (2024)

Geisler, S., Wollschl¨ ager, T., Abdalla, M., Gasteiger, J., G¨ unnemann, S.: At- tacking large language models with projected gradient descent. arXiv preprint arXiv:2402.09154 (2024)

arXiv 2024
[6]

URL https://arxiv.org/abs/1211.3711

Graves, A.: Sequence transduction with recurrent neural networks (2012). URL https://arxiv.org/abs/1211.3711

arXiv 2012
[7]

URL https://arxiv.org/abs/1708.06733

Gu, T., Dolan-Gavitt, B., Garg, S.: Badnets: Identifying vulnerabilities in the ma- chine learning model supply chain (2019). URL https://arxiv.org/abs/1708.06733

arXiv 2019
[8]

arXiv preprint arXiv:2104.13733 (2021)

Guo, C., Sablayrolles, A., J´ egou, H., Kiela, D.: Gradient-based adversarial attacks against text transformers. arXiv preprint arXiv:2104.13733 (2021)

arXiv 2021

Show all 23 references

[9]

URL https:// arxiv.org/abs/2106.09685

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models (2021). URL https:// arxiv.org/abs/2106.09685

2021 arXiv
[10]

arXiv preprint arXiv:1611.01144 (2016)

Jang, E., Gu, S., Poole, B.: Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016)

2016 arXiv
[11]

Jones, E., Dragan, A., Raghunathan, A., Steinhardt, J.: Automatically auditing large language models via discrete optimization. In: A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett (eds.) Proceedings of the 40th In- ternational Conference on Machine Learn...

2023
[12]

URL https://arxiv.org/abs/2309.01446

Lapid, R., Langberg, R., Sipper, M.: Open sesame! universal black box jailbreaking of large language models (2024). URL https://arxiv.org/abs/2309.01446

2024 arXiv
[13]

arXiv preprint arXiv:2104.08691 (2021)

Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021)

2021 arXiv
[14]

URL https://arxiv.org/abs/2406.05948

Li, X., Zhang, Y., Lou, R., Wu, C., Wang, J.: Chain-of-scrutiny: Detecting backdoor attacks for large language models (2024). URL https://arxiv.org/abs/2406.05948

2024
[15]

URL https://doi.org/10.18434/mds2-3235

Majurski, M.P.: Trojan detection software challenge - llm-pretrain-apr2024-train (2024). URL https://doi.org/10.18434/mds2-3235. Accessed: 2024-10-15

2024 doi
[16]

In: NeurIPS Competition Track (2023)

Mazeika, M., Zou, A., Mu, N., Phan, L., Wang, Z., Yu, C., Khoja, A., Jiang, F., O’Gara, A., Sakhaee, E., Xiang, Z., Rajabi, A., Hendrycks, D., Poovendran, R., Li, B., Forsyth, D.: Tdc 2023 (llm edition): The trojan detection challenge. In: NeurIPS Competition Track (2023)

2023
[17]

Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: P. Isabelle, E. Charniak, D. Lin (eds.) Proceedings of the 40th Annual Meeting of the Association for Computational 20 Vedant Bhasin, Matthew Yudin and Razvan Ste...

2002
[18]

URL https://arxiv.org/abs/2404.14461

Rando, J., Croce, F., Mitka, K., Shabalin, S., Andriushchenko, M., Flammarion, N., Tram` er, F.: Competition report: Finding universal jailbreak backdoors in aligned llms (2024). URL https://arxiv.org/abs/2404.14461

2024 arXiv
[19]

URL https://arxiv.org/abs/2311.14455

Rando, J., Tram` er, F.: Universal jailbreak backdoors from poisoned human feed- back (2024). URL https://arxiv.org/abs/2311.14455

2024 arXiv
[20]

URL https://arxiv.org/abs/ 2310.19737

Schwinn, L., Dobre, D., G¨ unnemann, S., Gidel, G.: Adversarial attacks and defenses in large language models: Old and new threats (2023). URL https://arxiv.org/abs/ 2310.19737

2023 arXiv
[21]

URL https://arxiv.org/abs/2307.09288

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bash- lykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C.C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N...

2023 arXiv
[22]

Advances in Neural Information Processing Systems 36 (2024)

Wen, Y., Jain, N., Kirchenbauer, J., Goldblum, M., Geiping, J., Goldstein, T.: Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Advances in Neural Information Processing Systems 36 (2024)

2024
[23]

arXiv preprint arXiv:2307.15043 (2023)

Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J.Z., Fredrikson, M.: Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 (2023)

2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

In: 2017 ieee symposium on security and privacy (sp), pp

Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 ieee symposium on security and privacy (sp), pp. 39–57. Ieee (2017)

work page 2017

[2] [2]

In: 2018 IEEE security and privacy workshops (SPW), pp

Carlini, N., Wagner, D.: Audio adversarial examples: Targeted attacks on speech- to-text. In: 2018 IEEE security and privacy workshops (SPW), pp. 1–7. IEEE (2018)

work page 2018

[3] [3]

URL https://arxiv.org/ abs/1706.03741

Christiano, P., Leike, J., Brown, T.B., Martic, M., Legg, S., Amodei, D.: Deep reinforcement learning from human preferences (2023). URL https://arxiv.org/ abs/1706.03741

arXiv 2023

[4] [4]

In: Proceedings of the First Workshop on Neural Machine Translation

Freitag, M., Al-Onaizan, Y.: Beam search strategies for neural machine translation. In: Proceedings of the First Workshop on Neural Machine Translation. Association for Computational Linguistics (2017). DOI 10.18653/v1/w17-3207. URL http: //dx.doi.org/10.18653/v1/W17-3207

doi:10.18653/v1/w17-3207 2017

[5] [5]

arXiv preprint arXiv:2402.09154 (2024)

Geisler, S., Wollschl¨ ager, T., Abdalla, M., Gasteiger, J., G¨ unnemann, S.: At- tacking large language models with projected gradient descent. arXiv preprint arXiv:2402.09154 (2024)

arXiv 2024

[6] [6]

URL https://arxiv.org/abs/1211.3711

Graves, A.: Sequence transduction with recurrent neural networks (2012). URL https://arxiv.org/abs/1211.3711

arXiv 2012

[7] [7]

URL https://arxiv.org/abs/1708.06733

Gu, T., Dolan-Gavitt, B., Garg, S.: Badnets: Identifying vulnerabilities in the ma- chine learning model supply chain (2019). URL https://arxiv.org/abs/1708.06733

arXiv 2019

[8] [8]

arXiv preprint arXiv:2104.13733 (2021)

Guo, C., Sablayrolles, A., J´ egou, H., Kiela, D.: Gradient-based adversarial attacks against text transformers. arXiv preprint arXiv:2104.13733 (2021)

arXiv 2021