Pith. sign in

REVIEW 4 major objections 5 minor 50 references

PrompTrend: Continuous Community-Driven Vulnerability Discovery and Assessment for Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that the dominant LLM threat is psychological manipulation from online communities, that successful attacks rarely transfer across model families, and that, within the Claude family, newer models are more vulnerable than…

desk verdict A useful monitoring concept undermined by circular validation and unsupported temporal claims. read the letter →

arxiv 2507.19185 v1 pith:KTHO2UEZ submitted 2025-07-25 cs.CR cs.AI

Tarek Gasmi , Ramzi Guesmi , Mootez Aloui , Jihene Bennaceur This is my paper · ORCID
classification cs.CRcs.AI
keywords LLMsecurityvulnerabilityassessmentcommunity-drivendiscoveryjailbreakattackspsychologicalmanipulationcontinuousthreatintelligenceAIsafetyriskscoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the real threat to commercial large language models is being discovered in online communities, not in formal benchmarks, and that a continuous monitoring system can capture it. The authors built PrompTrend, which collected 198 vulnerability prompts from chat, forum, and code-sharing platforms over five months and tested them across nine models using 71 transformation strategies. Their central finding is that psychological manipulation, such as emotional appeals and roleplay, succeeds at roughly twice the rate of technical obfuscation like encoding or prefix injection. They also report that successful attacks rarely transfer across model families, and that among the Claude models tested the newest versions failed more often than earlier ones. The paper matters because it challenges the assumption that newer, more capable models are automatically safer.

What carries the argument

The central mechanism is the PrompTrend pipeline paired with the PVAF scoring framework. PrompTrend is a three-stage collection, enrichment, and scoring system whose platform-specific agents filter social-media streams for vulnerability candidates, producing structured metadata for each candidate. PVAF evaluates each vulnerability across six dimensions, including harm potential, exploit sophistication, community adoption, cross-platform efficacy, temporal resilience, and propagation velocity, and combines them into a 0-100 risk score. The framework's distinguishing work is adding social dynamics to technical risk, and the paper validates it by showing that higher PVAF scores predict higher measured jailbreak success, with a relative risk of 1.5 between moderate- and low-risk categories and an AUC of 0.72.

What would settle it

Rerun the same 198-prompt corpus and 71 transformations against an older and a newer model of the same family under identical API settings; if the newer model's failure rate is not higher, the claimed four-fold regression collapses.

Watch

Extended reading notes

Core claim

The paper's core claim is that capability advancement does not guarantee security improvement and that community-driven psychological manipulation is the dominant vulnerability vector for current LLMs. Using 198 community-discovered vulnerabilities tested with 71 transformations on nine commercial models, the authors report a vulnerability hierarchy with Claude 4 Sonnet highest at 4.1% and GPT-4.5 lowest at 0.6%, with emotional manipulation at 4.9% success versus Base64 at 2.7%. They also report that chat-platform-sourced attacks are most effective against Claude-family models, that only 16.9% of successful attacks transfer across model families, and that a multidimensional scoring framework called PVAF reaches 78% classification accuracy. The authors argue these results demonstrate that static benchmarks miss the evolving, socially driven threat landscape and that security evaluation must be continuous and community-aware.

Load-bearing premise

The paper's central warning depends on treating a single snapshot of different current model versions as a trend over time, not on repeated measurements of the same system.

Editorial extensions

If this is right

  • If psychological attacks dominate, red-team exercises and safety training should be rebuilt around social-engineering scenarios rather than token-level perturbations.
  • If successful attacks rarely transfer across model families, defenses must be model-specific and universal jailbreak defenses are unlikely to suffice.
  • If the capability-security inversion within the Claude family is real, deployment decisions should not treat newer versions as automatically safer.
  • If PVAF risk scores predict jailbreak success, security teams can triage community-discovered prompts by score before full empirical testing.
  • If static benchmarks miss community-driven discovery, continuous platform monitoring becomes a necessary complement to formal evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the claimed regression of the Claude family is a cross-sectional comparison of different current model versions, so a longitudinal release-by-release test is needed before treating newer-is-less-safe as a general law.
  • My inference: the emotional-manipulation result could be tested against a benign-emotion control; if emotionally framed benign requests also raise compliance, the mechanism is over-helpfulness rather than a safety-policy gap.
  • My inference: the platform-channel effects, with chat-borne psychological attacks outperforming code-repository technical attacks, imply that monitoring priorities could be tuned per deployment, though the small effect sizes caution against large operational bets.
  • My inference: the 16.9% cross-model transferability figure may undercount universal attacks because the transformation taxonomy was partly seeded by the same community patterns being measured.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces PrompTrend, a monitoring system that collects LLM 'jailbreak' prompts from online communities (Reddit, Discord, GitHub, Twitter/X, security forums), filters them, and evaluates them with the PrompTrend Vulnerability Assessment Framework (PVAF). The authors report 198 unique vulnerabilities collected between January and May 2025, tested across nine commercial models using 71 transformation strategies. Their headline findings are that psychological manipulation outperforms technical obfuscation, that platform dynamics (especially Discord) shape attack effectiveness, that cross-model transferability is low, and that capability advancement does not improve security, with Claude 4 showing a 'four-fold regression' compared with earlier Claude versions. The paper also reports that PVAF achieves 78% classification accuracy, an AUC of 0.72, and a relative risk of 1.50 between low- and moderate-risk categories. The central contributions claimed are the monitoring architecture, the PVAF multidimensional scoring framework, and the empirical findings about model-specific vulnerability patterns.

Significance. If the central claims were sound, this would be a useful contribution: a community-driven vulnerability collection pipeline, a repeatable API testing protocol, and an attempt to incorporate social-adoption signals into vulnerability scoring are all valuable directions. The manuscript has visible strengths: Algorithm 1 gives a concrete testing protocol, the evaluation includes inter-rater reliability (Cohen's kappa = 0.76), the statistical reporting uses effect sizes and multiple-comparison corrections, and the paper includes ethical disclosure considerations. However, the two load-bearing claims do not survive scrutiny. The PVAF performance numbers come from a threshold choice made after seeing the data on the same dataset, making them circular; the 'capability advancement does not improve security' claim rests on treating a cross-sectional comparison of different model versions and tiers as a temporal trend, and Figure 11b even includes a model (GPT-4o) that is not in the tested set. The PVAF scoring definition is internally inconsistent, and several reported numbers do not reconcile. These problems reduce the significance of the empirical contributions as they stand.

major comments (4)
  1. [§6.4 and §5.4] The reported validation of PVAF is circular. Section 6.4 states that initial thresholds were recalibrated because vulnerabilities scoring 42–47 showed lower success rates than those scoring 34–41, and that the balanced terciles (0–33, 34–66, 67–100) 'restored monotonic risk progression'. The 78% accuracy, AUC 0.72, and relative risk 1.50 in Section 5.4 and Table 5 are computed on the same data that motivated the recalibration, with no held-out split, temporal replication, or independent test set. These numbers therefore do not establish predictive validity; they are in-sample goodness-of-fit statistics.
  2. [§5.1, Figure 11b, §6.1, Conclusion] The claim that Claude 4's 4.1% vulnerability rate is a 'four-fold regression' is an inference from a single cross-sectional snapshot, not a measured temporal trend. Section 4.1.1 and Section 6.6 describe the data as cross-sectional, and the three compared Claude models (Haiku, 3.5 Sonnet, 4 Sonnet) differ in tier and were each tested once; they are not repeated observations of the same system. Figure 11b also plots a GPT-4o data point dated May 2024, but Section 4.2.1 lists only GPT-4, O1, O3-Mini, and GPT-4.5 as the OpenAI models tested. Removing this unsupported temporal framing also removes the headline conclusion that capability advancement does not improve security.
  3. [§3.3.1, §3.3.2, §4.3, Eq. (1)] The PVAF scoring function is internally inconsistent and under-specified. Section 3.3.1 defines six dimensions with weights 0.20, 0.20, 0.15, 0.15, 0.15, 0.15, and Section 3.3.2 adds dynamic modifiers, but Eq. (1) in Section 4.3 is a three-term sum with equal weights 0.33 and omits Cross-Platform Efficacy, Temporal Resilience, Propagation Velocity, and all dynamic modifiers. The manuscript never states how the six dimensions are aggregated, how Community Adoption is computed from engagement signals, or whether dynamic modifiers were applied in the reported scores. In addition, the engagement-based Community Adoption dimension is derived from the same filtering signals used to select the 198 prompts (Sections 3.2.1 and 4.1.2), introducing circularity into the scoring. The reported PVAF values are therefore not reproducible from the description.
  4. [§5.4, Table 5, §4.2.2, §5.3] Several numerical results do not reconcile with each other. Table 5 reports a moderate-risk success rate of 16.90% and low-risk 11.27% on 22,152 test executions, while Section 5.3 gives an overall mean success rate of 2.0%; Section 4.2.2 reports 199,368 total executions. The text does not explain why the Table 5 denominator is 22,152, whether the success-rate definitions are the same, or how the 2.0% overall rate can coexist with the Table 5 rates. The high-risk row of Table 5 simultaneously reports '—' for the success rate and 0% of tests, which is not a meaningful comparison. These inconsistencies make the headline PVAF stratification result difficult to verify.
minor comments (5)
  1. [§4.1.2 and §4.1.1] Section 4.1.2 reports 2,800 unique vulnerabilities qualifying for detailed PVAF assessment, while Section 4.1.1 reports 198 unique vulnerability prompts; clarify the relationship between these numbers.
  2. [Figure 11b] Figure 11b is labeled as temporal evolution, but the underlying data are cross-sectional; the figure should be replotted as a model-version comparison or removed.
  3. [§3.2.2] Section 3.2.2 states that bridge-node identification is 'designed for...future deployments' while the surrounding text describes it as part of the implemented system; align the tense and scope.
  4. [Table 5] In Table 5, the High Risk row should use 'not observed' rather than '—' for the success rate, and the 0% test count should be explained.
  5. [§6.5] The statement in Section 6.5 that 'Claude 4 regresses' repeats the invalid temporal interpretation identified in Major Comment 2 even after the Limitations section.

Circularity Check

3 steps flagged · score 6.0 of 10

PVAF's validation is circular: the score embeds execution outcomes and was threshold-calibrated on the same data, so accuracy, AUC, and monotonic risk progression reduce by construction.

  1. self definitional [Section 3.3.1 (PVAF scoring) vs. Section 5.4 (PVAF validation)]
    "After empirical testing (Phase 2), PVAF incorporates execution results to calculate comprehensive scores including all six dimensions. ... The correlation between PVAF scores and jailbreak success (r = 0.318, p < 0.001) confirms positive association within the observed range."

    PVAF's Phase 2 score is computed using the same execution results (jailbreak/block classifications) that Section 5.4 uses as the outcome variable. The reported correlation, regression slope, AUC, and accuracy therefore compare the score to one of its own components; the positive association is built into the scoring function rather than discovered empirically.

  2. fitted input called prediction [Section 6.4 (PVAF Calibration) vs. Section 5.4 / Table 5]
    "Initial deployment revealed that vulnerabilities scoring 42–47 showed lower success rates than those scoring 34–41, prompting recalibration from compressed thresholds (0–19, 20–39, 40+) to balanced terciles (0–33, 34–66, 67–100) that restored monotonic risk progression."

    The risk-category cutoffs were adjusted until the observed success rates became monotone, and the same dataset is then reported in Table 5 as empirical validation (16.90% vs 11.27%, relative risk 1.50) and in Figure 14 as calibration/AUC evidence. The monotonic risk progression and 78% accuracy are post-hoc fits to the data, not independent confirmations.

1 more flagged steps
  1. self definitional [Section 3.2.1 (agent collection) vs. Section 3.3.1 / Eq. (1)]
    "Reddit agents prioritize threads with high engagement ratios (comments/upvotes > 0.3), recognizing that community validation often indicates significant discoveries [42]. ... Community Adoption (0.15) quantifies uptake through engagement metrics, reposting frequency, and cross-platform citations."

    Community Adoption is scored from engagement metrics, while the collection layer prioritized content using the same engagement signals. The CA term in Eq. (1) therefore partly re-encodes the inclusion rule used to build the 198-prompt corpus, making the social dimension of PVAF a re-description of the sampling filter rather than an independent measure of community dynamics.

full rationale

The central PVAF contribution is partially circular on two independent grounds. First, Phase 2 PVAF scores incorporate the execution results that Section 5.4 then uses as the outcome in the correlation, regression, AUC, and accuracy analyses; the predictor contains the target by construction. Second, Section 6.4 states that the risk thresholds were recalibrated until monotonic risk progression was restored, yet Table 5 and Figure 14 report exactly that monotonicity and the 78% accuracy / AUC 0.72 as validation on the same corpus. The Community Adoption dimension also reuses the engagement signals that guided inclusion of the 198 prompts, so part of the score is a re-description of the inclusion rule. The descriptive findings (psychological strategies outperforming technical obfuscations, Discord platform effects, limited cross-model transferability) are not circular and remain independent empirical measurements. No load-bearing self-citation was identified; the GuardVal comparison is an external check. The 'four-fold regression' Claude claim is an unsupported cross-sectional temporal inference, but that is a validity/correctness issue rather than a circularity issue and is not scored here.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims depend on a curated and filtered sample of 198 public English-language prompts, hand-set PVAF weights and thresholds that were recalibrated after seeing the data, and the assumption that a cross-sectional sweep of current model versions describes temporal security evolution. No held-out validation is described.

free parameters (5)
  • PVAF dimension weights = w1=w2=w3=0.33 in Eq. (1); 0.20/0.20/0.15/0.15/0.15/0.15 in Section 3.3.1
    Hand-chosen weights determine every PVAF score; the two specifications contradict each other and no fitting or derivation is shown.
  • PVAF risk thresholds = Low 0-33, Moderate 34-66, High 67-100
    Recalibrated in Section 6.4 after observing that scores 42-47 underperformed 34-41; therefore fit to the evaluation data.
  • Collection filters = keyword threshold 0.7, dedup cosine 0.85, engagement ratio >0.3, 127-term lexicon
    These thresholds determine which community posts enter the 198-vulnerability dataset and therefore shape all downstream success rates.
  • PVAF dynamic modifiers = Mutation +5 to +15, Corporate Response -5 to -20, Academic Citation +10, Tool Integration +15, Regulatory Attention +10
    Arbitrary point adjustments with no calibration evidence; they change risk scores and risk categories.
  • Transformation set size and composition = 71 transformations in 8 categories
    The attack space is defined by the authors' choice of transformations; no completeness or coverage argument is given.
assumptions (5)
  • domain assumption Public English-language forum content, filtered by keywords and engagement heuristics, is representative of in-the-wild LLM vulnerabilities.
    Used throughout Sections 4.1 and 5.3 to generalize from the collected sample to 'current language models'.
  • domain assumption Cross-sectional comparison of current model versions can be interpreted as a temporal security trend.
    Invoked in Section 5.1 and Figure 11b, where Claude Haiku versus Claude 4 Sonnet is read as security regression over time.
  • domain assumption API responses can be reliably classified into BLOCKED/FAIL/NEUTRAL/ERROR by the two-stage keyword-plus-manual-review procedure.
    This classification defines ASR, the paper's core outcome; inter-rater kappa is reported for NEUTRAL only (kappa=0.76).
  • ad hoc to paper The set of 71 transformations spans the practically relevant attack space.
    The transformation set is assembled by the authors without a completeness argument; the conclusion that 'psychological attacks dominate' depends on what is in this set.
  • standard math Standard statistical tests (McNemar, Friedman, Cochran's Q, chi-square) apply to the evaluation matrix.
    Used in Section 5 for model and transformation comparisons; assumes the response matrix entries are paired and independent as required.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PrompTrend: Continuous Community-Driven Vulnerability Discovery and Assessment for Large Language Models." pith.science (2026). https://pith.science/paper/KTHO2UEZ

@misc{pith2026250719185,
  author       = {Pith},
  title        = {Pith review of: PrompTrend: Continuous Community-Driven Vulnerability Discovery and Assessment for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KTHO2UEZ}},
  note         = {Machine review of arXiv:2507.19185}
}
read the original abstract

Static benchmarks fail to capture LLM vulnerabilities emerging through community experimentation in online forums. We present PrompTrend, a system that collects vulnerability data across platforms and evaluates them using multidimensional scoring, with an architecture designed for scalable monitoring. Cross-sectional analysis of 198 vulnerabilities collected from online communities over a five-month period (January-May 2025) and tested on nine commercial models reveals that advanced capabilities correlate with increased vulnerability in some architectures, psychological attacks significantly outperform technical exploits, and platform dynamics shape attack effectiveness with measurable model-specific patterns. The PrompTrend Vulnerability Assessment Framework achieves 78% classification accuracy while revealing limited cross-model transferability, demonstrating that effective LLM security requires comprehensive socio-technical monitoring beyond traditional periodic assessment. Our findings challenge the assumption that capability advancement improves security and establish community-driven psychological manipulation as the dominant threat vector for current language models.

Figures

Figures reproduced from arXiv: 2507.19185 by the authors.

Figure 1
Figure 1. Evolution Timeline of a Major Jailbreak Technique [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Conceptual Overview of the PrompTrend System [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. PrompTrend Three-Stage Processing Pipeline Architecture [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Stage 2 Metadata Enrichment Pipeline [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Multi-Platform Agent Deployment and Coordination Architecture [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Cross-Platform Vulnerability Propagation Network [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Conceptual Model of Cross-Platform Vulnerability Propagation (Illustrative Example) [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: PVAF Scoring Process and Dimension Integration [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: System Implementation and Data Flow Architecture [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Vulnerability Assessment Card with Multi-Dimensional Risk Metrics [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Model Vulnerability Landscape Perhaps most concerning is the temporal evolution revealed in Figure 11b: while OpenAI models demonstrate consistent security improvements over time (GPT-4o at 1.9% in May 2024 improving to GPT-4.5 at 0.6% by February 2025), the Claude fa…
Figure 12
Figure 12. Figure 12: Transformation Effectiveness Analysis [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: PVAF Score Distribution and Success Rates [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 14
Figure 14. Figure 14: Risk Classification Performance Metrics The confusion matrix analysis (not shown) for Low versus Moderate risk classification achieves 78% overall accuracy, with 82.0% correct identification of low-risk vulnerabilities and 75.8% correct identification of moderate-risk…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 37 canonical work pages

  1. [1]

    Multi-lingual multi-turn automated red team- ing for LLMs, 2024

    Amazon Science. Multi-lingual multi-turn automated red team- ing for LLMs, 2024. https://www.amazon.science/publications/ multi-lingual-multi-turn-automated-red-teaming-for-llms

    work page 2024

  2. [2]

    Responsible scaling policy, 2023

    Anthropic. Responsible scaling policy, 2023

    work page 2023

  3. [3]

    Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned

    Anthropic. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858, 2022

    arXiv 2022

  4. [4]

    T. Aven. Risk assessment and risk management: Review of recent advances on their foundation. European Journal of Operational Research, 253(1):1–13, 2016

    work page 2016

  5. [5]

    Brown et al

    T. Brown et al. Community-driven attack development in online forums. InProc. Workshop AI Safety, Montreal, Canada, 2024

    work page 2024

  6. [6]

    Carlini and D

    N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. InProc. IEEE Symp. Security and Privacy, pages 39–57, San Jose, CA, USA, 2017

    work page 2017

  7. [7]

    Discord invite exploits lead to rising threats

    Check Point Research. Discord invite exploits lead to rising threats. Technical report, Check Point Research, 2025. 28

    work page 2025

  8. [8]

    Chehbouni et al

    K. Chehbouni et al. From representational harms to quality-of-service harms: A case study on Llama 2 safety safeguards. InACL Findings, 2024

    work page 2024

Show all 50 references
  1. [9]

    J. L. Devore. Probability and Statistics for Engineering and the Sciences. Cengage Learning, Boston, MA, 2011

    2011

  2. [10]

    Security bug bounty programme, 2025

    Discord. Security bug bounty programme, 2025. Accessed 2025

    2025

  3. [11]

    Discord platform, 2024.https://discord.com

    Discord Inc. Discord platform, 2024.https://discord.com

    2024

  4. [12]

    Common vulnerability scoring system version 4.0 specification document

    Forum of Incident Response and Security Teams. Common vulnerability scoring system version 4.0 specification document. Technical report, FIRST, 2023. https://www.first.org/cvss/ v4.0/specification-document

    2023

  5. [13]

    GitHub platform, 2024.https://github.com

    GitHub Inc. GitHub platform, 2024.https://github.com

    2024

  6. [14]

    Greshake et al

    K. Greshake et al. Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. InProc. 16th ACM Workshop Artificial Intelligence and Security, Copenhagen, Denmark, 2023

    2023

  7. [15]

    A. Hern. ’many-shot jailbreak’ reveals how AI safety features can be bypassed.The Guardian, 2024

    2024

  8. [16]

    AI threat landscape report 2024

    HiddenLayer. AI threat landscape report 2024. Technical report, HiddenLayer, 2024

    2024

  9. [17]

    How discord is abused for cyber-crime

    Intel 471. How discord is abused for cyber-crime. Technical report, Intel 471, 2024

    2024

  10. [18]

    The Waluigi effect, 2023

    Janus. The Waluigi effect, 2023. LessWrong. https://www.lesswrong.com/posts/ D7PumeYTDPfBTp3i7/the-waluigi-effect-mega-post

    2023

  11. [19]

    Jin et al

    H. Jin et al. GUARD: Role-playing to generate natural-language jailbreaks.arXiv preprint arXiv:2402.03299, 2024

    2024

  12. [20]

    Karger et al

    D. Karger et al. Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the world wide web. In Proc. 29th Annual ACM Symp. Theory of Computing, pages 654–663, 1997

    1997

  13. [21]

    Kiela et al

    D. Kiela et al. Dynabench: Rethinking benchmarking in NLP. InProc. Conf. North American Chapter Association for Computational Linguistics, 2021. Online

    2021

  14. [22]

    Kim et al

    S. Kim et al. Cross-platform jailbreak transferability in large language models.arXiv preprint arXiv:2403.17829, 2024

    2024

  15. [23]

    S. Liu, C. Reuter, M.-A. Kaufhold, and S. Bartsch. Collaborative cyber threat intelligence: Detecting and responding to malware in the wild.Computers & Security, 77:663–676, 2018

    2018

  16. [24]

    HarmBench: Astandardizedevaluationframeworkforautomatedredteaming and robust refusal

    M.Mazeikaetal. HarmBench: Astandardizedevaluationframeworkforautomatedredteaming and robust refusal. InProc. Int. Conf. Machine Learning, Vienna, Austria, 2024

    2024

  17. [25]

    ATLAS: Adversarial threat landscape for AI systems, 2025

    MITRE Corporation. ATLAS: Adversarial threat landscape for AI systems, 2025. Accessed 2025

    2025

  18. [26]

    Morris et al

    J. Morris et al. TextAttack: A framework for adversarial attacks, data augmentation, and adversarial training in NLP. InProc. Conf. Empirical Methods Natural Language Processing, Singapore, 2023. 29

    2023

  19. [27]

    GPT-4.5 system card, 2025

    OpenAI. GPT-4.5 system card, 2025

    2025

  20. [28]

    Our updated preparedness framework, 2025

    OpenAI. Our updated preparedness framework, 2025

    2025

  21. [29]

    GPT-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    OpenAI. GPT-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  22. [30]

    Top 10 for large language model applications, 2025

    OWASP Foundation. Top 10 for large language model applications, 2025

    2025

  23. [31]

    Pathade et al

    C. Pathade et al. Red teaming the mind of the machine: A systematic evaluation of prompt injection and jailbreak vulnerabilities in LLMs.arXiv preprint arXiv:2505.04806, 2025

    2025 arXiv

  24. [32]

    Perez et al

    E. Perez et al. Red teaming language models with language models. arXiv preprint arXiv:2202.03286, 2022

    2022 arXiv

  25. [33]

    S. L. Pfleeger and D. D. Caputo. Leveraging behavioral science to mitigate cyber security risk. Computers & Security, 31(4):597–611, 2012

    2012

  26. [34]

    R. Pu, Y. Yang, and W. Yu. Feint and attack: Attention-based strategies for jailbreaking and protecting LLMs. arXiv preprint arXiv:2410.16327, 2024

    2024 arXiv

  27. [35]

    Reddit platform, 2024.https://www.reddit.com

    Reddit Inc. Reddit platform, 2024.https://www.reddit.com

    2024

  28. [36]

    Samvelyan et al

    M. Samvelyan et al. RAINBOWPLUS: Enhancing adversarial prompt generation via evolu- tionary quality-diversity search.arXiv preprint arXiv:2504.15047, 2024

    2024

  29. [37]

    Samvelyan et al

    M. Samvelyan et al. Rainbow teaming: Open-ended generation of diverse adversarial prompts. arXiv preprint arXiv:2402.16822, 2024

    2024 arXiv

  30. [38]

    X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang. Do anything now: Characterizing and evaluating in-the-wild jailbreak prompts on LLMs.arXiv preprint arXiv:2308.03825, 2024

    2024 arXiv

  31. [39]

    Shi et al

    W. Shi et al. Red teaming language model detectors with language models.Trans. Association for Computational Linguistics, 11:814–830, 2023

    2023

  32. [40]

    Shimony and S

    E. Shimony and S. Dvash. Operation grandma: A tale of LLM chatbot vulnerability, 2024. CyberArk Blog

    2024

  33. [41]

    HELM safety: Towards standardized safety evaluation of language models

    Stanford CRFM. HELM safety: Towards standardized safety evaluation of language models. arXiv preprint arXiv:2310.11200, 2024

    2024 arXiv

  34. [42]

    Stieglitz and L

    S. Stieglitz and L. Dang-Xuan. Social media and political communication: A social media analytics framework. Social Network Analysis and Mining, 3(4):1277–1291, 2013

    2013

  35. [43]

    Wagner, A

    C. Wagner, A. Dulaunoy, G. Wagener, and A. Iklody. Collaborative security: Moving toward cybersecurity as a public good. InProc. IEEE Security and Privacy Workshops, pages 266–272, 2016

    2016

  36. [44]

    A. Wei, N. Haghtalab, and J. Steinhardt. Jailbroken: How does LLM safety training fail? arXiv preprint arXiv:2307.02483, 2023

    2023 arXiv

  37. [45]

    Twitter/X platform, 2024.https://x.com

    X Corp. Twitter/X platform, 2024.https://x.com

    2024

  38. [46]

    Y. Xie, R. Li, C. Chen, and W. Jiang. GradSafe: Detecting jailbreak prompts for LLMs via safety-critical gradient analysis. InProc. 62nd Annual Meeting Association for Computational Linguistics, Bangkok, Thailand, 2024. 30

    2024

  39. [47]

    Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, and Y. Zhang. A survey on large language model (LLM) security and privacy: The good, the bad, and the ugly.High-Confidence Computing, page 100211, 2024

    2024

  40. [48]

    Zhang et al

    P. Zhang et al. GuardVal: Dynamic large language model jailbreak evaluation for comprehen- sive safety testing.arXiv preprint arXiv:2507.07735, 2025

    2025 arXiv

  41. [49]

    A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson. Universal and transfer- able adversarial attacks on aligned language models.arXiv preprint arXiv:2307.15043, 2023

    2023 arXiv

  42. [50]

    do anything now

    V. Zuo et al. "do anything now": In-the-wild jailbreak prompts on LLMs, 2024. GitHub repository. 31

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.