Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants

T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An automated, knowledge-graph-guided agent can surface 11-66% more safety flaws in AI coding and security-guidance assistants than existing red-teaming tools, and its violation cases improve alignment training by 17%.

desk verdict Plausible and timely red-teaming pipeline, but the abstract alone leaves the headline numbers unverifiable; the temporal-access and circular-evaluation risks are real. read the letter →

arxiv 2508.03936 v1 pith:FKCI5AIC submitted 2025-08-05 cs.CR cs.CLcs.LGcs.SE

Xiangzhe Xu , Guangyu Shen , Zian Su , Siyuan Cheng , Hanxi Guo , Lu Yan , Xuan Chen , Jiasheng Jiang
show 4 more authors
Xiaolong Jin Chengpeng Wang Zhuo Zhang Xiangyu Zhang
This is my paper
classification cs.CRcs.CLcs.LGcs.SE
keywords autonomousred-teamingAIcodingassistantssecurityguidanceknowledgegraphsspatial-temporalprobingsafetyevaluationalignmenttrainingLLMvulnerabilities
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that fixed benchmark red-teaming misses realistic corner-case vulnerabilities in AI coding and security-guidance assistants, and that an automated agent can do better by combining structured domain knowledge with two complementary probes: spatial exploration of realistic developer requests and temporal exploration of the model's own reasoning. It reports that this agent, ASTRA, finds 11-66% more safety issues than existing techniques across two evaluation domains and yields violation cases that improve alignment training by 17%. If correct, safety auditing of code assistants can move from static benchmark passes toward adaptive, model-specific red-teaming that doubles as a source of training data. The paper proposes the mechanism and quantifies its payoff; a sympathetic reader would read the contribution as a method plus evidence rather than a formal guarantee.

What carries the argument

The central mechanism is the domain-specific knowledge graph: a structured representation of software tasks, known weaknesses, and their relationships, built from offline abstractions and then updated online during exploration. The paper couples this graph to two complementary probes: spatial exploration of the input space using realistic developer-style prompts, and temporal exploration of the model's reasoning processes. The knowledge graph selects which regions of input and reasoning behavior to probe next, and the generated violations are fed back into alignment training, closing the loop between discovery and mitigation.

What would settle it

Take a deployed coding assistant whose API returns only final outputs, run full ASTRA and an ASTRA variant with the temporal stage disabled, and compare violation-discovery rates over the same budget. If the two rates are indistinguishable, the temporal exploration is carrying no measurable weight; if the 11-66% margin over prior methods survives only when reasoning traces are available, the headline result is conditional on an access model that most real-world assistants do not offer.

Watch

Extended reading notes

Core claim

ASTRA claims that red-teaming of AI software assistants can be made systematic and realistic by replacing static prompt sets with a domain knowledge graph that is built offline from known software weaknesses and adapted online while probing each target. The agent explores each model along two axes: spatial exploration, which generates realistic developer requests that stay close to actually plausible usage, and temporal exploration, which probes the model's reasoning process rather than only its final answer. Guided by the knowledge graph, these explorations surface corner-case violations that fixed benchmarks miss. The paper reports that, across code generation and security-guidance evaluat

Load-bearing premise

The temporal stage presupposes that a target assistant's reasoning process is observable or steerable from outside, but deployed closed coding assistants do not expose internal reasoning traces, so this half of the pipeline is not guaranteed to work on the products it is meant to audit.

Editorial extensions

If this is right

  • Fixed benchmark suites must be expected to underestimate safety risk for deployed coding assistants, because ASTRA's knowledge-graph-guided probing finds 11-66% more violations on realistic requests.
  • Violation-inducing cases are not just diagnostic; they are usable training material, since models fine-tuned on ASTRA-generated cases show 17% more effective alignment.
  • Safety red-teaming can become model-specific and adaptive: the online knowledge-graph updates let the same pipeline shift its probes as the target model's failure modes are discovered.
  • The reported margins apply to the full pipeline, so they are conditional on access to the target's reasoning process; a closed assistant that hides such traces would receive only the spatial half.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: use ASTRA's violation cases as preference pairs for preference-based alignment and measure whether safety behavior transfers to held-out attack categories; the paper reports improved alignment training but does not characterize generalization.
  • The knowledge-graph scaffolding is domain-agnostic in structure, so porting it to legal, financial, or medical advice assistants is a plausible next step, but only the code and security-guidance domains are demonstrated here.
  • Because temporal exploration requires observable or steerable reasoning, the strongest form of ASTRA is most directly applicable to open-weight models; black-box commercial assistants would rely on spatial probing unless providers expose reasoning traces.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The manuscript proposes ASTRA, a three-stage automated red-teaming system for AI coding and security-guidance assistants. Stage 1 builds domain-specific knowledge graphs of software tasks and weaknesses; Stage 2 performs 'spatial' input-space probing and 'temporal' reasoning-process probing, both guided by the graphs; Stage 3 generates violation-inducing cases for use in alignment training. The abstract claims that across two major evaluation domains ASTRA finds 11–66% more issues than existing techniques and that its generated cases lead to 17% more effective alignment training. The reader-provided assessment and the stress-test note identify as the main risks the lack of experimental detail, potential circularity of the alignment-gain evaluation, and the reliance on access to target models' reasoning processes. The version of the manuscript I received has a largely corrupted full text, so I was unable to verify derivations, experimental protocols, or the numerical claims beyond the abstract.

Significance. If the claims are correct, the contribution is significant: a knowledge-graph-guided agent that adaptively probes both inputs and reasoning traces could address an acknowledged limitation of fixed-benchmark red-teaming, and the generated violations could be used to improve alignment. The three-stage architecture is a plausible and timely design. However, the paper as submitted does not provide enough verifiable support for these claims. I can credit the novelty of the framing and the potential usefulness of the approach, but not the quantitative results, because the evaluation details and the body of the paper are not accessible in a usable form. The 17% alignment-gain number, in particular, needs a demonstration that the evaluation is not self-referential.

major comments (5)
  1. [Full text / all sections] The manuscript body as received is corrupted and unreadable: equations and prose are garbled, and text from an unrelated arXiv paper (2508.03939v1 [astro-ph.CO]) appears in the file. I could not verify any derivation, algorithm description, or experimental protocol. This is not a cosmetic issue; the central claims depend on details that are currently absent from the version under review.
  2. [Abstract / Evaluation] The headline result '11–66% more issues than existing techniques' is unsupported by named evaluation domains, named baselines, sample sizes, run counts, or error bars. The range is very wide, and without knowing what the two 'major evaluation domains' are and what the baseline methods are, the number cannot be interpreted or reproduced.
  3. [Stage 3 / Alignment training] The claim that ASTRA-generated cases lead to 17% more effective alignment training is vulnerable to circularity. The abstract does not state what data are used for post-training evaluation. If the evaluation uses ASTRA-generated cases, or new cases sampled from the same knowledge-graph-guided generator, the improvement may simply reflect overfitting to the generator. The paper must specify a held-out evaluation set that is independent of the generated training cases, ideally using external benchmarks.
  4. [Stage 2 / Temporal exploration] The temporal exploration stage assumes the target model's reasoning processes are observable or steerable from outside. Deployed closed coding assistants do not generally expose chain-of-thought or internal state, and the abstract names no target model that provides such access. The paper should analyze the robustness of the reported gains to targets without reasoning-trace access, for example by ablating the temporal component and showing how much of the 11–66% improvement remains.
  5. [Stage 1 / Weakness catalog] The knowledge graphs in Stage 1 define the weakness catalog, and the evaluation measures how many of those weaknesses ASTRA can hit. This makes the vulnerability metric partially self-referential. An external validation of the catalog—for example, comparing ASTRA-found issues against independently labeled expert annotations or established vulnerability databases—is needed to establish that the catalog is not merely imposing its own taxonomy.
minor comments (4)
  1. [Abstract] The phrase '11–66%' should be reported with confidence intervals or per-domain breakdowns; a single two-number range is not a statistical summary.
  2. [Abstract] The term 'online vulnerability exploration' is used without a definition; it should be clarified whether 'online' means during deployment, during probing, or in an online-learning sense.
  3. [Related work] The manuscript does not appear to cite specific prior red-teaming benchmarks or recent work on chain-of-thought probing. A precise comparison with named baselines is necessary.
  4. [Reproducibility] No code, data, or reproducibility statement is visible. For a systems paper, releasing the knowledge-graph construction and probing code would materially help evaluation.

Circularity Check

0 steps flagged · score 0.0 of 10

No demonstrated circularity in the supplied text; the alignment-gain evaluation concern is a possible risk, not an established reduction.

full rationale

Based on the readable abstract and surrounding context, ASTRA is presented as an empirical system that builds knowledge graphs, probes target models, and reports comparative gains (11-66% more issues than existing techniques; 17% more effective alignment training). No passage in the supplied text defines the evaluation metric in terms of ASTRA's own generated cases or knowledge graphs by construction. The abstract's statement that ASTRA 'generates high-quality violation-inducing cases to improve model alignment' concerns training data, not the post-training evaluation distribution, so it does not by itself show that the 17% improvement is forced. The concern that the evaluation set might be drawn from the same generator would be a correctness/overfitting risk, but the manuscript text provided does not exhibit the required reduction (e.g., 'evaluation set = ASTRA-generated set' or Eq. X = Eq. Y by construction). The temporal-exploration reliance on observable reasoning processes is an applicability limitation, not circularity. No self-citation, fitted-parameter-renamed-as-prediction, or imported uniqueness theorem is evident. Therefore, on the available evidence, the derivation chain is not shown to be circular.

Assumptions & free parameters 1 free parameters · 2 assumptions · 0 invented entities

The knowledge graph is a software artifact of the system, not an independently testable physical or formal entity, so no invented-entity entries are recorded. One free parameter class (probing budgets) is recorded because such knobs typically control reported gains, though no values were visible.

free parameters (1)
  • Probing budgets, thresholds, and stopping criteria for spatial-temporal exploration
    The abstract describes adaptive probing but reports no exploration budgets, acceptance thresholds, or stopping rules. These knobs control how many violations are found and are the usual place where reported gains are tuned; values could not be checked because the full text is unreadable.
assumptions (2)
  • domain assumption The two evaluation domains and the 'realistic' request distribution are representative of what developers actually ask coding assistants.
    The abstract's stated advantage over prior work is realism ('requests that developers might actually ask'), but the domains are unnamed and the realism of the input distribution is not demonstrated in the abstract. If the request distribution is unrepresentative, the reported violation rates do not transfer.
  • domain assumption The target models' reasoning processes are observable or steerable for temporal exploration.
    Stage (2) probes 'its reasoning processes'. Closed commercial assistants do not expose chain-of-thought or internal state, and the abstract names no target models. If reasoning traces are unavailable, the temporal component collapses into ordinary input probing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants." pith.science (2026). https://pith.science/paper/FKCI5AIC

@misc{pith2026250803936,
  author       = {Pith},
  title        = {Pith review of: ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FKCI5AIC}},
  note         = {Machine review of arXiv:2508.03936}
}
read the original abstract

AI coding assistants like GitHub Copilot are rapidly transforming software development, but their safety remains deeply uncertain-especially in high-stakes domains like cybersecurity. Current red-teaming tools often rely on fixed benchmarks or unrealistic prompts, missing many real-world vulnerabilities. We present ASTRA, an automated agent system designed to systematically uncover safety flaws in AI-driven code generation and security guidance systems. ASTRA works in three stages: (1) it builds structured domain-specific knowledge graphs that model complex software tasks and known weaknesses; (2) it performs online vulnerability exploration of each target model by adaptively probing both its input space, i.e., the spatial exploration, and its reasoning processes, i.e., the temporal exploration, guided by the knowledge graphs; and (3) it generates high-quality violation-inducing cases to improve model alignment. Unlike prior methods, ASTRA focuses on realistic inputs-requests that developers might actually ask-and uses both offline abstraction guided domain modeling and online domain knowledge graph adaptation to surface corner-case vulnerabilities. Across two major evaluation domains, ASTRA finds 11-66% more issues than existing techniques and produces test cases that lead to 17% more effective alignment training, showing its practical value for building safer AI systems.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reasoning Up the Instruction Ladder for Controllable Language Models

    cs.CL 2025-10 conditional novelty 6.0 of 10

    RLVR on ~7K verifiable system/user conflict examples teaches LLMs to prioritize higher-priority instructions, improving instruction-hierarchy and safety benchmarks.

Reference graph

Works this paper leans on

42 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    https://repoaudit-home.github.io/index.html, 2025

    Repoaudit: Auditing code as human. https://repoaudit-home.github.io/index.html, 2025. Accessed: 2025-04-24

    work page 2025

  2. [2]

    Code Review Tool: Amazon CodeGuru Security

    Amazon. Code Review Tool: Amazon CodeGuru Security . https://aws.amazon.com/codeguru/, 2025. [Online; accessed 4-May-2025]

    work page 2025

  3. [3]

    Purple llama cyberseceval: A secure coding benchmark for language models

    Manish Bhatt, Sahana Chennabasappa, Cyrus Nikolaidis, Shengye Wan, Ivan Evtimov, Dominik Gabi, Daniel Song, Faizan Ahmad, Cornelius Aschermann, Lorenzo Fontana, et al. Purple llama cyberseceval: A secure coding benchmark for language models. arXiv preprint arXiv:2312.04724 , 2023

    arXiv 2023

  4. [4]

    Jailbreaking black box large language models in twenty queries

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419 , 2023

    arXiv 2023

  5. [5]

    When llm meets drl: Advancing jailbreaking efficiency via drl-guided search

    Xuan Chen, Yuzhou Nie, Wenbo Guo, and Xiangyu Zhang. When llm meets drl: Advancing jailbreaking efficiency via drl-guided search. arXiv preprint arXiv:2406.08705 , 2024

    arXiv 2024

  6. [6]

    Masterkey: Automated jailbreak across multiple large language model chatbots

    Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. Masterkey: Automated jailbreak across multiple large language model chatbots. arXiv preprint arXiv:2307.08715 , 2023

    arXiv 2023

  7. [7]

    A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily

    Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pape...

    work page 2024

  8. [8]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 , 2024

    arXiv 2024

Show all 42 references
  1. [9]

    How ai is being audited—and why it matters, 2024

    Financial Times . How ai is being audited—and why it matters, 2024. https://www.ft.com/content/8a54932d-d9a9-4a69-969d-89d8b2de149f

    2024

  2. [10]

    Stochastic relaxation, gibbs distributions, and the bayesian restoration of images

    Stuart Geman and Donald Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence , PAMI-6(6):721--741, 1984

    1984

  3. [11]

    Deliberative alignment: Reasoning enables safer language models

    Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, et al. Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339 , 2024

    2024 arXiv

  4. [12]

    competency

    Divij Handa, Zehua Zhang, Amir Saeidi, Shrinidhi Kumbhar, and Chitta Baral. When" competency" in reasoning opens the door to vulnerability: Jailbreaking llms via novel complex ciphers. arXiv preprint arXiv:2402.10601 , 2024

    2024

  5. [13]

    Large language models for code: Security hardening and adversarial testing

    Jingxuan He and Martin Vechev. Large language models for code: Security hardening and adversarial testing. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security , pages 1865--1879, 2023

    2023

  6. [14]

    Artprompt: Ascii art-based jailbreak attacks against aligned llms

    Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. Artprompt: Ascii art-based jailbreak attacks against aligned llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: L...

    2024

  7. [15]

    Red queen: Safeguarding large language models against concealed multi-turn jailbreaking

    Yifan Jiang, Kriti Aggarwal, Tanmay Laud, Kashif Munir, Jay Pujara, and Subhabrata Mukherjee. Red queen: Safeguarding large language models against concealed multi-turn jailbreaking. arXiv preprint arXiv:2409.17458 , 2024

    2024 arXiv

  8. [16]

    Multiverse: Exposing large language model alignment problems in diverse worlds

    Xiaolong Jin, Zhuo Zhang, and Xiangyu Zhang. Multiverse: Exposing large language model alignment problems in diverse worlds. arXiv preprint arXiv:2402.01706 , 2024

    2024 arXiv

  9. [17]

    Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers

    Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers. arXiv preprint arXiv:2402.16914 , 2024

    2024 arXiv

  10. [18]

    The reversal curse: Llms trained on ‘a is b’ fail to learn ‘b is a’

    Xisen Li, Jiefu Liu, Chunting Zhang, Colin Raffel, Kristina Tau, James Zou, and Dan Jurafsky. The reversal curse: Llms trained on ‘a is b’ fail to learn ‘b is a’. arXiv preprint arXiv:2305.13283 , 2023

    2023 arXiv

  11. [19]

    Deepinception: Hypnotize large language model to be jailbreaker

    Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. Deepinception: Hypnotize large language model to be jailbreaker. arXiv preprint arXiv:2311.03191 , 2023

    2023 arXiv

  12. [20]

    Making them ask and answer: Jailbreaking large language models in few queries via disguise and reconstruction

    Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen. Making them ask and answer: Jailbreaking large language models in few queries via disguise and reconstruction. In 33rd USENIX Security Symposium (USENIX Security 24) , pages 4711--4728, 2024

    2024

  13. [21]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In The Twelfth International Conference on Learning Representations , 2024

    2024

  14. [22]

    Flipattack: Jailbreak llms via flipping

    Yue Liu, Xiaoxin He, Miao Xiong, Jinlan Fu, Shumin Deng, and Bryan Hooi. Flipattack: Jailbreak llms via flipping. arXiv preprint arXiv:2410.02832 , 2024

    2024 arXiv

  15. [23]

    Codechameleon: Personalized encryption framework for jailbreaking large language models

    Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, Tao Gui, Qi Zhang, and Xuanjing Huang. Codechameleon: Personalized encryption framework for jailbreaking large language models. arXiv preprint arXiv:2402.16717 , 2024

    2024 arXiv

  16. [24]

    Ai in software market size, share & trends analysis report, 2023–2033, 2024

    Market.US . Ai in software market size, share & trends analysis report, 2023–2033, 2024. https://market.us/report/ai-in-software-market/

    2023

  17. [25]

    Tree of attacks: Jailbreaking black-box llms automatically

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box llms automatically. Advances in Neural Information Processing Systems , 37:61065--61105, 2024

    2024

  18. [26]

    Mitre att&ck framework

    MITRE Corporation . Mitre att&ck framework. https://attack.mitre.org/, 2024. Accessed: 2025-05-18

    2024

  19. [27]

    Allen Newell and Herbert A. Simon. Human Problem Solving . Prentice-Hall, Englewood Cliffs, NJ, 1972

    1972

  20. [28]

    Safety alignment should be made more than just a few tokens deep

    Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should be made more than just a few tokens deep. In ICLR , 2025

    2025

  21. [29]

    Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues

    Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao. Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues. arXiv preprint arXiv:2410.10700 , 2024

    2024

  22. [30]

    Amazon nova ai challenge, trusted ai: Advancing secure, ai-assisted software development

    Sattvik Sahai, Prasoon Goyal, Michael Johnston, Anna Gottardi, Yao Lu, Lucy Hu, Luke Dai, Shaohua Liu, Samyuth Sagi, Hangjie Shi, Desheng Zhang, Lavina Vaz, Leslie Ball, Maureen Murray, Rahul Gupta, and Shankar Ananthakrishnan. Amazon nova ai challenge, trusted ai: Advancing s...

    2025

  23. [31]

    Pal: Proxy-guided black-box attack on large language models

    Chawin Sitawarin, Norman Mu, David Wagner, and Alexandre Araujo. Pal: Proxy-guided black-box attack on large language models. arXiv preprint arXiv:2402.09674 , 2024

    2024 arXiv

  24. [32]

    All in how you ask for it: Simple black-box method for jailbreak attacks

    Kazuhiro Takemoto. All in how you ask for it: Simple black-box method for jailbreak attacks. Applied Sciences , 14(9):3558, 2024

    2024

  25. [33]

    Cognitive overload: Jailbreaking large language models with overloaded logical thinking

    Nan Xu, Fei Wang, Ben Zhou, Bang Zheng Li, Chaowei Xiao, and Muhao Chen. Cognitive overload: Jailbreaking large language models with overloaded logical thinking. arXiv preprint arXiv:2311.09827 , 2023

    2023 arXiv

  26. [34]

    Prosec: Fortifying code llms with proactive security alignment

    Xiangzhe Xu, Zian Su, Jinyao Guo, Kaiyuan Zhang, Zhenting Wang, and Xiangyu Zhang. Prosec: Fortifying code llms with proactive security alignment. arXiv preprint arXiv:2411.12882 , 2024

    2024 arXiv

  27. [35]

    Jigsaw puzzles: Splitting harmful questions to jailbreak large language models

    Hao Yang, Lizhen Qu, Ehsan Shareghi, and Gholamreza Haffari. Jigsaw puzzles: Splitting harmful questions to jailbreak large language models. arXiv preprint arXiv:2410.11459 , 2024

    2024 arXiv

  28. [36]

    Chain of attack: a semantic-driven contextual multi-turn attacker for llm

    Xikang Yang, Xuehai Tang, Songlin Hu, and Jizhong Han. Chain of attack: a semantic-driven contextual multi-turn attacker for llm. arXiv preprint arXiv:2405.05610 , 2024

    2024 arXiv

  29. [37]

    Fuzzllm: A novel and universal fuzzing framework for proactively discovering jailbreak vulnerabilities in large language models

    Dongyu Yao, Jianshu Zhang, Ian G Harris, and Marcel Carlsson. Fuzzllm: A novel and universal fuzzing framework for proactively discovering jailbreak vulnerabilities in large language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Proc...

    2024

  30. [38]

    Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts

    Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253 , 2023

    2023 arXiv

  31. [39]

    How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisti...

    2024

  32. [40]

    Make them spill the beans! coercive knowledge extraction from (production) llms, 2023

    Zhuo Zhang, Guangyu Shen, Guanhong Tao, Siyuan Cheng, and Xiangyu Zhang. Make them spill the beans! coercive knowledge extraction from (production) llms, 2023

    2023

  33. [41]

    Improving llm safety alignment with dual-objective optimization

    Xuandong Zhao, Will Cai, Tianneng Shi, David Huang, Licong Lin, Song Mei, and Dawn Song. Improving llm safety alignment with dual-objective optimization. arXiv preprint arXiv:2503.03710 , 2025

    2025

  34. [42]

    Improving alignment and robustness with circuit breakers

    Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, J Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.