Pith. sign in

REVIEW 4 major objections 7 minor 58 references

Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a language model can detect jailbreak attacks during inference and, by fine-tuning a LoRA adapter on the detected examples in an online fashion, drive the attack success rate to zero without prior offline training on…

desk verdict A genuinely new online jailbreak defense with a clever detector, but the 'complete immunity' claim outruns the evidence on generalization. read the letter →

arxiv 2505.22271 v1 pith:PRTKP5AX submitted 2025-05-28 cs.CR cs.AIcs.CL

Yongcan Yu , Yanbo Wang , Ran He , Jian Liang This is my paper · ORCID
classification cs.CRcs.AIcs.CL
keywords test-timedefensejailbreakdetectiongisttokenLoRAfine-tuningmultimodallargelanguagemodelsadaptationLLMsafetyadaptive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Test-time Immunization (TIM), a defense framework that lets a deployed language model adapt to jailbreak attacks as they happen rather than being pre-trained against every possible attack. A lightweight detector, built from a trainable gist token and a binary classifier over intermediate hidden states, flags harmful responses during generation. When a jailbreak is detected, the model fine-tunes a LoRA adapter on the jailbreak instruction paired with a refusal response, while the detector is updated separately so the fine-tuning does not degrade detection. The paper reports that after only a handful of attack examples, the attack success rate on the last half of the test stream drops to 0% across Figstep, MM-SafetyBench, and I-FSJ, with near-zero over-defense on normal instructions. If correct, models can become immune to a new jailbreak type within a single test stream, using only the attack itself as training data.

What carries the argument

The core object is the gist token: a trainable token inserted at the end of the sequence whose intermediate-layer hidden state is classified by a small binary head to determine whether the model has been jailbroken. Because the keys and values of earlier tokens are cached during autoregressive generation, the gist token's hidden state can be computed at roughly 1/1000 of the generation cost, making detection overhead about 0.4%. Two design choices carry the argument: decoupling the detector from the defense by using intermediate-layer features for detection and later-layer LoRA for fine-tuning, so online parameter updates do not degrade detection; and test-time adaptation of the detector on detected jailbreak pairs, which recovers much of the recall lost to distribution shift. Regularization with clean question-answering data during fine-tuning keeps the model from over-rejecting normal instructions.

What would settle it

Run TIM against a long-prompt jailbreak (e.g., I-FSJ) with the detector fixed rather than adapted; the paper reports the frozen detector's TPR drops to 18.2%, so if ASR-50 stays well above zero, the universal-defense claim fails exactly when detection recall collapses.

Watch

Extended reading notes

Core claim

TIM's central claim is that jailbreak detection is substantially easier than jailbreak defense, so a model can convert a single detected attack into a targeted defense. The framework trains a gist token that is appended to the end of the input sequence; its hidden state, read from an intermediate layer, is fed to a binary classifier that decides whether the model's response should be flagged as jailbroken. When an attack is detected, the framework stores the jailbreak instruction together with a refusal answer in jailbreak memory and performs supervised fine-tuning of a LoRA adapter on the later transformer layers. Because the detector reads intermediate-layer features and the LoRA updates touch only later layers, the fine-tuning does not corrupt detection. The detector is also adapted during testing on the collected jailbreak examples, which the paper credits with raising true-positive detection from 18.2% to 40% on LLaMA3 under the long-prompt I-FSJ attack. The reported result is 0% ASR-50 on all jailbreak attack datasets, with the model needing fewer than 10 examples on Figstep to learn the defense.

Load-bearing premise

The whole defense starts only after the detector flags an attack, so the method rests on the detector staying accurate enough on real jailbreak prompts, which can be far longer than the short examples it was trained on (the paper notes a drop to 18.2% true-positive rate on LLaMA3 under I-FSJ without detector adaptation).

Editorial extensions

If this is right

  • A deployed model can reach near-zero attack success on a jailbreak type after seeing only a handful of attacks in the test stream (fewer than 10 on Figstep), without any offline training on that attack.
  • The same framework covers both text-only and image-based jailbreaks on LLMs and multimodal LLMs, so one defense pipeline replaces attack-specific methods such as rephrasing or image-shielding.
  • Test-time adaptation of the detector is what rescues detection on long prompts, lifting TPR on LLaMA3 under I-FSJ from 18.2% to 40% and keeping FPR near zero.
  • Mixing clean question-answering data into the fine-tuning keeps over-defense low, so normal instructions continue to be answered while jailbreak instructions are refused.
  • The defense learned on one attack transfers to simpler attacks (defending on MM-SafetyBench transfers to Figstep), though not in the harder direction; online adaptation is what covers emerging attack types.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the claim holds, TIM could be combined with static defenses: a static guard reduces the attack stream to a trickle, and TIM learns to suppress whatever leaks through, giving two layers that cover different failure modes.
  • The reliance on the detector suggests a natural adaptive attack: craft jailbreak prompts that keep the gist token's hidden state close to benign responses (for example, by constraining prompt length or paraphrasing), forcing the defense to run on the weaker, unadapted detector.
  • The decoupling principle may generalize: any online fine-tuning that must not disturb a concurrent probe (a safety monitor, a factuality checker, a quality classifier) could adopt intermediate-layer detection with later-layer training.
  • A testable extension is a systematic sweep over attack mixtures and lengths to find where the 0% ASR-50 result starts to degrade; the paper's hybrid-attack experiment suggests graceful degradation, but does not map the boundary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes TIM, a test-time defense framework for LLMs and MLLMs. It trains a gist token and a binary classifier to detect jailbroken responses during generation with low overhead. When a jailbreak is detected, the framework fine-tunes a LoRA adapter on the jailbreak instruction paired with a refusal response, and also updates the detector via test-time adaptation. The authors evaluate on Figstep, MM-SafetyBench, I-FSJ, and GCG, reporting near-zero ASR and 0% ASR-50 on all attack datasets, with ablations supporting the value of the gist token and the decoupled design.

Significance. If the claims held, this would be a valuable step toward deployable defenses that adapt online to evolving jailbreaks. The gist-token detector is cheap, and the decoupling mechanism is a sensible way to protect detector stability during safety fine-tuning. However, the central evidence for the 'universal' claim is weakened by the circularity of the ASR-50 metric and by the unverified assumption of LoRA generalization when detector TPR is low. The reported 0.0 ASR-50 on LLaMA3 under I-FSJ with only 40% TPR is a strong claim that requires direct measurement. The paper is transparent about limitations (Appendix E), which is appreciated, but the experiments need restructuring to support the title's scope.

major comments (4)
  1. [§4.3, Table 3] The reported 0.0 ASR-50 for LLaMA3 under I-FSJ is not directly supported by the detector's measured TPR. After test-time adaptation the detector catches only 40.0% of jailbreak prompts; the remaining 60% never enter the jailbreak memory Mj and therefore never produce a LoRA training signal. Obtaining 0.0 ASR-50 in the last half of the stream therefore requires that the LoRA, trained on the detected 40%, generalizes to the undetected 60%. No experiment partitions the stream into adaptation and held-out evaluation, and no per-sample analysis of rejection rates conditional on detection is reported. Please add such a breakdown or an evaluation on a disjoint held-out set of jailbreak prompts.
  2. [§4.2, Eq. (5)] ASR-50 is computed on the last 50% of the same test stream that is used for online adaptation. Detected jailbreak prompts are immediately added to Mj and fine-tuned; these same prompts can appear in the last 50% and are scored there. Thus a near-zero ASR-50 is partly by construction and does not demonstrate immunity to unseen attacks. For the 'universal' claim, the evaluation should use a held-out set of jailbreak prompts from the same attack family (or a new family) that is not used for adaptation.
  3. [§4.3, Table 7] The static transferability experiment shows ASR 84.3% after adapting to Figstep and evaluating on MM-SafetyBench (only a 15.5-point reduction from the vanilla 99.8). This result suggests the learned defense is substantially attack-specific, which undercuts the 'universal' framing. While the authors correctly state the method is online, the paper should either provide evidence that online adaptation to MM-SafetyBench attacks after Figstep exposure reduces ASR quickly, or temper the universality claim.
  4. [§A.3] All experiments appear to be single runs without multiple seeds or error bars. Given the stochasticity of online fine-tuning (batch composition, optimizer, training steps), the strong 0.0 ASR-50 values may not be stable. At minimum, the main tables should report mean and standard deviation over at least three independent runs.
minor comments (7)
  1. [§1] The heading 'Introdcution' should be 'Introduction'.
  2. [Table 5] The column labels in Table 5 are misaligned and difficult to parse; please reformat the table so that the reader can clearly see the breakdown of inference cost, detection cost, and training cost.
  3. [References] References [7] and [8] appear to be the same paper (Gou et al., 'Eyes closed, safety on'); please remove the duplicate citation.
  4. [§A.4] The word 'respones' should be 'responses' in the description of prefix matching.
  5. [Algorithm 1] The label 'Initailize' should be 'Initialize'.
  6. [§4.3] In the 'Results under Different Jailbreak Data Ratios' subsection, the text says 'The results presented in the table' but the results are shown in Figure 5; please correct the reference.
  7. [Appendix E] The phrase 'larger model (70 B+)' should be 'larger model (70B+)' for consistency with standard notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: TIM's defense results are empirical online-adaptation measurements, not predictions forced by fitted inputs or self-citations.

full rationale

I walked the claimed derivation chain. There is no mathematical derivation from first principles; TIM is an empirical system: a gist-token detector is trained with cross-entropy (Eq. 4), detected jailbreaks are stored in Mj, and a LoRA adapter is trained online on (jailbreak instruction, refusal) pairs. The headline ASR-50 metric (Eq. 5, Section 4.1) measures ASR on the last half of the test stream after the model has already adapted on earlier detected jailbreaks. Because jailbreak samples appear once in the stream and ASR-50 is computed on later, not previously collected, samples, near-zero ASR-50 is an online generalization measurement, not a metric that is equal to the training objective by construction. The same reasoning covers the detector: test-time adaptation of Cd uses Md constructed from the stream, so accumulated TPR/ACC figures are transductive and can be inflated by memorization, but this is an evaluation-protocol weakness, not a circular derivation. The self-citations ([9], [18], [36], [42], [50]) are used only to position the work in test-time adaptation and backdoor-defense literature; none is load-bearing, none imports a uniqueness theorem, and none smuggles an ansatz. The honest negative result in Table 7 (Figstep to MM-SafetyBench ASR 84.3) and the admitted limitation in Appendix E (detector decay on extremely long inputs) further show the authors are not forcing the central claim by construction. I find no instance where an equation or fitted parameter reduces to the result it is said to predict, so no circularity step can be quoted.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

This paper is an empirical defense method, not a first-principles derivation. The free parameters are hand-chosen hyperparameters, the main domain assumption is that hidden-state classification on an appended token can detect harmful responses, and the only invented entity is the trainable gist token. No new physical or mathematical entities are postulated.

free parameters (6)
  • Detector training hyperparameters (lr, epochs, batch size) = 1e-3, 5, 32
    Section A.3; chosen by hand, detector accuracy depends on them.
  • Defense LoRA learning rate = 0.001 (MLLMs), 0.002 (LLMs)
    Section A.3; not justified by any selection criterion.
  • LoRA rank and layer scope = rank 16, last 15 transformer blocks
    Section A.3; arbitrary architectural choices that shape adaptation capacity.
  • Batch sizes for regularization, refusal training, and detector training = 40, 1, 6
    Section A.3; tuned for stability of the online updates.
  • Test-time training steps = 1 defense step, 5 detector steps per jailbreak
    Section A.3; directly controls adaptation speed and is chosen by hand.
  • Jailbreak to normal test data ratio = 1:1 in main runs; varied from 1:0.5 to 1:4 in Figure 5
    Section A.3 and Figure 5; the evaluation curve depends on how many attack samples appear in the test stream.
assumptions (4)
  • domain assumption Answer harmfulness can be extracted from the hidden state of an appended gist token using a binary classifier.
    Central to detection; no formal guarantee, supported only by empirical accuracy on a handful of attacks.
  • domain assumption Prefix matching against a fixed refusal list is a valid measure of whether a response is safe.
    Section A.4; responses that do not start with these prefixes are counted as successful jailbreaks, even if they are semantically safe or partially harmful.
  • domain assumption LoRA applied to layers after the intermediate detection layer does not alter the detection-relevant hidden states.
    Section 3.4; grounded by ablations but not by a formal argument about gradient flow or feature drift.
  • domain assumption Fine-tuning on a small number of detected jailbreak refusals generalizes to other instances of the same or later attacks.
    Section 3.3; transfer evidence is partial, since adapting on Figstep leaves 84.3% ASR on MM-SafetyBench.
invented entities (1)
  • Gist token tg
    purpose: Trainable token appended after the generated response to summarize harmfulness for the jailbreak classifier.
    New component introduced by this paper; its usefulness is supported only by internal ablations such as TIM (w/o gist), with no external benchmark or formal property.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models." pith.science (2026). https://pith.science/paper/PRTKP5AX

@misc{pith2026250522271,
  author       = {Pith},
  title        = {Pith review of: Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PRTKP5AX}},
  note         = {Machine review of arXiv:2505.22271}
}
read the original abstract

While (multimodal) large language models (LLMs) have attracted widespread attention due to their exceptional capabilities, they remain vulnerable to jailbreak attacks. Various defense methods are proposed to defend against jailbreak attacks, however, they are often tailored to specific types of jailbreak attacks, limiting their effectiveness against diverse adversarial strategies. For instance, rephrasing-based defenses are effective against text adversarial jailbreaks but fail to counteract image-based attacks. To overcome these limitations, we propose a universal defense framework, termed Test-time IMmunization (TIM), which can adaptively defend against various jailbreak attacks in a self-evolving way. Specifically, TIM initially trains a gist token for efficient detection, which it subsequently applies to detect jailbreak activities during inference. When jailbreak attempts are identified, TIM implements safety fine-tuning using the detected jailbreak instructions paired with refusal answers. Furthermore, to mitigate potential performance degradation in the detector caused by parameter updates during safety fine-tuning, we decouple the fine-tuning process from the detection module. Extensive experiments on both LLMs and multimodal LLMs demonstrate the efficacy of TIM.

Figures

Figures reproduced from arXiv: 2505.22271 by the authors.

Figure 1
Figure 1. Similar to a biological immune system, TIM aims to progressively enhance its resistance against various [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. The overview of test-time immunization. (1): The LLMs with pre-guarded strategy can defend against [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Detailed workflow of test-time immunization. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Performance of different variants of the proposed method. All metrics are normalized, and the methods [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 5
Figure 5. Figure 5: Experimental results under different jailbreak data ratios. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 4
Figure 4. Figure 4: Results under hybrid jailbreak attack. We [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Changes in metrics during the test process against Figstep. TIM-NA represents TIM (w/o adapt.) [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Changes in metrics during the testing against MM-SafetyBench. TIM-NA represents TIM (w/o adapt.) [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 32 canonical work pages

  1. [1]

    Jail- breaking black box large language models in twenty queries

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jail- breaking black box large language models in twenty queries. InWorkshop on Proc. NeurIPS, 2024

    work page 2024

  2. [2]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.See https://vicuna

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6, 2023

    2023

  3. [3]

    Security and privacy challenges of large language models: A survey.ACM Computing Surveys, 2024

    Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. Security and privacy challenges of large language models: A survey.ACM Computing Surveys, 2024

    work page 2024

  4. [4]

    Libre: A practical bayesian approach to adversarial detection

    Zhijie Deng, Xiao Yang, Shizhen Xu, Hang Su, and Jun Zhu. Libre: A practical bayesian approach to adversarial detection. InProc. CVPR, 2021

    work page 2021

  5. [5]

    Immune: Improving safety against jailbreaks in multi-modal llms via inference-time alignment.arXiv preprint arXiv:2411.18688, 2024

    Soumya Suvra Ghosal, Souradip Chakraborty, Vaibhav Singh, Tianrui Guan, Mengdi Wang, Ahmad Beirami, Furong Huang, Alvaro Velasquez, Dinesh Manocha, and Amrit Singh Bedi. Immune: Improving safety against jailbreaks in multi-modal llms via inference-time alignment.arXiv preprint arXiv:2411.18688, 2024

    arXiv 2024

  6. [6]

    Figstep: Jailbreaking large vision-language models via typographic visual prompts.arXiv preprint arXiv:2311.05608, 2023

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. Figstep: Jailbreaking large vision-language models via typographic visual prompts.arXiv preprint arXiv:2311.05608, 2023

    arXiv 2023

  7. [8]

    Eyes closed, safety on: Protecting multimodal llms via image-to-text transformation

    Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T Kwok, and Yu Zhang. Eyes closed, safety on: Protecting multimodal llms via image-to-text transformation. InProc. ECCV, 2024

    work page 2024

  8. [9]

    Backdoor defense via test-time detecting and repairing

    Jiyang Guan, Jian Liang, and Ran He. Backdoor defense via test-time detecting and repairing. InProc. CVPR, 2024

    work page 2024

Show all 58 references
  1. [10]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InProc. ICLR, 2022

    2022

  2. [11]

    Token-level adversarial prompt detection based on perplexity measures and contextual information.arXiv preprint arXiv:2311.11509, 2023

    Zhengmian Hu, Gang Wu, Saayan Mitra, Ruiyi Zhang, Tong Sun, Heng Huang, and Viswanathan Swaminathan. Token-level adversarial prompt detection based on perplexity measures and contextual information.arXiv preprint arXiv:2311.11509, 2023

    2023 arXiv

  3. [12]

    Baseline defenses for adversarial attacks against aligned language models.arXiv preprint arXiv:2309.00614, 2023

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models.arXiv preprint arXiv:2309.00614, 2023. 9

    2023 arXiv

  4. [13]

    Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    arXiv

  5. [14]

    Jailbreakzoo: Survey, landscapes, and horizons in jailbreaking large language and vision-language models.arXiv preprint arXiv:2407.01599, 2024

    Haibo Jin, Leyang Hu, Xinuo Li, Peiyan Zhang, Chonghan Chen, Jun Zhuang, and Haohan Wang. Jailbreakzoo: Survey, landscapes, and horizons in jailbreaking large language and vision-language models.arXiv preprint arXiv:2407.01599, 2024

    2024

  6. [15]

    A survey of reinforcement learning from human feedback.arXiv preprint arXiv:2312.14925, 2023

    Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. A survey of reinforcement learning from human feedback.arXiv preprint arXiv:2312.14925, 2023

    2023

  7. [16]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  8. [17]

    Certifying llm safety against adversarial prompting.arXiv preprint arXiv:2309.02705, 2023

    Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Aaron Jiaxun Li, Soheil Feizi, and Himabindu Lakkaraju. Certifying llm safety against adversarial prompting.arXiv preprint arXiv:2309.02705, 2023

    2023 arXiv

  9. [18]

    A comprehensive survey on test-time adaptation under distribution shifts

    Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribution shifts. International Journal of Computer Vision, pages 1–34, 2024

    2024

  10. [19]

    Improving adversarial robustness for 3d point cloud recognition at test-time through purified self-training.arXiv preprint arXiv:2409.14940, 2024

    Jinpeng Lin, Xulei Yang, Tianrui Li, and Xun Xu. Improving adversarial robustness for 3d point cloud recognition at test-time through purified self-training.arXiv preprint arXiv:2409.14940, 2024

    2024 arXiv

  11. [20]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InProc. ECCV, 2014

    2014

  12. [21]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. InProc. NeurIPS, 2023

    2023

  13. [22]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProc. CVPR, 2024

    2024

  14. [23]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https://llava-vl.github.io/ blog/2024-01-30-llava-next/

    2024

  15. [24]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. InProc. ICLR, 2024

    2024

  16. [25]

    Mm-safetybench: A benchmark for safety evaluation of multimodal large language models

    Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. Mm-safetybench: A benchmark for safety evaluation of multimodal large language models. InProc. ECCV, 2024

    2024

  17. [26]

    A comprehensive overview of large language models.arXiv preprint arXiv:2307.06435, 2023

    Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. A comprehensive overview of large language models.arXiv preprint arXiv:2307.06435, 2023

    2023 arXiv

  18. [27]

    Dad: Data-free adversarial defense at test time

    Gaurav Kumar Nayak, Ruchit Rawal, and Anirban Chakraborty. Dad: Data-free adversarial defense at test time. InProc. WACV, pages 3562–3571, 2022

    2022

  19. [28]

    Gpt-4 technical report

    R OpenAI. Gpt-4 technical report. arxiv 2303.08774.View in Article, 2(5), 2023

    2023 arXiv

  20. [29]

    Rapid response: Mitigating llm jailbreaks with a few examples.arXiv preprint arXiv:2411.07494, 2024

    Alwin Peng, Julian Michael, Henry Sleight, Ethan Perez, and Mrinank Sharma. Rapid response: Mitigating llm jailbreaks with a few examples.arXiv preprint arXiv:2411.07494, 2024

    2024 arXiv

  21. [30]

    Instruction tuning with gpt-4

    Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023

    2023 arXiv

  22. [31]

    Llm self defense: By self examination, llms know they are being tricked

    Mansi Phute, Alec Helbling, Matthew Daniel Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. Llm self defense: By self examination, llms know they are being tricked. InThe Second Tiny Papers Track at ICLR, 2024

    2024

  23. [32]

    Mllm-protector: Ensuring mllm’s safety without hurting performance.Proc

    Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang. Mllm-protector: Ensuring mllm’s safety without hurting performance.Proc. EMNLP, 2024

    2024

  24. [33]

    Visual adversarial examples jailbreak aligned large language models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. InProc. AAAI, 2024

    2024

  25. [34]

    Improving language understanding by generative pre-training

    Alec Radford. Improving language understanding by generative pre-training. 2018. 10

    2018

  26. [35]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProc. CVPR, 2022

    2022

  27. [36]

    Can we trust the unlabeled target data? towards backdoor attack and defense on model adaptation.arXiv preprint arXiv:2401.06030, 2024

    Lijun Sheng, Jian Liang, Ran He, Zilei Wang, and Tieniu Tan. Can we trust the unlabeled target data? towards backdoor attack and defense on model adaptation.arXiv preprint arXiv:2401.06030, 2024

    2024 arXiv

  28. [37]

    Learning to summarize with human feedback

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea V oss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. InProc. NeurIPS, 2020

    2020

  29. [38]

    Measuring the accuracy of diagnostic systems.Science, 240(4857):1285–1293, 1988

    John A Swets. Measuring the accuracy of diagnostic systems.Science, 240(4857):1285–1293, 1988

    1988

  30. [39]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  31. [40]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  32. [41]

    Tent: Fully test-time adaptation by entropy minimization

    Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. InProc. ICLR, 2021

    2021

  33. [42]

    Do we really need curated malicious data for safety alignment in multi-modal large language models?arXiv preprint arXiv:2504.10000, 2025

    Yanbo Wang, Jiyang Guan, Jian Liang, and Ran He. Do we really need curated malicious data for safety alignment in multi-modal large language models?arXiv preprint arXiv:2504.10000, 2025

    2025 arXiv

  34. [43]

    Defending llms against jailbreaking attacks via backtranslation

    Yihan Wang, Zhouxing Shi, Andrew Bai, and Cho-Jui Hsieh. Defending llms against jailbreaking attacks via backtranslation. InProc. ACL Findings, 2024

    2024

  35. [44]

    Adashield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting

    Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, and Chaowei Xiao. Adashield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting. InProc. ECCV, 2024

    2024

  36. [45]

    Jailbreak and guard aligned language models with only few in-context demonstrations.arXiv preprint arXiv:2310.06387, 2023

    Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. Jailbreak and guard aligned language models with only few in-context demonstrations.arXiv preprint arXiv:2310.06387, 2023

    2023 arXiv

  37. [46]

    Defending chatgpt against jailbreak attack via self-reminders.Nature Machine Intelligence, 5(12):1486–1496, 2023

    Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. Defending chatgpt against jailbreak attack via self-reminders.Nature Machine Intelligence, 5(12):1486–1496, 2023

    2023

  38. [47]

    Gradsafe: Detecting jailbreak prompts for llms via safety-critical gradient analysis

    Yueqi Xie, Minghong Fang, Renjie Pi, and Neil Gong. Gradsafe: Detecting jailbreak prompts for llms via safety-critical gradient analysis. InProc. ACL, 2024

    2024

  39. [48]

    Wizardlm: Empowering large pre-trained language models to follow complex instructions

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. Wizardlm: Empowering large pre-trained language models to follow complex instructions. In Proc. ICLR, 2024

    2024

  40. [49]

    Jailbreak attacks and defenses against large language models: A survey.arXiv preprint arXiv:2407.04295, 2024

    Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. Jailbreak attacks and defenses against large language models: A survey.arXiv preprint arXiv:2407.04295, 2024

    2024 arXiv

  41. [50]

    Stamp: Outlier-aware test-time adaptation with stable memory replay

    Yongcan Yu, Lijun Sheng, Ran He, and Jian Liang. Stamp: Outlier-aware test-time adaptation with stable memory replay. InProc. ECCV, 2024

    2024

  42. [51]

    Instruction tuning for large language models: A survey.arXiv preprint arXiv:2308.10792, 2023

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. Instruction tuning for large language models: A survey.arXiv preprint arXiv:2308.10792, 2023

    2023

  43. [52]

    Jailguard: A universal detection framework for llm prompt-based attacks.arXiv preprint arXiv:2312.10766, 2024

    Xiaoyu Zhang, Cen Zhang, Tianlin Li, Yihao Huang, Xiaojun Jia, Ming Hu, Jie Zhang, Yang Liu, Shiqing Ma, and Chao Shen. Jailguard: A universal detection framework for llm prompt-based attacks.arXiv preprint arXiv:2312.10766, 2024

    2024 arXiv

  44. [53]

    Defending large language models against jailbreaking attacks through goal prioritization

    Zhexin Zhang, Junxiao Yang, Pei Ke, Fei Mi, Hongning Wang, and Minlie Huang. Defending large language models against jailbreaking attacks through goal prioritization. InProc. ACL, 2024

    2024

  45. [54]

    The first to know: How token distributions reveal hidden knowledge in large vision-language models? InProc

    Qinyu Zhao, Ming Xu, Kartik Gupta, Akshay Asthana, Liang Zheng, and Stephen Gould. The first to know: How token distributions reveal hidden knowledge in large vision-language models? InProc. ECCV, 2024

    2024

  46. [55]

    A survey of large language models.arXiv preprint arXiv:2303.18223, 2023

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models.arXiv preprint arXiv:2303.18223, 2023. 11

    2023 arXiv

  47. [56]

    Improved few-shot jailbreaking can circumvent aligned language models and their defenses

    Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Jing Jiang, and Min Lin. Improved few-shot jailbreaking can circumvent aligned language models and their defenses. InProc. NeurIPS, 2024

    2024

  48. [57]

    Minigpt-4: Enhancing vision- language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision- language understanding with advanced large language models. InProc. ICLR, 2024

    2024

  49. [58]

    Safety fine-tuning at (almost) no cost: A baseline for vision large language models

    Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales. Safety fine-tuning at (almost) no cost: A baseline for vision large language models. InProc. ICML, 2024

    2024

  50. [59]

    Universal and transferable adversarial attacks on aligned language models.arXiv preprint arXiv:2307.15043, 2023

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models.arXiv preprint arXiv:2307.15043, 2023. A The Details of Experimental Setup A.1 Dataset Construction To construct th...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.