Pith. sign in

REVIEW 5 cited by

Uncovering Safety Risks of Large Language Models through Concept Activation Vector

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.12038 v5 pith:NXOEHGX4 submitted 2024-04-18 cs.CL

Zhihao Xu , Ruixuan Huang , Changyu Chen , Xiting Wang This is my paper
classification cs.CL
keywords llmssafetyattackattacksrisksactivationconceptcurrent
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Despite careful safety alignment, current large language models (LLMs) remain vulnerable to various attacks. To further unveil the safety risks of LLMs, we introduce a Safety Concept Activation Vector (SCAV) framework, which effectively guides the attacks by accurately interpreting LLMs' safety mechanisms. We then develop an SCAV-guided attack method that can generate both attack prompts and embedding-level attacks with automatically selected perturbation hyperparameters. Both automatic and human evaluations demonstrate that our attack method significantly improves the attack success rate and response quality while requiring less training data. Additionally, we find that our generated attack prompts may be transferable to GPT-4, and the embedding-level attacks may also be transferred to other white-box LLMs whose parameters are known. Our experiments further uncover the safety risks present in current LLMs. For example, in our evaluation of seven open-source LLMs, we observe an average attack success rate of 99.14%, based on the classic keyword-matching criterion. Finally, we provide insights into the safety mechanism of LLMs. The code is available at https://github.com/SproutNan/AI-Safety_SCAV.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail

    cs.LG 2025-12 conditional novelty 6.0 of 10

    Reliable concept presence in transformers is concentrated in the extreme high-activation tail of in-concept tokens; thresholding that tail improves concept detection and localization.

  2. Probing the Difficulty Perception Mechanism of Large Language Models

    cs.CL 2025-10 conditional novelty 6.0 of 10

    LLMs linearly encode math-problem difficulty in their final-token representations, and specific final-layer attention heads are specialized for easy vs hard problems.

  3. Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks

    cs.CL 2025-08 conditional novelty 6.0 of 10

    IMAGINE pre-synthesizes intent-concealed jailbreak-like instructions via iterative latent-space expansion, and DPO with that data reduces jailbreak attack success rates on Qwen2.5, Llama3.1 and Llama3.2.

  4. Internal Value Alignment in Large Language Models through Controlled Value Vector Activation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    ConVA identifies value-specific directions in an LLM's internal activations from context-matched GPT-4o-generated examples and gates minimal activation steering to control outputs across ten Schwartz values.

  5. Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

    cs.CL 2025-08 reject novelty 5.0 of 10

    Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.

Pith tools