Pith. sign in

REVIEW 4 cited by

Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attack

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.01109 v6 pith:P3SQX66C submitted 2024-02-02 cs.LG cs.CL

Tiansheng Huang , Sihao Hu , Ling Liu This is my paper · ORCID
classification cs.LGcs.CL
keywords harmfulvaccinealignmentfinetuningalignment-brokenattackdatadrift
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The new paradigm of finetuning-as-a-service introduces a new attack surface for Large Language Models (LLMs): a few harmful data uploaded by users can easily trick the finetuning to produce an alignment-broken model. We conduct an empirical analysis and uncover a \textit{harmful embedding drift} phenomenon, showing a probable cause of the alignment-broken effect. Inspired by our findings, we propose Vaccine, a perturbation-aware alignment technique to mitigate the security risk of users finetuning. The core idea of Vaccine is to produce invariant hidden embeddings by progressively adding crafted perturbation to them in the alignment phase. This enables the embeddings to withstand harmful perturbation from un-sanitized user data in the finetuning phase. Our results on open source mainstream LLMs (e.g., Llama2, Opt, Vicuna) demonstrate that Vaccine can boost the robustness of alignment against harmful prompts induced embedding drift while reserving reasoning ability towards benign prompts. Our code is available at \url{https://github.com/git-disl/Vaccine}.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Vulnerability-Aware Alignment splits safety training data into fragile and robust groups, then uses group robust optimization and adversarial perturbations, cutting harmful response rates after harmful fine-tuning by ...

  2. MoGU V2: Toward a Higher Pareto Frontier Between Model Usability and Security

    cs.CL 2025-09 conditional novelty 5.0 of 10

    MoGUv2 embeds small routers in the deeper layers of LLMs to dynamically blend a helpful variant and a refusal variant, improving safety against jailbreak and fine-tuning attacks while preserving usability.

  3. Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraint

    cs.CL 2025-09 conditional novelty 5.0 of 10

    ProCon anchors each sample's hidden-state projection onto the LLM's initial refusal direction during instruction fine-tuning, reducing refusal-direction drift and safety risks with limited task-performance loss.

  4. Pruning Strategies for Backdoor Defense in LLMs

    cs.LG 2025-08 conditional novelty 4.0 of 10

    Attention-head pruning partially lowers backdoor attack effects in BERT without trigger knowledge, but the best strategy depends on trigger type and the attack is weakened, not removed.

Pith tools