REVIEW 13 cited by
Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Larger language models (LLMs) have taken the world by storm with their massive multi-tasking capabilities simply by optimizing over a next-word prediction objective. With the emergence of their properties and encoded knowledge, the risk of LLMs producing harmful outputs increases, making them unfit for scalable deployment for the public. In this work, we propose a new safety evaluation benchmark RED-EVAL that carries out red-teaming. We show that even widely deployed models are susceptible to the Chain of Utterances-based (CoU) prompting, jailbreaking closed source LLM-based systems such as GPT-4 and ChatGPT to unethically respond to more than 65% and 73% of harmful queries. We also demonstrate the consistency of the RED-EVAL across 8 open-source LLMs in generating harmful responses in more than 86% of the red-teaming attempts. Next, we propose RED-INSTRUCT--An approach for the safety alignment of LLMs. It constitutes two phases: 1) HARMFULQA data collection: Leveraging CoU prompting, we collect a dataset that consists of 1.9K harmful questions covering a wide range of topics, 9.5K safe and 7.3K harmful conversations from ChatGPT; 2) SAFE-ALIGN: We demonstrate how the conversational dataset can be used for the safety alignment of LLMs by minimizing the negative log-likelihood over helpful responses and penalizing over harmful responses by gradient accent over sample loss. Our model STARLING, a fine-tuned Vicuna-7B, is observed to be more safely aligned when evaluated on RED-EVAL and HHH benchmarks while preserving the utility of the baseline models (TruthfulQA, MMLU, and BBH).
Forward citations
Cited by 13 Pith papers
-
Guiding LLM Decision-Making with Fairness Reward Models
A single process-level reward model, trained on weakly labeled biased versus unbiased reasoning, transfers across tasks and models to reduce equalized odds gaps in LLM decision-making.
-
No Single Neuron of Failure: Distributed Safety Alignment Against White-Box Attacks
A new white-box defense, DSA, masks a model's strongest refusal neurons and applies dropout during training so safety is redistributed across many compensatory neurons, cutting attack success under heavy pruning from ...
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
Internal Value Alignment in Large Language Models through Controlled Value Vector Activation
ConVA identifies value-specific directions in an LLM's internal activations from context-matched GPT-4o-generated examples and gates minimal activation steering to control outputs across ten Schwartz values.
-
CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representations
A GAN learns to shift malicious prompts into the safe region of an LLM's internal embedding space, and its discriminator is reused as a no-fine-tuning defense filter.
-
MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation
MGC, a two-stage compiler framework, generates functional malware by decomposing malicious intents into benign-appearing MDIR components that strong aligned LLMs will implement, bypassing safety alignment.
-
PL-Guard: Benchmarking Language Model Safety for Polish
A small Polish BERT classifier proved more robust than larger fine-tuned LLMs at classifying safe versus unsafe Polish content, including under character-level adversarial perturbations.
-
Hatevolution: What Static Benchmarks Don't Tell Us
Static hate speech benchmarks rank models differently from time-sensitive evaluations, with correlation coefficients near zero or negative, so high benchmark scores do not guarantee robustness to language change.
-
LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges
MalwareBench adds 11 jailbreak methods to 320 malicious code requests and shows mainstream LLMs reject only about half of the combined prompts.
-
Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation
A multi-agent adversarial evaluation platform with six progressive attack strategies shows that role-playing LLMs degrade under sustained pressure, with automated judging correlating with human ratings.
-
ToxSearch: Evolving Prompts for Toxicity Search in Large Language Models
Prompts evolved with small lexical mutations elicit toxic output from aligned LLMs, and these prompts transfer across models with toxicity reduced by roughly 30–54%.
-
SATORI: Static Test Oracle Generation for REST APIs
SATORI statically infers REST API test oracles from OpenAPI specs via LLMs, reporting F1 74.3%, above AGORA+'s 69.3%, with 18 confirmed bugs; the supplied full text, however, is a different paper.
-
Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks
CoopGuard's defer-tempt-analyze-coordinate agents cut reported jailbreak success and raise attacker token costs on the new EMRA benchmark, but the deceptive-rate metric is partly defined by the paper's own scoring rubric.
Discussion (0). Sign in to comment.