REVIEW 5 cited by
Scaling Trends in Language Model Robustness
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Increasing model size has unlocked a dazzling array of capabilities in modern language models. At the same time, even frontier models remain vulnerable to jailbreaks and prompt injections, despite concerted efforts to make them robust. As both attack and defense gain access to more compute, and as models become larger, what happens to robustness? We argue that to answer this question requires a \emph{scaling} approach, which we employ in an extensive study of language model robustness across several classification tasks, model families, and adversarial attacks. We find that in the absence of explicit safety training, larger models are not consistently more robust; however, scale improves sample efficiency in adversarial training, though it worsens compute efficiency. Further, we find that increasing attack compute smoothly improves attack success rate against both undefended and adversarially trained models. Finally, after exploring robustness transfer across attacks and threat models, we combine attack and defense scaling rates to study the offense-defense balance. We find that while attack scaling outpaces adversarial training across all models studied, larger adversarially trained models might give defense the advantage in the long run. These results underscore the utility of the scaling lens, and provide a paradigm for evaluating future attacks and defenses on frontier models.
Forward citations
Cited by 5 Pith papers
-
Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
Online self-play between attacker and defender roles of a single LLM improves safety robustness and attack diversity across Llama and Qwen models.
-
On the Scaling of Robustness and Effectiveness in Dense Retrieval
Robustness in dense retrieval follows power-law scaling laws, but with different exponents than effectiveness, and an adaptive weighting method (Pareto training) can improve the joint tradeoff more efficiently than sc...
-
Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates
Combining suffix-window representation finetuning with an ActGrad-pruned surrogate cuts latent-adversarial-training FLOPs per step by 48.1% with only 0.0118% trainable parameters, while accepting higher attack success rates.
-
Can You Trick the Grader? Adversarial Persuasion of LLM Judges
Strategically inserted persuasive sentences inflate LLM judges' scores for incorrect math solutions across six benchmarks and fourteen models, but the study lacks length-matched controls separating rhetoric from lengt...
-
We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems
MCP-powered LLM agents are vulnerable to prompt injection from third-party services, and simple detection or filtering defenses do not reliably stop these attacks.
Discussion (0). Sign in to comment.