Pith. sign in

REVIEW 10 cited by

Fast is better than free: Revisiting adversarial training

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2001.03994 v1 pith:OAEHB3YQ submitted 2020-01-12 cs.LG stat.ML

Eric Wong , Leslie Rice , J. Zico Kolter This is my paper · ORCID
classification cs.LGstat.ML
keywords trainingadversarialrobustmethodfastfgsmaccuracyclassifier
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD). In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice. Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost. Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy to PGD attacks with $\epsilon=8/255$ in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at $\epsilon=2/255$ in 12 hours, in comparison to past work based on "free" adversarial training which took 10 and 50 hours to reach the same respective thresholds. Finally, we identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail. All code for reproducing the experiments in this paper as well as pretrained model weights are at https://github.com/locuslab/fast_adversarial.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A greedy, query-budgeted attack-ensemble framework approximates the minimum-perturbation frontier and yields an ε-free defense ranking (DOI).

  2. Adversarial Training Improves Generalization Under Distribution Shifts in Bioacoustics

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Output-space adversarial training improved clean-data performance and adversarial robustness of two bird sound classifiers across seven soundscape test sets, and stabilized prototype-based explanations.

  3. Exploring Visual Prompting: Robustness Inheritance and Beyond

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Visual prompts built on robust source models inherit adversarial robustness but lose standard accuracy; a max-pooling over logit blocks (PBL) improves accuracy while keeping most robustness.

  4. Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Combining suffix-window representation finetuning with an ActGrad-pruned surrogate cuts latent-adversarial-training FLOPs per step by 48.1% with only 0.0118% trainable parameters, while accepting higher attack success rates.

  5. Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers

    cs.CV 2026-07 conditional novelty 5.0 of 10

    FDT adds foveation and binary fixation modules to DeiT so multi-scale tokens are selected dynamically in one pass, improving ImageNet100 accuracy, MACs, and robustness without adversarial training.

  6. Robust Bidirectional Associative Memory via Regularization Inspired by the Subspace Rotation Algorithm

    cs.LG 2025-11 conditional novelty 5.0 of 10

    B-SRA, a gradient-free orthogonal-rotation trainer, plus OWM/GPA regularizers, makes BAM markedly more robust to masking, noise, and FGSM/FFGSM/BIM/PGD attacks than Bidirectional Backpropagation.

  7. Diffusion-based Cumulative Adversarial Purification for Vision Language Models

    cs.CV 2025-06 conditional novelty 5.0 of 10

    DiffCAP purifies adversarial images for vision-language models by injecting cumulative Gaussian noise until embeddings stabilize, then denoising, and outperforms prior defenses on captioning, VQA, and classification b...

  8. SemSegBench & DetecBench: Benchmarking Reliability and Generalization Beyond Classification

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A large-scale benchmark of 76 segmentation and 61 detection models shows that robustness to attacks and corruptions does not reliably track clean accuracy, and that transformer backbones generalize better under shift.

  9. Enhancing Adversarial Transferability through Block Stretch and Shrink

    cs.LG 2025-11 reject novelty 4.0 of 10

    A block stretch-and-shrink input transformation improves black-box adversarial transferability in experiments on 1000 ImageNet images, but the submitted manuscript contains missing figures and an abstract describing a...

  10. Assessing the Resilience of Automotive Intrusion Detection Systems to Adversarial Manipulation

    cs.CR 2025-06 conditional novelty 4.0 of 10

    Gradient-based evasion attacks can lower the detection rate of CAN-bus intrusion detection systems, with effectiveness depending on attacker knowledge, dataset, and detector architecture.

Pith tools