Unlabeled Data Improves Adversarial Robustness

Code

• github.com/yaircarmon/semisup-adv
  OfficialIn paperpytorch★ 0
• worksheets.codalab.org/worksheets/0x9df253b24dac4a2b930108be9c6e5496
  Officialnone★ 0
• github.com/MarinePICOT/Adversarial-Robustness-via-Fisher-Rao-Regularization
  pytorch★ 5
• github.com/yguooo/semisup-adv
  pytorch★ 0

Abstract

We demonstrate, theoretically and empirically, that adversarial robustness can significantly benefit from semisupervised learning. Theoretically, we revisit the simple Gaussian model of Schmidt et al. that shows a sample complexity gap between standard and robust classification. We prove that unlabeled data bridges this gap: a simple semisupervised learning procedure (self-training) achieves high robust accuracy using the same number of labels required for achieving high standard accuracy. Empirically, we augment CIFAR-10 with 500K unlabeled images sourced from 80 Million Tiny Images and use robust self-training to outperform state-of-the-art robust accuracies by over 5 points in (i) _ robustness against several strong attacks via adversarial training and (ii) certified _2 and _ robustness via randomized smoothing. On SVHN, adding the dataset's own extra training set with the labels removed provides gains of 4 to 10 points, within 1 point of the gain from using the extra labels.

Tasks

Reproductions