SOTAVerified

Theoretically Principled Trade-off between Robustness and Accuracy

2019-01-24Code Available1· sign in to hype

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, Michael. I. Jordan

arXivPDF

Code Available — Be the first to reproduce this paper.

Reproduce

Code

Abstract

We identify a trade-off between robustness and accuracy that serves as a guiding principle in the design of defenses against adversarial examples. Although this problem has been widely studied empirically, much remains unknown concerning the theory underlying this trade-off. In this work, we decompose the prediction error for adversarial examples (robust error) as the sum of the natural (classification) error and boundary error, and provide a differentiable upper bound using the theory of classification-calibrated loss, which is shown to be the tightest possible upper bound uniform over all probability distributions and measurable predictors. Inspired by our theoretical analysis, we also design a new defense method, TRADES, to trade adversarial robustness off against accuracy. Our proposed algorithm performs well experimentally in real-world datasets. The methodology is the foundation of our entry to the NeurIPS 2018 Adversarial Vision Challenge in which we won the 1st place out of ~2,000 submissions, surpassing the runner-up approach by 11.41\% in terms of mean _2 perturbation distance.

Tasks

Adversarial AttackAdversarial DefenseAdversarial RobustnessGeneral Classification

Benchmark Results

DatasetModelMetricClaimedVerifiedStatus
CIFAR-10TRADES [zhang2019b]Attack: PGD2045.9—Unverified

Reproductions