BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

Code

• github.com/xqx12/daily-info
  pytorch★ 322
• github.com/ain-soph/trojanzoo
  pytorch★ 302
• github.com/trx14/TrojanNet
  tf★ 88
• github.com/DreamtaleCore/Refool
  pytorch★ 50
• github.com/aisafety-hkust/backdoor_safety_tuning
  pytorch★ 27
• github.com/aisafety-hkust/stable_backdoor_purification
  pytorch★ 27
• github.com/locuslab/breaking-poisoned-classifier
  pytorch★ 26
• github.com/xandery-geek/BackdoorAttacks
  pytorch★ 22
• github.com/sakshiudeshi/Neo
  none★ 0
• github.com/kooscii/badnets
  none★ 0

Abstract

Deep learning-based techniques have achieved state-of-the-art performance on a wide variety of recognition and classification tasks. However, these networks are typically computationally expensive to train, requiring weeks of computation on many GPUs; as a result, many users outsource the training procedure to the cloud or rely on pre-trained models that are then fine-tuned for a specific task. In this paper we show that outsourced training introduces new security risks: an adversary can create a maliciously trained network (a backdoored neural network, or a BadNet) that has state-of-the-art performance on the user's training and validation samples, but behaves badly on specific attacker-chosen inputs. We first explore the properties of BadNets in a toy example, by creating a backdoored handwritten digit classifier. Next, we demonstrate backdoors in a more realistic scenario by creating a U.S. street sign classifier that identifies stop signs as speed limits when a special sticker is added to the stop sign; we then show in addition that the backdoor in our US street sign detector can persist even if the network is later retrained for another task and cause a drop in accuracy of 25\% on average when the backdoor trigger is present. These results demonstrate that backdoors in neural networks are both powerful and---because the behavior of neural networks is difficult to explicate---stealthy. This work provides motivation for further research into techniques for verifying and inspecting neural networks, just as we have developed tools for verifying and debugging software.

Tasks

Reproductions