BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

11 Mar 2019 | Tianyu Gu, Brendan Dolan-Gavitt, Siddharth Garg
BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain Deep learning has achieved state-of-the-art performance in various tasks, but training is computationally expensive, leading to outsourcing or using pre-trained models. This paper shows that outsourcing training introduces security risks: an adversary can create a backdoored neural network (BadNet) that performs well on training data but misclassifies inputs with a specific trigger. The authors demonstrate this with a handwritten digit classifier and a U.S. traffic sign detector that misclassifies stop signs with a sticker. The backdoor persists even after retraining for a new task, causing a 25% accuracy drop. BadNets are stealthy and difficult to detect, highlighting the need for verification and inspection techniques. The paper also explores transfer learning, showing that backdoors can survive retraining for new tasks. The authors analyze the Caffe Model Zoo, identifying vulnerabilities in the model supply chain and recommending security measures. They conclude that pre-trained models must be sourced from trusted providers with integrity guarantees, and that the software supply chain security practices should be applied to machine learning.BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain Deep learning has achieved state-of-the-art performance in various tasks, but training is computationally expensive, leading to outsourcing or using pre-trained models. This paper shows that outsourcing training introduces security risks: an adversary can create a backdoored neural network (BadNet) that performs well on training data but misclassifies inputs with a specific trigger. The authors demonstrate this with a handwritten digit classifier and a U.S. traffic sign detector that misclassifies stop signs with a sticker. The backdoor persists even after retraining for a new task, causing a 25% accuracy drop. BadNets are stealthy and difficult to detect, highlighting the need for verification and inspection techniques. The paper also explores transfer learning, showing that backdoors can survive retraining for new tasks. The authors analyze the Caffe Model Zoo, identifying vulnerabilities in the model supply chain and recommending security measures. They conclude that pre-trained models must be sourced from trusted providers with integrity guarantees, and that the software supply chain security practices should be applied to machine learning.
Reach us at info@study.space