How To Backdoor Federated Learning

How To Backdoor Federated Learning

6 Aug 2019 | Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, Vitaly Shmatikov
Federated learning enables thousands of participants to collaboratively train deep learning models without sharing their private training data. However, this paper introduces a new class of attacks called *model poisoning*, which is more powerful than traditional data poisoning attacks. A malicious participant can use *model replacement* to introduce backdoor functionality into the joint model, such as modifying an image classifier to assign attacker-chosen labels to images with certain features or forcing a word predictor to complete sentences with attacker-chosen words. These attacks can be performed by a single participant or multiple colluding participants. The paper evaluates model replacement under different assumptions for standard federated-learning tasks and shows that it outperforms data poisoning attacks. Federated learning employs secure aggregation to protect the confidentiality of participants' local models, making it difficult to detect anomalies in participants' contributions. To demonstrate the effectiveness of model replacement, the paper develops and evaluates a generic *constrain-and-scale* technique that incorporates evasion of defenses into the attacker's loss function during training. This technique can evade sophisticated anomaly detectors that measure cosine similarity between submitted models and the joint model. Another simpler technique, *train-and-scale*, is also developed to evade anomaly detectors that consider the model's weights or accuracy on the main task. The paper provides a detailed analysis of the attack, including its effectiveness in different scenarios and the impact on model accuracy. It also discusses the limitations of existing defenses and the challenges posed by the non-i.i.d. nature of training data in federated learning. The experiments on image classification and word prediction tasks show that the proposed attacks are highly effective, even with a small fraction of malicious participants.Federated learning enables thousands of participants to collaboratively train deep learning models without sharing their private training data. However, this paper introduces a new class of attacks called *model poisoning*, which is more powerful than traditional data poisoning attacks. A malicious participant can use *model replacement* to introduce backdoor functionality into the joint model, such as modifying an image classifier to assign attacker-chosen labels to images with certain features or forcing a word predictor to complete sentences with attacker-chosen words. These attacks can be performed by a single participant or multiple colluding participants. The paper evaluates model replacement under different assumptions for standard federated-learning tasks and shows that it outperforms data poisoning attacks. Federated learning employs secure aggregation to protect the confidentiality of participants' local models, making it difficult to detect anomalies in participants' contributions. To demonstrate the effectiveness of model replacement, the paper develops and evaluates a generic *constrain-and-scale* technique that incorporates evasion of defenses into the attacker's loss function during training. This technique can evade sophisticated anomaly detectors that measure cosine similarity between submitted models and the joint model. Another simpler technique, *train-and-scale*, is also developed to evade anomaly detectors that consider the model's weights or accuracy on the main task. The paper provides a detailed analysis of the attack, including its effectiveness in different scenarios and the impact on model accuracy. It also discusses the limitations of existing defenses and the challenges posed by the non-i.i.d. nature of training data in federated learning. The experiments on image classification and word prediction tasks show that the proposed attacks are highly effective, even with a small fraction of malicious participants.
Reach us at info@study.space
[slides and audio] How To Backdoor Federated Learning