The paper "Evaluations of Machine Learning Privacy Defenses are Misleading" by Michael Aerni, Jie Zhang, and Florian Tramèr from ETH Zurich identifies significant flaws in existing empirical evaluations of machine learning privacy defenses. These evaluations, which often use membership inference attacks, can lead to misleading conclusions about the effectiveness of these defenses. The authors highlight three main issues: 1. **Aggregating Attack Success Over a Dataset**: Current evaluations aggregate attack success across all samples, failing to capture the privacy leakage of the most vulnerable samples. 2. **Weak or Non-Adaptive Attacks**: Many evaluations use weak attacks that do not fully exploit the capabilities of the defended model, or fail to adapt the attack to the specific defense mechanisms. 3. **Comparison to Weak DP Baselines**: Empirical defenses are often compared to weak differential privacy (DP) baselines, which do not provide meaningful provable guarantees and are not comparable to the high-utility defenses being evaluated. To address these issues, the authors propose a more reliable evaluation methodology: 1. **Focus on the Most Vulnerable Samples**: Evaluate membership inference success for the most vulnerable samples using a set of "canaries" that approximate the most vulnerable samples. 2. **Use State-of-the-Art Membership Inference Attacks**: Adapt the attack to the specifics of the defense to ensure it is strong and adaptive. 3. **Compare to Strong DP Baselines**: Tune DP-SGD to achieve high utility while maximizing empirical privacy, and compare the empirical defenses to these strong baselines. Through five case studies, the authors demonstrate that prior evaluations underestimate privacy leakage by an order of magnitude. None of the empirical defenses they study are competitive with a properly tuned, high-utility DP-SGD baseline. The paper aims to provide a more principled evaluation framework and highlights the need for reproducible research, releasing all code for their evaluation methodology and implementations.The paper "Evaluations of Machine Learning Privacy Defenses are Misleading" by Michael Aerni, Jie Zhang, and Florian Tramèr from ETH Zurich identifies significant flaws in existing empirical evaluations of machine learning privacy defenses. These evaluations, which often use membership inference attacks, can lead to misleading conclusions about the effectiveness of these defenses. The authors highlight three main issues: 1. **Aggregating Attack Success Over a Dataset**: Current evaluations aggregate attack success across all samples, failing to capture the privacy leakage of the most vulnerable samples. 2. **Weak or Non-Adaptive Attacks**: Many evaluations use weak attacks that do not fully exploit the capabilities of the defended model, or fail to adapt the attack to the specific defense mechanisms. 3. **Comparison to Weak DP Baselines**: Empirical defenses are often compared to weak differential privacy (DP) baselines, which do not provide meaningful provable guarantees and are not comparable to the high-utility defenses being evaluated. To address these issues, the authors propose a more reliable evaluation methodology: 1. **Focus on the Most Vulnerable Samples**: Evaluate membership inference success for the most vulnerable samples using a set of "canaries" that approximate the most vulnerable samples. 2. **Use State-of-the-Art Membership Inference Attacks**: Adapt the attack to the specifics of the defense to ensure it is strong and adaptive. 3. **Compare to Strong DP Baselines**: Tune DP-SGD to achieve high utility while maximizing empirical privacy, and compare the empirical defenses to these strong baselines. Through five case studies, the authors demonstrate that prior evaluations underestimate privacy leakage by an order of magnitude. None of the empirical defenses they study are competitive with a properly tuned, high-utility DP-SGD baseline. The paper aims to provide a more principled evaluation framework and highlights the need for reproducible research, releasing all code for their evaluation methodology and implementations.