The paper "PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps" by Ruixuan Liu, Tianhao Wang, Yang Cao, and Li Xiong explores the privacy risks associated with pre-trained language models. The authors propose the PreCurious framework, which aims to increase the privacy risk of membership inference and data extraction attacks by manipulating the pre-trained model's memorization stage and guiding fine-tuning with seemingly legitimate configurations. The key intuition behind PreCurious is to control the memorization level of the pre-trained model and exploit side information, such as stopping criteria and fine-tuning methods, to guide fine-tuning victims. The effectiveness of PreCurious is demonstrated through experiments on various datasets and fine-tuning techniques, showing that it can significantly amplify privacy risks compared to fine-tuning on benign models. The paper also discusses the stealthiness of PreCurious and its ability to break the privacy-utility trade-off, highlighting the importance of auditing and defense strategies.The paper "PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps" by Ruixuan Liu, Tianhao Wang, Yang Cao, and Li Xiong explores the privacy risks associated with pre-trained language models. The authors propose the PreCurious framework, which aims to increase the privacy risk of membership inference and data extraction attacks by manipulating the pre-trained model's memorization stage and guiding fine-tuning with seemingly legitimate configurations. The key intuition behind PreCurious is to control the memorization level of the pre-trained model and exploit side information, such as stopping criteria and fine-tuning methods, to guide fine-tuning victims. The effectiveness of PreCurious is demonstrated through experiments on various datasets and fine-tuning techniques, showing that it can significantly amplify privacy risks compared to fine-tuning on benign models. The paper also discusses the stealthiness of PreCurious and its ability to break the privacy-utility trade-off, highlighting the importance of auditing and defense strategies.