This paper introduces the first evaluation of copyright takedown methods for language models (LMs), which are trained on extensive data, including potentially copyrighted material. The authors propose COTAeval, a framework to assess the effectiveness of these methods in preventing the generation of protected content while preserving uncopyrightable factual knowledge and maintaining model utility and efficiency. They examine several strategies, including system prompts, decoding-time filtering interventions, and unlearning approaches. The findings indicate that no single method excels across all metrics, highlighting the need for further research to address unresolved challenges in this area. The paper also discusses the legal and ethical implications of copyright takedowns and suggests that they should be part of a broader strategy to manage complex legal scenarios.This paper introduces the first evaluation of copyright takedown methods for language models (LMs), which are trained on extensive data, including potentially copyrighted material. The authors propose COTAeval, a framework to assess the effectiveness of these methods in preventing the generation of protected content while preserving uncopyrightable factual knowledge and maintaining model utility and efficiency. They examine several strategies, including system prompts, decoding-time filtering interventions, and unlearning approaches. The findings indicate that no single method excels across all metrics, highlighting the need for further research to address unresolved challenges in this area. The paper also discusses the legal and ethical implications of copyright takedowns and suggests that they should be part of a broader strategy to manage complex legal scenarios.