10 Mar 2024 | Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, Adams Wai-Kin Kong
The paper introduces MACE (Mass Concept Erasure), a framework designed to prevent text-to-image (T2I) diffusion models from generating images that embody unwanted concepts. MACE addresses the challenges of concept erasure by balancing generality and specificity, allowing for the erasure of up to 100 concepts simultaneously. The method leverages closed-form cross-attention refinement and LoRA (Low-Rank Adaptation) finetuning to eliminate the information of target concepts while maintaining the integrity of unrelated concepts. Extensive evaluations across four tasks—object erasure, celebrity erasure, explicit content erasure, and artistic style erasure—show that MACE outperforms existing methods in terms of efficacy, generality, and specificity. The paper also discusses the limitations and future directions for scaling up the erasure scope.The paper introduces MACE (Mass Concept Erasure), a framework designed to prevent text-to-image (T2I) diffusion models from generating images that embody unwanted concepts. MACE addresses the challenges of concept erasure by balancing generality and specificity, allowing for the erasure of up to 100 concepts simultaneously. The method leverages closed-form cross-attention refinement and LoRA (Low-Rank Adaptation) finetuning to eliminate the information of target concepts while maintaining the integrity of unrelated concepts. Extensive evaluations across four tasks—object erasure, celebrity erasure, explicit content erasure, and artistic style erasure—show that MACE outperforms existing methods in terms of efficacy, generality, and specificity. The paper also discusses the limitations and future directions for scaling up the erasure scope.