16 Apr 2024 | Zhiyu Hu, Yang Zhang, Minghao Xiao, Wenjie Wang, Fuli Feng, Xiangnan He
The paper introduces the Adapter Partition and Aggregation (APA) framework for exact and efficient unlearning in Large Language Model-based Recommendation (LLMRec). LLMRec customizes LLMs using parameter-efficient fine-tuning (PEFT) with recommendation data, but this raises privacy concerns. The APA framework addresses the challenge of unlearning by partitioning training data into shards and retraining only the adapters affected by unusable data. It employs parameter-level adapter aggregation with sample-adaptive attention to maintain recommendation performance and reduce inference costs. Extensive experiments on real-world datasets demonstrate the effectiveness and efficiency of the APA framework, showing improved recommendation performance and unlearning efficiency compared to existing methods. The key contributions include a novel problem formulation, the APA technique, and extensive experimental validation.The paper introduces the Adapter Partition and Aggregation (APA) framework for exact and efficient unlearning in Large Language Model-based Recommendation (LLMRec). LLMRec customizes LLMs using parameter-efficient fine-tuning (PEFT) with recommendation data, but this raises privacy concerns. The APA framework addresses the challenge of unlearning by partitioning training data into shards and retraining only the adapters affected by unusable data. It employs parameter-level adapter aggregation with sample-adaptive attention to maintain recommendation performance and reduce inference costs. Extensive experiments on real-world datasets demonstrate the effectiveness and efficiency of the APA framework, showing improved recommendation performance and unlearning efficiency compared to existing methods. The key contributions include a novel problem formulation, the APA technique, and extensive experimental validation.