WiKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing

WiKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing

5 Jun 2024 | Chenhui Hu, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao
WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing This paper introduces WilKE, a knowledge editing method that addresses the issue of performance degradation in lifelong knowledge editing. Current knowledge editing methods primarily focus on single editing, which is insufficient for lifelong editing scenarios. The study reveals that during lifelong editing, knowledge editing methods suffer from toxicity buildup and toxicity flash, which are caused by pattern unmatch. WilKE selects the editing layer based on the degree of pattern matching for different editing knowledge across various layers in language models. Experimental results show that WilKE achieves significant improvements in performance compared to state-of-the-art knowledge editing methods. Specifically, in lifelong editing scenarios, WilKE demonstrates an average improvement of 46.2% and 67.8% on editing GPT2-XL and GPT-J, respectively. The method does not require predefined editing layers and instead selects the most suitable layer for each editing task based on pattern matching. The results indicate that WilKE significantly enhances overall performance in lifelong editing compared to existing methods. The study highlights the importance of addressing pattern unmatch in lifelong knowledge editing and provides a novel approach to improve knowledge editing methods.WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing This paper introduces WilKE, a knowledge editing method that addresses the issue of performance degradation in lifelong knowledge editing. Current knowledge editing methods primarily focus on single editing, which is insufficient for lifelong editing scenarios. The study reveals that during lifelong editing, knowledge editing methods suffer from toxicity buildup and toxicity flash, which are caused by pattern unmatch. WilKE selects the editing layer based on the degree of pattern matching for different editing knowledge across various layers in language models. Experimental results show that WilKE achieves significant improvements in performance compared to state-of-the-art knowledge editing methods. Specifically, in lifelong editing scenarios, WilKE demonstrates an average improvement of 46.2% and 67.8% on editing GPT2-XL and GPT-J, respectively. The method does not require predefined editing layers and instead selects the most suitable layer for each editing task based on pattern matching. The results indicate that WilKE significantly enhances overall performance in lifelong editing compared to existing methods. The study highlights the importance of addressing pattern unmatch in lifelong knowledge editing and provides a novel approach to improve knowledge editing methods.
Reach us at info@study.space
[slides and audio] WilKE%3A Wise-Layer Knowledge Editor for Lifelong Knowledge Editing