Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection

Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection

26 Jan 2024 | Chen Liu, Shibo He, Qihang Zhou, Shizhong Li, Wenchao Meng
The paper introduces AnomalyLLM, a novel knowledge distillation-based approach for time series anomaly detection (TSAD) that leverages large language models (LLMs). TSAD aims to identify abnormal data patterns, but traditional methods often require extensive training data, which is limited in real-world scenarios. AnomalyLLM addresses this issue by training a student network to mimic the output of a teacher network pre-trained on large-scale datasets. During testing, anomalies are detected based on the discrepancy between the student and teacher networks' outputs. To prevent the student network from overlearning the teacher's representations, the method incorporates prototypical signals and uses synthetic anomalies to enhance the representation gap. Extensive experiments on 15 datasets demonstrate that AnomalyLLM achieves state-of-the-art performance, improving accuracy by at least 14.5% on the UCR dataset. The key contributions include the first application of knowledge distillation in TSAD, the adaptation of LLMs for time series representation, and the integration of prototypical signals and data augmentation to maintain the discrepancy between the teacher and student networks.The paper introduces AnomalyLLM, a novel knowledge distillation-based approach for time series anomaly detection (TSAD) that leverages large language models (LLMs). TSAD aims to identify abnormal data patterns, but traditional methods often require extensive training data, which is limited in real-world scenarios. AnomalyLLM addresses this issue by training a student network to mimic the output of a teacher network pre-trained on large-scale datasets. During testing, anomalies are detected based on the discrepancy between the student and teacher networks' outputs. To prevent the student network from overlearning the teacher's representations, the method incorporates prototypical signals and uses synthetic anomalies to enhance the representation gap. Extensive experiments on 15 datasets demonstrate that AnomalyLLM achieves state-of-the-art performance, improving accuracy by at least 14.5% on the UCR dataset. The key contributions include the first application of knowledge distillation in TSAD, the adaptation of LLMs for time series representation, and the integration of prototypical signals and data augmentation to maintain the discrepancy between the teacher and student networks.
Reach us at info@study.space
[slides] Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection | StudySpace