Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models

Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models

20 Jul 2024 | Yucheng Yang*, Kwonjoon Lee, Behzad Dariush, Yinzi Cao, and Shao-Yuan Lo
**Summary:** This paper introduces AnomalyRuler, a novel rule-based reasoning framework for Video Anomaly Detection (VAD) using Large Language Models (LLMs). The framework consists of two stages: induction and deduction. In the induction stage, LLMs are trained on few-shot normal reference samples to derive rules for detecting anomalies. In the deduction stage, these rules are applied to test videos to identify anomalous frames. AnomalyRuler is designed to be robust and efficient, requiring only few-normal-shot prompting without full-shot training, enabling fast adaptation to various VAD scenarios. The framework incorporates strategies such as rule aggregation, perception smoothing, and robust reasoning to enhance performance. Comprehensive experiments across four VAD benchmarks demonstrate AnomalyRuler's state-of-the-art detection performance and reasoning ability. AnomalyRuler is open-source and available at https://github.com/Yuchen413/AnomalyRuler. The paper highlights the importance of reasoning in VAD to build trustworthy systems and addresses the limitations of direct LLM use in VAD tasks. AnomalyRuler is the first reasoning approach for one-class VAD, offering strong domain adaptability and effective anomaly detection with detailed reasoning.**Summary:** This paper introduces AnomalyRuler, a novel rule-based reasoning framework for Video Anomaly Detection (VAD) using Large Language Models (LLMs). The framework consists of two stages: induction and deduction. In the induction stage, LLMs are trained on few-shot normal reference samples to derive rules for detecting anomalies. In the deduction stage, these rules are applied to test videos to identify anomalous frames. AnomalyRuler is designed to be robust and efficient, requiring only few-normal-shot prompting without full-shot training, enabling fast adaptation to various VAD scenarios. The framework incorporates strategies such as rule aggregation, perception smoothing, and robust reasoning to enhance performance. Comprehensive experiments across four VAD benchmarks demonstrate AnomalyRuler's state-of-the-art detection performance and reasoning ability. AnomalyRuler is open-source and available at https://github.com/Yuchen413/AnomalyRuler. The paper highlights the importance of reasoning in VAD to build trustworthy systems and addresses the limitations of direct LLM use in VAD tasks. AnomalyRuler is the first reasoning approach for one-class VAD, offering strong domain adaptability and effective anomaly detection with detailed reasoning.
Reach us at info@study.space
Understanding Follow the Rules%3A Reasoning for Video Anomaly Detection with Large Language Models