Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models

Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models

20 Jul 2024 | Yuchen Yang1*, Kwonjoon Lee2, Behzad Dariush2, Yinzhi Cao1, and Shao-Yuan Lo2
Video Anomaly Detection (VAD) is crucial for applications such as security surveillance and autonomous driving, but existing methods often lack rationale behind their detection, hindering public trust. This paper introduces AnomalyRuler, a novel rule-based reasoning framework for VAD using Large Language Models (LLMs). AnomalyRuler consists of two main stages: induction and deduction. In the induction stage, LLMs are fed with few-shot normal reference samples and summarize these patterns to induce rules for detecting anomalies. In the deduction stage, these rules are applied to identify anomalous frames in test videos. To enhance robustness, the framework includes rule aggregation, perception smoothing, and robust reasoning strategies. AnomalyRuler is the first reasoning approach for one-class VAD, requiring only a few normal samples and enabling fast adaptation to various VAD scenarios. Extensive experiments on four VAD benchmarks demonstrate AnomalyRuler's state-of-the-art performance, reasoning ability, and domain adaptability.Video Anomaly Detection (VAD) is crucial for applications such as security surveillance and autonomous driving, but existing methods often lack rationale behind their detection, hindering public trust. This paper introduces AnomalyRuler, a novel rule-based reasoning framework for VAD using Large Language Models (LLMs). AnomalyRuler consists of two main stages: induction and deduction. In the induction stage, LLMs are fed with few-shot normal reference samples and summarize these patterns to induce rules for detecting anomalies. In the deduction stage, these rules are applied to identify anomalous frames in test videos. To enhance robustness, the framework includes rule aggregation, perception smoothing, and robust reasoning strategies. AnomalyRuler is the first reasoning approach for one-class VAD, requiring only a few normal samples and enabling fast adaptation to various VAD scenarios. Extensive experiments on four VAD benchmarks demonstrate AnomalyRuler's state-of-the-art performance, reasoning ability, and domain adaptability.
Reach us at info@study.space
Understanding Follow the Rules%3A Reasoning for Video Anomaly Detection with Large Language Models