A Signal Analysis of Network Traffic Anomalies

A Signal Analysis of Network Traffic Anomalies

Nov. 6-8, 2002 | Paul Barford, Jeffery Kline, David Plonka and Amos Ron
This paper presents a signal analysis of four types of network traffic anomalies: outages, flash crowds, attacks, and measurement failures. The study uses IP flow and SNMP data collected over six months at the border router of a large university. Wavelet filters are shown to be effective in exposing details of both ambient and anomalous traffic. A pseudo-spline filter tuned at specific aggregation levels reveals distinct characteristics of each anomaly type. An effective way to detect anomalies is through the detection of sharp increases in the local variance of filtered data. The study evaluates traffic anomalies at different points in the network based on topological distance from the source or destination. Anomalies can be exposed even when aggregated with large amounts of additional traffic. The paper compares traffic anomaly signals seen in SNMP and IP flow data, showing that SNMP data can also effectively expose anomalies. The paper outlines a framework for characterizing network traffic, focusing on time-frequency characteristics of IP flow and SNMP data collected at the University of Wisconsin-Madison over six months. The study includes a catalog of 109 distinct traffic anomalies identified by the campus network engineering group. The Integrated Measurement Analysis Platform for Internet Traffic (IMAPIT) is developed to facilitate this work, containing a data management system and a robust signal analysis utility. Wavelet analysis is used to decompose network traffic data into frequency components. The study shows that wavelet analysis can effectively expose anomalies by detecting sharp increases in local variance. The paper also discusses the use of deviation scoring to automate anomaly detection, which is a first step in attempting to automate the process of identifying anomalies through multi-resolution techniques. The study finds that short-lived anomalies are more difficult to expose in data due to their similarity to normal bursty network behavior. However, combining data from mid and high frequency levels can effectively expose these anomalies. The paper also discusses the importance of isolating anomalies at different points in the network, showing that methods work well whether the measurement point is close to or distant from the anomaly. The paper is organized into sections describing the data sets, signal analysis methods, results, performance evaluation, and future work. The study compares the effectiveness of different anomaly detection methods, including deviation scoring and Holt-Winters forecasting. The results show that both methods perform well in detecting anomalies with low false-negative rates. The paper concludes that wavelet analysis and deviation scoring are effective tools for identifying network traffic anomalies.This paper presents a signal analysis of four types of network traffic anomalies: outages, flash crowds, attacks, and measurement failures. The study uses IP flow and SNMP data collected over six months at the border router of a large university. Wavelet filters are shown to be effective in exposing details of both ambient and anomalous traffic. A pseudo-spline filter tuned at specific aggregation levels reveals distinct characteristics of each anomaly type. An effective way to detect anomalies is through the detection of sharp increases in the local variance of filtered data. The study evaluates traffic anomalies at different points in the network based on topological distance from the source or destination. Anomalies can be exposed even when aggregated with large amounts of additional traffic. The paper compares traffic anomaly signals seen in SNMP and IP flow data, showing that SNMP data can also effectively expose anomalies. The paper outlines a framework for characterizing network traffic, focusing on time-frequency characteristics of IP flow and SNMP data collected at the University of Wisconsin-Madison over six months. The study includes a catalog of 109 distinct traffic anomalies identified by the campus network engineering group. The Integrated Measurement Analysis Platform for Internet Traffic (IMAPIT) is developed to facilitate this work, containing a data management system and a robust signal analysis utility. Wavelet analysis is used to decompose network traffic data into frequency components. The study shows that wavelet analysis can effectively expose anomalies by detecting sharp increases in local variance. The paper also discusses the use of deviation scoring to automate anomaly detection, which is a first step in attempting to automate the process of identifying anomalies through multi-resolution techniques. The study finds that short-lived anomalies are more difficult to expose in data due to their similarity to normal bursty network behavior. However, combining data from mid and high frequency levels can effectively expose these anomalies. The paper also discusses the importance of isolating anomalies at different points in the network, showing that methods work well whether the measurement point is close to or distant from the anomaly. The paper is organized into sections describing the data sets, signal analysis methods, results, performance evaluation, and future work. The study compares the effectiveness of different anomaly detection methods, including deviation scoring and Holt-Winters forecasting. The results show that both methods perform well in detecting anomalies with low false-negative rates. The paper concludes that wavelet analysis and deviation scoring are effective tools for identifying network traffic anomalies.
Reach us at info@study.space