This paper presents PAYL, a payload-based anomaly detection system for network intrusion detection. PAYL automatically models normal application payload behavior in a fully unsupervised and efficient manner. During training, it computes byte frequency distributions and their standard deviations for normal traffic to a specific host and port. During detection, it uses Mahalanobis distance to compare new data against the precomputed profile. If the distance exceeds a threshold, an alert is generated. The method is tested on the 1999 DARPA IDS dataset and a live dataset from Columbia University, achieving nearly 100% accuracy with a 0.1% false positive rate for port 80 traffic.
Many existing intrusion detection systems (IDS) are signature-based, effective against known attacks but not new ones. Anomaly detection systems, on the other hand, model normal behavior and detect deviations that may indicate security breaches. Some attacks exploit protocol vulnerabilities or scan networks, detectable through packet headers or traffic analysis. Others, like worms, deliver malicious payloads in normal connections, detectable by inspecting payloads or the effects of worm execution.
The proposed method detects anomalies in payloads expected for a network service or application. It first learns a model of normal payloads, then detects anomalies by comparing new payloads to this model using Mahalanobis distance. Anomalous payloads trigger alerts, which can be correlated with other data to take action, such as filtering or rerouting traffic.
The method is applicable to any network system, service, or port, allowing site-specific payload anomaly detection. It is not dependent on signatures for new worms or exploits. Additionally, it can detect encrypted channels, indicating unauthorized secure tunnels. The paper focuses on the detection method, not correlation or mitigation strategies. This approach enables early detection of worms or other malicious payloads, preventing their spread within a network.This paper presents PAYL, a payload-based anomaly detection system for network intrusion detection. PAYL automatically models normal application payload behavior in a fully unsupervised and efficient manner. During training, it computes byte frequency distributions and their standard deviations for normal traffic to a specific host and port. During detection, it uses Mahalanobis distance to compare new data against the precomputed profile. If the distance exceeds a threshold, an alert is generated. The method is tested on the 1999 DARPA IDS dataset and a live dataset from Columbia University, achieving nearly 100% accuracy with a 0.1% false positive rate for port 80 traffic.
Many existing intrusion detection systems (IDS) are signature-based, effective against known attacks but not new ones. Anomaly detection systems, on the other hand, model normal behavior and detect deviations that may indicate security breaches. Some attacks exploit protocol vulnerabilities or scan networks, detectable through packet headers or traffic analysis. Others, like worms, deliver malicious payloads in normal connections, detectable by inspecting payloads or the effects of worm execution.
The proposed method detects anomalies in payloads expected for a network service or application. It first learns a model of normal payloads, then detects anomalies by comparing new payloads to this model using Mahalanobis distance. Anomalous payloads trigger alerts, which can be correlated with other data to take action, such as filtering or rerouting traffic.
The method is applicable to any network system, service, or port, allowing site-specific payload anomaly detection. It is not dependent on signatures for new worms or exploits. Additionally, it can detect encrypted channels, indicating unauthorized secure tunnels. The paper focuses on the detection method, not correlation or mitigation strategies. This approach enables early detection of worms or other malicious payloads, preventing their spread within a network.