Toward Comprehensive Traffic Generation for Online IDS Evaluation

Abstract

We describe a traffic generation framework for conducting online evaluations of network intrusion detection systems over a wide range of realistic conditions. The framework integrates both benign and malicious traffic, enabling generation of IP packet streams with diverse characteristics from the perspective of {\em (i) packet content} (both header and payload), {\em (ii) packet mix} (order of packets in streams) and {\em (iii) packet volume} (arrival rate of packets in streams). We begin by describing a methodology for defining trust which forms the basis of our method for systematic extraction of ``benign'' traffic from live streams. We then detail how we combine these traces with application-specific automata to generate benign traffic streams. Next, we describe a methodology for malicious traffic generation, and techniques for integration with benign traffic to produce a range of realistic workload compositions. We realize our traffic generation framework in a tool we call Trident, and demonstrate its utility through a series of laboratory-based experiments using traces collected from our departmental border router, DARPA Intrusion Detection Evaluation data sets provided by Lincoln Lab, and a suite of malicious traffic modules that reproduce a broad range of attacks commonly seen in today's networks. Our experiments demonstrate the effects of varying packet content, mix, and volume on the performance of intrusion detection systems.