Data analysis is the centralmost process in intrusion detection. You can collect Snort alerts all day long, but if you do not use them to analyze and understand what has happened on your network, they are pretty much useless. These alerts carry a lot of information which is invaluable in gaining an understanding of what is happening on your network, as well as detecting attacks and uncovering malicious behavior from internal or external sources.

Snort uses alerts to record its findings and communicate them to the user. An alert is a message which a detection mechanism (e.g., preprocessor or Snort rule) passes when it matches an event to a known pattern. Alerts will be of central interest in this chapter, and that is where we will start our journey through the maze of data analysis. But before we head that way, why are we interested in data analysis at all? What does data analysis try to accomplish? These are the four use cases driving data analysis:

• Real-time alerting

• Attack detection and verification

• Incident analysis

• Reporting

The first case, real-time alerting is somewhat different from the others. It is the only process which requires a real-time component to analyze the alert stream in order to escalate the events based on specific combinations. Once you start analyzing Snort alerts, you will realize that certain combinations of alerts express more com- plex situations, and you will need a way to catch those things in real time...