TABLE OF CONTENTS System and network logs are often called the untapped riches The now-famous humorous security calendar proclaims Logs: Let em Rot. Others just quietly choose to follow this maxim and ignore logs at their own peril altogether.
On the other hand, as computer and Internet technology continues to spread and computers start playing an even more important role in our lives, the records that they produce, a.k.a. logs, start to play a bigger role. From firewalls and x to databases and enterprise applications, to wireless access points and Voice over Internet Protocol (VoIP) gateways, logs are being spewed forth at an ever-increasing pace. Both security and other Information Technology (IT) components not only increase in numbers, but also often come with more logging enabled out of the box. An example of this trend includes Linux systems and Web servers that now ship with increased levels of logging. All those systems, both legacy and modern, are known to generate copious amounts of logs, audit trails, records, and alerts, that beg for constant attention.
But this is easier said than done. Immense volumes of log data are being generated on payment card processing networks, necessitating more efficient ways of managing, storing, and searching through log data, both reactively after a suspected incident and proactively in search of potential risks. For example, a typical retailer generates hundreds of thousands of log messages per day amounting to many terabytes per year. An online merchant can generate upwards of 500,000 log messages every day. One of America s largest retailers...
