Overview

The term data mining evokes an image of the old-time panner for gold sifting through mounds of dirt trying to find those elusive valuable nuggets that make the whole process worthwhile. The translation into the information world is the data analyst sifting through terabytes of data looking for the corresponding knowledge nugget. This image is so powerful that the original meaning of data mining has been lost in the media hype that surrounds information exploitation. Now any information worker with a query tool connected to a database running ad hoc queries is called a "data miner," and the concepts that I discuss in this chapter are lost in that translation.

As an alternative to proactive business intelligence (BI) operations, the knowledge discovery process is a means for finding new intelligence from collections of data. Although the methods discussed in this chapter have traditionally been referred to as data mining, that term has become overloaded and so we will use the more correct term, knowledge discovery. Knowledge discovery refers to the process of discovering patterns that lead to actionable knowledge from large data sets through one or more traditional data mining techniques, such as market basket analysis and clustering. A lot of the knowledge discovery methodology has evolved from the combination of the worlds of statistics and computer science.

In this chapter, a lot of which is adapted from the very fine writings of Michael Berry and Gordon Linoff, we will look at the business use of...