Overview

As one would expect, data is a critical element in data mining success, a veritable "pillar" if you will. Although organizations have been capturing data for decades, it has primarily been to support mundane record-keeping activities. Advances in computer storage technology have made it possible to save virtually every scrap of data generated by large enterprises. Nevertheless, valuable "nuggets" of information are often contained within vast computer archives.

The fundamental motivation for data mining is the goal of discovering and exploiting such valuable patterns in data. Apparent patterns may be regarded as manifest indications, but not proof of the presence of actionable information latent in the data. Similarly, the apparent absence of patterns can only sometimes be reasonably equated with the absence of actionable information.

It is also the case that poor data preparation or representation and improperly used tools can make existing patterns hard to detect, and make random noise look patterned. Unaided human intuition can be a poor guide in data mining. For these reasons this chapter discusses several important considerations regarding data: meta data, data representation (quantization and coding), feature extraction and enhancement, data quality, relevance, independence, data preparation, and feature selection.

Data miners can expect to be confronted with larger and larger repositories of data to mine. They must realize that actionable patterns are not the result of data volume, but of information content. Patterns don't appear because a megabyte, gigabyte, or terabyte of data have been amassed. Patterns appear because information is present. By...