Overview

Data mining has two components: Discovery, during which meaningful patterns are detected in data and characterized formally, and exploitation, during which meaningful patterns are used to create useful applications (models). In this chapter, and the next, discovery of knowledge from data is addressed. Exploitation is the focus of Chapter 10.

The process of discovering knowledge in data is referred to as Knowledge Discovery (KD). When the data is being worked from a conventional data store (as opposed to a data warehouse, online transaction system, etc.), the acronym KDD is sometimes used (Knowledge Discovery in Databases).

Knowledge discovery can be performed manually (by a human) or automatically (by a machine). This treatment of knowledge discovery will address, at a conceptual level, the elements of both. In-depth treatments of knowledge discovery can be quite technical. One of the greatest intellectual achievements of the age (Godel Incompleteness Theorem, 1931) was made during a formal study of "knowledge." Interested readers are encouraged to consult the literature.

8.1 Knowledge Is Connections

It is difficult to give a universally applicable definition of the term "knowledge." The notion used here is that knowledge is a sequential connection between facts. For example, stock brokers "know" that when the Federal Reserve Board (Fed) lowers interest rates, the stock market goes up. This "knowledge" is a connection between a prior fact ("Fed lowers rates") and a subsequent fact ("market goesup"). The prior fact is called the antecedent, and the subsequent fact, the