Chapter 5: Sampling, Variability, and Confidence

Chapter 5: Sampling, Variability, and Confidence

Sampling, or First Catch Your Hare!

Mrs. Beaton?s famous English cookbook is alleged to have contained a recipe for Jugged Hare that started, ?First. Catch your hare.? It is too good a line to pass up, true or not. If you want the dish, catching the hare is the place to start. If you want to mine data, catching the ?hare? in the data is the place to start. So what is the ?hare? in data? The hare is the information content enfolded into the data set. Just as hare is the essence of the recipe for Jugged Hare, so information is the essence of the recipe for building training and test data sets.

Clearly, what is needed is enough data so that all of the relationships at all levels?superstructure, macrostructure, and microstructure?are captured. An easy answer would seem to be to use all the data. After all, with all of the data being used, it is a sure thing that any relationship of interest that the data contains is there to be found. Unfortunately, there are problems with the idea of using all of the data.

5.1.1 How Much Data?

One problem with trying to use all of the data, perhaps the most common problem, is simply that all of the data is not available. It is usual to call the whole of data the population . Strictly speaking, the data is not the population; the data is simply a set of...

< Previous Excerpt Next Excerpt >

Purchase This Book

Data Preparation for Data Mining

TABLE OF CONTENTS

Chapter 5: Sampling, Variability, and Confidence

Contact Preferences

This is embarrasing...

Customize Your GlobalSpec Experience

Select Your Free Newsletters

Industry Newsletters

Select Your Free Product Alerts

This is embarrasing...