TABLE OF CONTENTS One of the most difficult and frustrating phases of data mining is getting access to the right data. In government there are always issues between agencies and agreements to be sorted out, not to mention formats that need to be reconciled, all of which require several meetings before arrangements can be made. In private industry, there are the issues of privacy and cost. These are some of the minor, but very real, obstacles that accompany most data mining projects. Of greater significance are the issues revolving around what data is required for the desired objective. However, in the aftermath of 9/11 a new sense of urgency has evolved, in the face of which these obstacles pale in comparison to failing to resolve these data integration issues.
The value of any data mining model is very much dependent on the quality of the data used to construct it; for this reason it is critical that some creative discussions be held and consideration be made about what data is available at the start of the project. Aside from the data that is internally available, thought should be given to what external data sources could provide valuable insight to the data mining analysis. In this chapter we will discuss the closed and open sources of data available both online and offline and how to integrate and prepare the data prior to its analysis.
Data mining is about predicting behavior or profiling individuals; as such, it is critical to have access...
