9.1 Early History

Data warehousing began in the medical community with epidemiology studies. Historical data was kept in standardized formats that allowed researchers to look for patterns in the data and take appropriate actions. Cost was not much of a problem since the alternative to epidemiology is epidemics. People have always understood "your money or your life" as a sales pitch.

Over the years, the cost of storage and computing power decreased drastically. Before cheap storage, any research done by commercial data processing users either was a special project that collected its own data or involved digging thru archival storage. Archival storage meant "magnetic tapes" and a lot of manual labor to extract data from them. Once you got your data out of archival storage, it would not be uniformly formatted. Changes in the way that a company did business, new packages, and patches to old applications all made historical research difficult.

Another factor was that archival storage was destroyed. In the American legal system, old data can be used in court against you, but you are not obligated to retain records past a certain lifetime. One of the jobs of a records manager was to destroy out-of-date records. For the most part, this meant paper shredding, but it also meant erasing magnetic tapes.

The technology has, of course, changed. Commercial users can keep large amounts of data in active storage. Suddenly, you were actually using the "high-end" metric prefixes like tera- and peta- in...