Data Quality: The Field Guide

Data quality, like the Information Age, is in its infancy. We should expect it to expand and evolve in numerous directions, some predictable, some not. This chapter summarizes clear trends, even though the details are yet to emerge.
First, demands for high-quality data will grow and the demands will be fantastically diverse. We've already noted that different data customers the employee working within an information chain, the manager, and the general public have vastly different needs. This will accelerate.
Second, demands for new data and new information products will also grow, perhaps even faster. To date, there has been something of a commodity approach to data. Data suppliers provide pretty much the same data to everyone. But data are subtle and nuanced and even slightly different needs should motivate different data. A recent example involves the consumer price index (CPI). An issue that played out in the national news involved whether the CPI overstated inflation (a classic data quality issue). If so, government pay-outs for Social Security and other programs could accelerate much faster than desired. Part of the issue was that changing the CPI metric to better suit one purpose might compromise others.
A better approach than adjusting the CPI metric would have been to define issue-specific CPIs. Thus, if one program's goal is to adjust seniors' income so they are not left out, there should be a "senior-CPI" specially crafted to meet the needs of that program and its constituencies. The market basket used in calculating this CPI would reflect...