The goal of profiling data is to discover metadata when it is not available and to validate metadata when it is available. Data profiling is a process of analyzing raw data for the purpose of characterizing the information embedded within a data set. Data profiling incorporates column analysis, data type determination, and intercolumn association discovery. The result is a constructive process of information inference to prepare a data set for later integration.

Data profiling is a hierarchical process that attempts to build an assessment of the metadata associated with a collection of data sets. The bottom level of the hierarchy characterizes the values associated with individual attributes. At the next level, the assessment looks at relationships between multiple columns within a single table. At the highest level, the profile describes relationships that exist between data attributes across different tables.

Data profiling includes (among others) the following activities.

• Data model inference, which attempts to derive the data model from undocumented data.

• Type inference, a process to determine the data types associated with the data in each column of a table.

• Value range analysis, which explores the possibility that the values within a column fall within a defined value range (such as 0 to 100).

• Cardinality and uniqueness Cardinality refers to the number of discrete values that appear within a column; uniqueness tests to see that each row in a table has a unique value for any particular set of attributes.

• Frequency distribution, which...