Computational Intelligence for Missing Data Imputation, Estimation, and Management: Knowledge Optimization Techniques

This chapter introduces a novel paradigm to impute missing data that combines a decision tree, autoassociative neural network (AANN) model and a principal component analysis-neural network (PCA-NN) based model. These models are designed to answer the crucial question of whether the optimization bounds actually matter. For each model, the decision tree is used to predict search bounds for a hybrid simulated annealing and genetic algorithm method that minimizes an error function derived from the respective model. The models' ability to impute missing data is tested and then compared using HIV sero-prevalance data. Results indicate an average increase in accuracy of 13% with the AANN based model's average accuracy increasing from 75.8% to 86.3% while that of the PCA-NN based model increasing from 66.1% to 81.6%.
Missing data is a widely recognized problem affecting large databases that creates problems in many applications that depend on access to complete data records such as data visualization and reporting tools. This problem also limits data analysts interested in making policy decisions based on statistical inference from the data and thus estimating missing data is often invaluable as it preserves information and produces better, less biased estimates than simple techniques (Fogarty, 2006; Abdella, 2005; Nelwamondo, 2006) such as listwise deletion and mean-value substitution (Yansaneh, Wallace, & Marker, 1998; Allison, 2000).
Inferences made from available data for a certain applications depend on the completeness and quality of the data being used in the analysis. Thus, inferences made from a complete data are most likely...