Norman E. Fenton, Member, IEEE Computer Society, and Martin Neil, Member, IEEE Computer Society

• N.E. Fenton and M. Neil are with the Centre for Software Reliability, Northampton Square, London ECIV 0HB, England. E-mail: {n.fenton, martin}@csr.city.ac.uk.

Manuscript received 3 Sept. 1997; revised 25 Aug. 1998.

Recommended for acceptance by R. Hamlet.

For information on obtaining reprints of this article, please send e-mail to: tse@computer.org , and reference IEEECS Log Number 105579.

Abstract Many organizations want to predict the number of defects (faults) in software systems, before they are deployed, to gauge the likely delivered quality and maintenance effort. To help in this numerous software metrics and statistical models have been developed, with a correspondingly large literature. We provide a critical review of this literature and the state-of-the-art. Most of the wide range of prediction models use size and complexity metrics to predict defects. Others are based on testing data, the "quality" of the development process, or take a multivariate approach. The authors of the models have often made heroic contributions to a subject otherwise bereft of empirical studies. However, there are a number of serious theoretical and practical problems in many studies. The models are weak because of their inability to cope with the, as yet, unknown relationship between defects and failures. There are fundamental statistical and data quality problems that undermine model validity. More significantly many prediction models tend to model only part of the underlying problem and seriously misspecify...