TABLE OF CONTENTS The previous chapters dealt mainly with what should be tested, how, and why. If we were to turn all our testing ideas into automated test cases, then the test effort to implementation effort ratio would be at least 10 to 1, if not higher. One important question is therefore, When will we have sufficiently tested?
Let's be honest, only very few of us are addicted to testing. Most development teams suffer rather from the opposite phenomenon. There are too few tests for us to always deliver or restructure with a clear conscience. The other important question is therefore, When have we tested too little?
There are numerous factors that play a role in determining the optimal testing effort. The most important ones follow:
Complete testing with the declared intention to verify the correctness of a program is impossible to achieve for all nontrivial programs. The objective of our testing efforts; therefore, can only be to find as many faults as possible at a manageable effort level.
There is an acceptable error level for each system. How high this level is depends on the type of system: the software to control a radiological unit should certainly not contain as many errors as a Web application for sock subscriptions. [1] The acceptable error level is normally specified by the customer in metrics like mean time between failure or the like. The actual error level achieved by an implementation is usually hard to predict without running the...
