Before implementing any validation scheme, you must ask two important questions.

• How do I know if this is the right scheme?

• Is this scheme scalable to grow with my product?

When your total test time surpasses seventy-five percent of your execution resources, you will see a benefit in using CBTM. Execution resources are the total amount of hours available for people and computers to execute tests. As your total test time increases, the benefit increases. CBTM becomes most beneficial when your total test time surpasses your total available execution resources. This represents the point at which you have to run a portion of your test suite as opposed to the entire suite. CBTM allows you to make informed decisions regarding which tests not to run. Before reaching this threshold, CBTM benefits you by moving defect discovery forward in the validation cycle as mentioned in the section above.

In order to be fully scalable, CBTM requires a full time test suite manager. As your test suite grows, you will eventually require dedicated management headcount to keep your test suite operating efficiently. One great aspect of CBTM is the ease in which it integrates with automated testing. Automated testing allows you to scale your test capability by adding or removing test machines as needed. Chapter 11 fully explores the concept of automation.