When to stop testing ?
Testing in software systems is a complex task due to interdependency of system, complexity of applications etc. Complete testing is not possible for almost all projects and a determination needs to be made early on by the project test team , SQA and PM on when to stop testing.
This will be outlined in the detailed test plan and it should be discussed and concurred upon with the customer. It is generally a good idea to list down what testing is going to be left out.
Eg: It is agreed that test all branches of code will not be done, due to the time constraint of project. Successful execution of Test case xyz will imply that the system works as intended.
Some of the common factors and constraints that should be considered when decided on when to stop testing are:
1. Testing budget of the project. Or when the cost of continued testing does not justify the project cost.
2. Resouces available and their skills.
3. Project deadline and test completion deadline.
4. Critical or Key Test cases successfully completed. Certain test cases even if they fail may not be show stoppers.
5. Functional coverage, code coverage, meeting the client requirements to certain point.
6. Defect rates fall below certain specified level & High priority bugs are resolved.
7. Project progresses from Alpha, to beta and so on.
Testing is potentially endless process. Once the product is delivered the customer starts testing everyday when they use the product. So the decision has to be made early , as to what is the acceptable risk , based on level of testing possible.