When to stop testing. What's your opinions?
Admin advise me move it to new thread. So I think it's better when I create in Bangalore 2006 Study Group. It's very effective for us to learn and discuss in group. We could share and study much more new knowledge.
Return the question: When to stop testing. Admin and I are dicussing on it. I will relate it below (or reference in http://www.csqa.info/welcome_join_here#comment-130 ):
pmyhang's opinion:
It is an unobvious question, stop testing maybe test in each phase or all test activity. But for me, it's up to each project. And the person make decision is Project Manager. Because he/she responses in allocating schedule, resource, budget,..in project. How about you? What is your opinions?
Admin'n opinion:
You have indeed hit the nail... When to stop testing is a favorite question and the answer to that will depend upon some assumptions that you will have to make.
You did accurately point out that each project has a allocated schedule, budget, Quality requirements and resources.
Its not possible to test every flow and conduct all types of testing on every project. So during the early stages of project the Test lead or PM will have to make a test plan. This will include the entry and exit criteria and types of tests to be carried out.
So each project is treated on case to case basis and if the question does not give much information then we have to make some assumptions before we answer the question.
pmyhang's opinion:
I agree with Admin. First, we should define what quality goals. After that, we could make a appropriate plan, as long as we meet the customer requirements and they could accept the product we delivered.
@Admin: I wonder whether my answer meets questioner's expectance? Admin, please share experience to me :)
Do you have a specific subjective question(s) related to this topic, maybe we can try answering that and that will help us understand better
- Login or register to post comments
Submitted by admin on October 26, 2006 - 2:49am.It's one of the questions in CSQA's CBOK. Because it is unclear so I am not sure my answer is clear and full. You got CSQA, so according to you, how way to answer the question to get mark absolutely.
- Login or register to post comments
Submitted by pmyhang on October 26, 2006 - 3:23am.I guess there is no straight answer for this, but for the purpose of CSQA certification you can pick following points in exam and elaborate.
Testing in software systems is a complex task due to interdependency of system, complexity of applications etc. Complete testing is not possible for almost all projects and a determination needs to be made early on by the project test team , SQA and PM on when to stop testing.
This will be outlined in the detailed test plan and it should be discussed and concurred upon with the customer. It is generally a good idea to list down what testing is going to be left out.
Eg: It is agreed that test all branches of code will not be done, due to the time constraint of project. Successful execution of Test case xyz will imply that the system works as intended.
Some of the common factors and constraints that should be considered when decided on when to stop testing are:
1. Testing budget of the project. Or when the cost of continued testing does not justify the project cost.
2. Resouces available and their skills.
3. Project deadline and test completion deadline.
4. Critical or Key Test cases successfully completed. Certain test cases even if they fail may not be show stoppers.
5. Functional coverage, code coverage, meeting the client requirements to certain point.
6. Defect rates fall below certain specified level & High priority bugs are resolved.
7. Project progresses from Alpha, to beta and so on.
Testing is potentially endless process. Once the product is delivered the customer starts testing everyday when they use the product. So the decision has to be made early , as to what is the acceptable risk , based on level of testing possible.
- Login or register to post comments
Submitted by csqa on October 26, 2006 - 4:59pm.Thank CSQA for sharing!
I think the way you analyse are in detailed and it is fair completion.
Is there any other opinions? :)
- Login or register to post comments
Submitted by pmyhang on October 27, 2006 - 4:50am.