Interesting Debate

Me and Cem have been having a very interesting debate over complete testing.

According to Cem the correct answer to the question should be choice ‘b’
Qs ) Which is the best definition of complete testing:
Choose one answer.
a. You have completed every test in the test plan.
b. You have discovered every bug in the program.
c. You have reached the scheduled ship date.
d. You have tested every statement, branch and combination of branches in the program.

And I say answer should be choice ‘d’. Cem reasons, How can testing be complete if there might still be bugs remaining?

And I go back and say that I feel ‘b’ and ‘d’ complement each other.
‘d’ is probably one way to achieve ‘b’.

I reason that all the business units across the world ( I am leaving the small software’s out of scope here, as they can be tested completely most of the times) plan their product releases. Before a release, it is certified that the testing has been complete. The products are released, but still we find 'n' number of customer bugs reported and which directly or indirectly lead to more regressions introduced in the software. These regressions are sometimes identified by the QA teams and sometimes are reported back as new bugs by the customers.
I have till date not seen any white paper from any of the big business houses where they have not had a single bug reported ( either internal or external) against a release.
Now, coming back to my interpretation….. What do you actually try to do in complete testing scenario ?? you try and have such test cases that will check all possible statements, branches and combination of branches in the program. Basically, I try and ensure that all my complexity node points are well covered during testing.

Yes, when I say ‘complete testing’, my idea is that ideally what ever I can think of as a test engineer ( with all my white box and black box test cases) I have covered all the possible scenarios I can think of. But factually, there is always a chance that there are few uncovered, un-trodden pathways lurking around the numerous code lines. More the components, more the dependencies, more the chances of failures. This is the simple testing ‘mantra’ that fits into my thoughts……..

It will be nice if we can have your views on my thoughts ……..

Before I end ….Can anyone recall embarrassment of Bill Gates as he stood to release Windows ’98 and then publicly had to call off the release coz of a failure at the time of release and all this was going on live ……….broadcasting all over the world ……

and last but not the least ....... I am not debating against the definition, but the thought that goes behind the definition ......