Friday, February 23, 2007

Test coverage

My perception:
Test coverage is the quality metric suggesting the quality of test cases and hence of test adequacy criteria.

In short .... we are trying to quantify test adequacy.

A test case/suite is meant to cover or execute test objects like statements, conditions, paths etc as per its test adequacy criterion. The test adequacy criteria might at times be part of agreement between the client / customer and the service provider.

The more number of test objects the test adequacy criteria covers, more will be its coverage. And the more test coverage given by a test case/suite, better will be its quality.

General definition says ...
Test coverage = (no. of test objects covered or executed at least once) / (total number of test objects)

The concept can be illustrated as follows:
Suppose one test case/test suite covers 80 stmts out of 100, 8 branches out of 10, and 20 paths out of 25 then it is giving 80% stmts coverage, 80% branch coverage and 80% path coverage. And it will be a better test case/suite than that which provides lesser coverage than 80%.As an ideal case test cases/test suite should give 100% test coverage. But in practice it does not happen.

Actually test coverage analysis is a type of the code coverage analysis. The academic world more often uses the term "test coverage" while practitioners more often use "code coverage". Test coverage provides a quantifiable measure of how well the test suite actually tests the product. The most basic form of test coverage is to measure what procedures were and were not executed during the test suite.

There are many other test coverage measures.
1.Statement Coverage
2.Decision Coverage
3.Condition Coverage
4.Multiple Condition Coverage
5.Condition/Decision Coverage
6.Path Coverage
7.Function Coverage
8.Call Coverage
9.Data Flow Coverage
10.Object Code Branch Coverage
11.Loop Coverage
12.Relational Operator Coverage and many more.

Test coverage specifies the extend to which test suite ( at times test criteria) fulfils its defined objectives in the system from its inception to retirement like analysis,design,...... for practitioners (BRD, FDD, TDD, ......)
In each phase of SDLC, nature of test coverage varies but purpose remains same.

Test suites which encompasses test cases and results of each one of them, provides big time support during implementation.
In my opinion, test coverage techniques need to mature a little more so that there are no consequences where one phase (or may be more but rarely all) has higher test coverage but others strive to obtain average.

Hence result of that either others are not performed well or suppose to be eliminated.

No comments: