Wednesday, June 27, 2007

One Simple Metric

At our last Agile Iowa meeting on Thursday, Jared Richardson, he was talking about Agile Testing Strategies. He mentioned a metric he used that I found very interesting. He took three variables and created a report that listed the top 10 method/assemblies that were at risk.
Here is the (simple) equation:
Cyclomatic Complexity (CC):
Usage (U): How many other methods/classes rely on this code.
Test Coverage Percentage (TC): The percentage of the code that is exercised by a unit test
Risk Factor = CC x U x (1 - TC)
Let's look at a couple examples:
This first table shows a method used by 50 other methods (Usage) with a Cyclomatic complexity (CC) of 20. I then vary the code coverage for this method. Without any unit tests, the risk factor is a cool "g" (or Grand). As the test coverage increases the Risk Factor decreases. When the test coverage reaches 100%, the Risk Factor is reduced to zero. The function is linear, so decreasing the usage or the complexity will also reduce the Risk Factor. So it make sense that if a method isn't used, it has a risk factor of zero.
CC Usage Code Coverage Risk Factor
20 50 0 1,000
20 50 0.1 900
20 50 0.5 500
20 50 0.9 100
The following methods all have the same risk factor of 250. Do you think this is reasonable? Should one of these factors be weighted heavier?
CC Usage Code Coverage Risk Factor
10 50 .5 250
50 10 0.5 250
50 50 0.9 250
For now I'm going to see how I can get the usage and the code coverage numbers automated... I can tweak the algorithm more later.

No comments:

Post a Comment