Code Coverage’s Mixed Message

I’m in agreement with Robert Glass when he says “100% test coverage is insufficient. 35% of the faults are missing logic paths.” It’s not controversial, but I’d like to give my perspective on it.

If you have an automated unit test suite, low code coverage is an indication that you need more tests. Unfortunately, high code coverage does not tell you if you have enough tests or the right tests. Adding to Robert Glass’ observation, executed code is not necessarily tested code. Imagine a test case that runs through many lines of code, but never checks that they are doing the right thing. At best this is the “I don’t have any bad pointers” test.
Continue reading