Is the code coverage a sufficient metric

When writing unit tests, the common goal is to achieve the highest possible code coverage, ideally 100%. But does reaching that goal mean the code is actually tested correctly? Can you say there are no bugs in your application? Let's explore this with an example.

public class Foo
{
    private int _add = 42;

    public int Bar(int x)
    {
        x += _add;
        return x;
    }
}

We logically write the following test:

[TestMethod()]
public void BarTest()
{
    Foo target = new Foo();
    Assert.AreEqual(42, target.Bar(0));
}

With this test, we have a code coverage of 100%. Nothing extraordinary here. Let's change the function:

public int Bar(int x)
{
    x += _add;
    _add++;
    return x;
}

We introduced a bug: the field _add is now incremented on every call. However, the test still passes and code coverage remains at 100%. Verifying that repeated calls return the same result would catch this issue.

[TestMethod()]
public void BarTest()
{
    Foo target = new Foo();
    Assert.AreEqual(target.Bar(0), target.Bar(0));
}

To conclude, code coverage is a useful indicator for identifying untested code. However, it is not sufficient on its own. You can cover every line and still miss bugs if your assertions are weak or incomplete.

Do you have a question or a suggestion about this post? Contact me!