Computing code coverage for a .NET Core project with Azure DevOps and Coverlet

Creating automated tests is important to be sure your application behaves as it should. The more tests you write, the more use cases are covered. Sometimes it can be hard to know which parts of the application are well-tested, and which ones are not. That's why code coverage is interesting. It can be a useful measure to detect potential risk areas in the application (i.e. the most complex methods with the least coverage). Let's see how you can do that easily with .NET Core and Azure DevOps.

First, you need to compute the code coverage when you run the tests. There are multiple solutions, some are free such as coverlet, and some are paid such as dotCover. In this post, I'll use coverlet. First, you need to install the NuGet package coverlet.msbuild. This package is integrated directly with dotnet test as we'll see later.

<Project Sdk="Microsoft.NET.Sdk">

  ...

  <ItemGroup>
    <PackageReference Include="coverlet.msbuild" Version="2.5.1">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>
  </ItemGroup>

  ...

</Project>

You can test that everything works great by running the command dotnet test /p:CollectCoverage=true. After the test run, a JSON file should be created next to the csproj file. Azure DevOps doesn't support this file, but coverlet can output the result file in many standard formats. So, you can use the cobertura format which is supported by Azure DevOps.

dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura

After running the command for each test project, you get one code coverage file per project. These files are not very convenient to read. Let's generate a cool UI using ReportGenerator. This free tool allows you to generate a website to navigate into the files and see the lines covered by the tests. You can install as a .NET global tool:

dotnet tool install -g dotnet-reportgenerator-globaltool
reportgenerator -reports:tests/**/coverage.cobertura.xml -targetdir:Report -reporttypes:HtmlInline_AzurePipelines;Cobertura

You now have a website that conveniently displays the code coverage. Note that there are multiple output formats. Here I use HtmlInline_AzurePipelines because at the end we would like to view this web site directly in Azure Pipelines, so it seems to be the best output for that.

Now let's integrate the tests and the code coverage on the build system (Azure Pipeline). Here's the YAML file that describes the build steps:

variables:
  buildConfiguration: Release
  DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1

steps:
# Install the latest version of the dotnet sdk
- task: DotNetCoreInstaller@0
  displayName: 'Use .NET Core sdk 2.2.103'
  inputs:
    version: 2.2.103

# build all projects
- task: DotNetCoreCLI@2
  displayName: dotnet build
  inputs:
    projects: 'src/**/*.csproj'
    arguments: '--configuration $(BuildConfiguration)'

# Run all tests with "/p:CollectCoverage=true /p:CoverletOutputFormat=cobertura" to generate the code coverage file
- task: DotNetCoreCLI@2
  displayName: dotnet test
  inputs:
    command: test
    arguments: '--configuration $(BuildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura'
    projects: 'tests/**/*.csproj'
    nobuild: true

# Generate the report using ReportGenerator (https://github.com/danielpalme/ReportGenerator)
# First install the tool on the machine, then run it
- script: |
    dotnet tool install -g dotnet-reportgenerator-globaltool
    reportgenerator -reports:$(Build.SourcesDirectory)/tests/**/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/CodeCoverage -reporttypes:HtmlInline_AzurePipelines;Cobertura
  displayName: Create Code coverage report

# Publish the code coverage result (summary and web site)
# The summary allows to view the coverage percentage in the summary tab
# The web site allows to view which lines are covered directly in Azure Pipeline
- task: PublishCodeCoverageResults@1
  displayName: 'Publish code coverage'
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: '$(Build.SourcesDirectory)/CodeCoverage/Cobertura.xml'
    reportDirectory: '$(Build.SourcesDirectory)/CodeCoverage'

When you run a build, you now have the code coverage result in the Summary tab of the build:

A new tab is also available: "Code Coverage". When you click on it, you'll see the generated website.

If you want an example of coverlet and Azure DevOps, you can look at one of my GitHub project Meziantou.Framework and its first pipeline with coverlet integrated. You'll also notice that the result is reported on GitHub:

Voilà! Thanks to coverlet and ReportGenerator you can easily get a quick view of the parts of your applications that may need more tests.

Do you have a question or a suggestion about this post? Contact me!