SonarQube Test Coverage with MsTest

Question

I have been trying to get SonarQube working with a simple dot net app. I have had some success getting it up and running but code coverage is not working.

It looks like many other people have faced this issue when SonarQube discontinued support for many of the 'go to' coverage tool such as DotCover and OpenCover via Gallio

Examples which I have followed are:

• Sonar Runner errors out during processing of .coveragexml file produced from Visual Studio's MSTest

• VS2013 CodeCoverage.exe runsettings file never parses

I have tried a few of the VS command line tools to generate a .coverage file

vstest.console.exe .\UnitTestProject1\bin\Debug\UnitTestProject1.dll /EnableCodeCoverage

and

CodeCoverage.exe collect /output:DynamicCodeCoverage.coverage .\UnitTestProject1\bin\Debug\UnitTestProject1.dll

And written some code like to covert it into a .coveragexml file from here

To get the following XML:

<?xml version="1.0" standalone="yes"?>
<CoverageDSPriv>
  <Module>
    <ModuleName>unittestproject1.dll</ModuleName>
    <ImageSize>32768</ImageSize>
    <ImageLinkTime>0</ImageLinkTime>
    <LinesCovered>12</LinesCovered>
    <LinesPartiallyCovered>0</LinesPartiallyCovered>
    <LinesNotCovered>0</LinesNotCovered>
    <BlocksCovered>9</BlocksCovered>
    <BlocksNotCovered>0</BlocksNotCovered>
    <NamespaceTable>
      <BlocksCovered>9</BlocksCovered>
      <BlocksNotCovered>0</BlocksNotCovered>
      <LinesCovered>12</LinesCovered>
      <LinesNotCovered>0</LinesNotCovered>

And even used XSLT stylesheet provide in one that could be used by the SonarQube runner

<?xml version="1.0" encoding="utf-8"?>
<results>
  <modules>
    <module name="unittestproject1.dll" path="unittestproject1.dll" block_coverage="100" line_coverage="100" blocks_covered="9" blocks_not_covered="0" lines_covered="12" lines_partially_covered="0" lines_not_covered="0">
      <functions>
        <function name="Setup" type_name="UnitTest1" block_coverage="100" line_coverage="100" blocks_covered="1" blocks_not_covered="0" lines_covered="2" lines_partially_covered="0" lines_not_covered="0">
          <ranges>
            <range source_id="1" covered="yes" start_line="13" start_column="9" end_line="13" end_column="10" />
            <range source_id="1" covered="yes" start_line="15" start_column="9" end_line="15" end_column="10" />
          </ranges>
        </function>

when I run Sonar

1. MSBuild.SonarQube.Runner.exe Begin
2. MSBuild
3. MSBuild.SonarQube.Runner.exe end

I get errors like Caused By: unknown XML Node, Expect Coverage but got Results

This is because its does not like the structure of my XML, but I am not sure what is is expecting and how much work I have to do on the coverage file to convert it into a format that Sonar likes

Hopefully I have been going down the wrong path and there is a simple way to integrate VS Coverage or coveragexml files into Sonar without too much work

Extra Information on my Sonar plugins are

1. c# = 4.1
2. Generic Coverage = 1.1

Answer 1

Both OpenCover and dotCover reports are supported by the C# 4.1 plugin. Set the sonar.cs.dotcover.reportsPaths or sonar.cs.opencover.reportsPaths property respectively for both tools to import code coverage.

Gallio is not exactly the go-to tool: the project is inactive since 2013. The main issue with the SonarQube C# Plugin 2.x plugin which relied on Gallio is that it was launching Gallio by itself - not allowing the end-user to customize how tests should be launched and coverage collected.

Now the situation is much easier: Launch your favorite code coverage tool, ask it to produce a report, and feed it to the MSBuild SonarQube Runner.

If you are using Team Foundation Server 2013, enabling code coverage is as choosing the Enable Code Coverage option in the build definition.

Now, it is very unfortunate and confusing that Microsoft has two different .coveragexml formats, and that the SonarQube C# Plugin only supports one of them (that is, for now. see http://jira.sonarsource.com/browse/SONARNTEST-3).

While waiting for that ticket to be fixed, here are the steps to produce the expected .coveragexml report (note: Replace 14 by 12 in the various paths if you are using VS 2013 instead of 2015):

1. MSBuild.SonarQube.Runner begin /k:SonarQube_Project_Key /n:SonarQube_Project_Name /v:1.0 /d:sonar.cs.vscoveragexml.reportsPaths=%CD%\VisualStudio.coveragexml
2. msbuild
3. "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" collect /output:VisualStudio.coverage "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "UnitTestProject1\bin\Debug\UnitTestProject1.dll"
4. "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" analyze /output:VisualStudio.coveragexml VisualStudio.coverage
5. MSBuild.SonarQube.Runner end

I would not recommend to use XSLTs to convert code coverage report formats, use the CodeCoverage.exe Microsoft tool instead.

Answer 2

Please ensure below things:

1) Do you have the coverage file being generated by the build process?

If this is not generated, then possibly build script needs to be updated to generate one or else need to add an explicit step in build automation tool to generate it.

For e.g. "C:\Program Files\dotnet\dotnet.exe" test <Target-filename>.csproj --logger:trx --collect:"Code Coverage"

2) Ensure that an xml version of the CodeCoverage file is generated.

So there will be two files:

CodeCoverage (generated using dotnet command)

CodeCoverageXml (This is generated by using "CodeCoverage.exe analyze /output: newfilename Your_CodeCoverage_file" )

3) Is the correct path of Coveragefile provided while invoking SonarQube?

4) Does your build server have correct version of 'Dotnet' installed and ensure that same version of dotnet is executed?

Sometimes build server has multiple versions of Dotnet (e.g. v2.0.3 and v2.1.0)

To generate the coverage report, it is very important to use the correct version or else it will show as zero or wont generate the Coverage file itself.