Adding code coverage with nyc

In this post, I’ll add code coverage to the build pipeline and configure TeamCity to break the build if the code coverage drops.

Installation

The tool we’ll use is nyc and it’s installed simply as a dev dependency with npm install --save-dev nyc.

npm scripts

I’ll also add two npm scripts in package.json:

"nyc": "nyc npm test",
"nyc-junit": "nyc npm run test-junit"

They both use the npm scripts configured in the previous post about unit tests. The second one produces the unit test XML report, so it will be used on the build server.

Configuring nyc

nyc can be configured by adding a nyc element in package.json:

  "nyc": {
    "all": true,
    "reporter": [
      "text",
      "html",
      "teamcity"
    ],
    "exclude": [
      "wdio.conf.js",
      "coverage/**/*.js",
      "test/**/*.js"
    ]
  }

I’ll explain a bit my configuration:

"all": true specifies that the code coverage should include source code that is not referenced via unit tests. This is not the default, which I personally find odd. Without this flag, it's possible to keep adding code without unit tests and you'll never notice a drop in the code coverage. I prefer to calculate the code coverage for all source code, so that if unit tests are missing, the build will break.
I use three reporters: text, html and teamcity. The first one prints a short summary table to the console, which is the default behavior. The second one generates a HTML report in the coverage folder. You can use this report to see what you have covered and what you're missing. That last reporter sends the code coverage metrics to TeamCity.
The exclude array indicates files I don't want to calculate in the code coverage. That's the configuration of WebdriverIO, the tests and the code coverage report.

Results

This is how the HTML report looks like on a file level (I only have one file to test in the sample project anyway):

As I said in the previous post, I only added a unit test for the /version endpoint, so the other two functions are not covered and appear with a red-pink color.

In TeamCity, the code coverage metrics are visible on the Overview tab:

Failure conditions

It is possible to configure nyc to break the build when the code coverage drops under a certain fixed value. TeamCity allows you to take a different approach: break the build if code coverage drops by a certain amount compared to the previous successful build:

TeamCity failure condition gives dynamic thresholds

This is great if you have a project with a low code coverage and you are determined to improve it. Every green build sets automatically the bar higher, without having to manually update the thresholds: