grunt-code-coverage-enforcer
Enforces code coverage thresholds for a project with lcov files. This means it works with a wide range of code coverage tools like blanketjs, istanbul, .....
Introduction
Most code coverage enforcers are tied to a specific code coverage tool like blanketjs or istanbul. This plugin works on the lcov file that can be generated by most test runners.
As a result you can work with any tests runners that are capable of generating the widely used LCOV file format.
Why is our plugin better
Most other coverage threshold tools fail the build only when you have a test and the test doesn't meet the threshold criteria. Most tools will actually pass the build if you dont write any tests for your project.
This plugin will check and fail if there are files that have not been covered at all.
You can always exclude the files that you specifically do not want to test for coverage. Since the policy is opt-out, less files will slip through the cracks ;-)
Getting Started
npm install grunt-code-coverage-enforcer --save-dev
Once the plugin has been installed, it may be enabled inside your Gruntfile with this line of JavaScript:
grunt.loadNpmTasks('grunt-code-coverage-enforcer');
The "code-coverage-enforcer" task
Enabling lcov reporter in your test runner.
You will need to enable lcov reporter in your test runner. The specific config for this may vary for each runner. Please check your specific runner for instructions.
Sample Intern config
intern: {
runner: {
options: {
config: "<intern- config>",
runType: "runner",
reporters: ["console", "lcov"]
}
}
}
Sample Karma config
(http://karma-runner.github.io/0.8/config/coverage.html)
coverageReporter = {
type : 'lcovonly',
dir : 'coverage/',
file : 'lcov.info'
}
Sample Istanbul config
(https://github.com/gotwarlost/istanbul, https://www.npmjs.org/package/grunt-istanbul-coverage, https://www.npmjs.org/package/grunt-mocha-istanbul)
Cli version of istanbul has a report parameter that you need to set to lcov or lcovonly. By default it should generate the lcov files. Most istanbul plugin allow this to be passed into their grunt config as well.
Here is one suc example for the grunt-mocha-istanbul plugin
options: {
reportFormats: ['cobertura','lcovonly']
}
Mocha
Mocha has an lcov reporter that can be foounf here https://nodejsmodules.org/pkg/mocha-lcov-reporter You can also use istanbul with Mocha using the plugin here https://github.com/pocesar/grunt-mocha-istanbul
This should typically generate an lcov.info file that will specified as input into the code-coverage-enforcer plugin.
Overview
In your project's Gruntfile, add a section named code-coverage-enforcer
to the data object passed into grunt.initConfig()
.
grunt.initConfig({
"code-coverage-enforcer": {
options: {
lcovfile: "lcov.info",
lines: 60,
functions: 60,
branches: 60,
src: "src",
includes: ["src/**/*.js"],
excludes: ["src/uncoveredfile.js"]
}
},
})
the default threshold values are
{
lines: 50,
functions: 50,
branches: 0,
includes: ["**/*.js"],
src: process.cwd(),
excludes: [],
logCurrentCoverage: false,//logs the coverage information for all the include files if true
passMessage: "Yay! All is well!",//Message configuration for passed coverage checks
failMessage: "Failed to meet code coverage threshold requirements.", //Message configuration for failed coverage checks.
failBuild: true, //If you want to fail the build if coverage checks pass or fail.
failBuildThreshold: 0 // Fine grained configuration to configure to build failures
}
You can also provide multiple configurations for different features/packages of your project.
Options
lines
Type: Number
Default value: 50
A Number value that is used to specify in % the line coverage required.
functions
Type: Number
Default value: 50
A Number value that is used to specify in % the function coverage required.
branches
Type: Number
Default value: 50
A Number value that is used to specify in % the branches coverage required.
src
Type: string
Default value: src
The location of the folder within which we will search for files.
includes
Type: matcher
Default value: *
The matcher required to compare files within the src location that are to be included
excludes
Type: [matcher]
Default value: []
The matchers required to compare files within the src location that are to be excluded.
logCurrentCoverage
Type: boolean
Default value: [false]
Can be used to log the current coverage limits.
passMessage
Type: string
Default value: Yay! All is well!
Message to print when the coverage checks pass
failMessage
Type: string
Default value: Failed to meet code coverage threshold requirements.
Message to print when the coverage checks fail
failBuild
Type: boolean
Default value: true
Whether to fail the build when the coverage checks fail. Recommend to keep this configuration value to true always. You can set it to false, when you are initially trying/testing out this grunt task.
failBuildThreshold
Type: Number
Default value: 0
This is a fine grained control that is added to control the behavior of the grunt task for existing projects that may not have good code coverage for all the files in the project. For e.g. in your project x if most of the files have coverage above 40%, but there are some files in the project whose coverage is below 40% and you expect that those files don't get the love they deserve, you can set the failBuildThreshold to 40%. This will put the files below 40% whose coverage dropped in the needs attention section and continue to fail build for the files whose coverage dropped for the files with existing coverage more than 40%. Needless to mention that the way the task would know the previous coverage configuration is if you tell this task what the existing coverage are by passing the information in 'src' option
Usage Examples
Default Options
In this example, the default options are used
grunt.initConfig({
"code-coverage-enforcer": {
options: {
lcovfile: "lcov.info",
includes: ["src/**/*.js"],
excludes: ["src/uncoveredfile.js"]
}
},
})
Custom Options
In this example, custom options are used to specify the threshold limits
grunt.initConfig({
"code-coverage-enforcer": {
options: {
lcovfile: "lcov.info",
lines: 60,
functions: 60,
branches: 60,
src: "src",
includes: ["src/**/*.js"],
excludes: ["src/uncoveredfile.js"]
}
},
})
With the release of version 0.2.0, you will be able to customize the code-coverage on and individual folder/package level. The configuration options on an individual folder/package level can be specified in the following way:
grunt.initConfig({
"code-coverage-enforcer": {
options: {
lcovfile: "lcov.info",
lines: 60, //Global line coverage configuration
functions: 60, // Global function coverage configuration
branches: 60, // Global branch coverage configuration
src: [{
path:"middleware",
lines:90
}, {
path:"backend",
lines: 90,
functions: 80
}, {
path:"frontend",
lines: 90,
functions: 80,
branches:70,
includes:["**.js"],
excludes:["frontend/exclude.js"]
}
includes: ["src/**/*.js"],
excludes: ["src/uncoveredfile.js"]
}
},
})
In the above example we configured our project to have the following three coverage configuration
- 90% line coverage, 60% function coverage, 60% branch coverage for all the files in middleware folder.
- 90% line coverage, 80% function coverage, 60% branch coverage for all the files in backend folder.
- 90% line coverage, 80% function coverage, 70% branch coverage for all the files in frontend folder except exclude.js file.
If any of the configuration option is not provided for a given folder/package, the default configurations are chosen.
Recommendations
We recommend using a specific task names build acceptance that will run the code-coverage-enforcer along with other build acceptance type of tasks.
Here is an example config that should go into your Gruntfile
grunt.registerTask("build-acceptance", ["code-coverage-enforcer"]);
You can now simply invoke this from command line using
grunt build-acceptance
Contributing
The project includes jshint and jscs config as part of its dev grunt tasks. Please run grunt lint to ensure that the code passes the code conventions.
Once you have your code ready ready please make a pull request to the author.
Release History
March 17 2-15 0.3.0: Performance improvements, Refactoring of the code by using existing grunt tasks to simplify the code. Added a few more configuration options that will help with finer controls on how coverage enforcer functions.
Release History
March 17 2-15 0.2.0: Threshold configuration now supports per folder configuration in addition to one config for all.
June 19 2014 Initial Release.
License
Copyright (c) 2014 Intuit Inc.
Licensed under the MIT license.