Mutation testing (or mutation analysis or program mutation) is used to design new software tests and evaluate the quality of existing software tests. Each mutated version is called a mutant and tests detect and reject mutants by causing the behavior of the original version to differ from the mutant.
- Limit the mutation operators used
- Limit the number of mutations per class
- Tweak the number of threads
- Target only specific portions of your codebase
- Don't run this with every commit as it takes time to run these test....may be as smoke test
Add Pitest plugin to gradle
id 'info.solidsoft.pitest' version '1.5.2'
Create pitest task with your needed configuration
pitest {
mainSourceSets.set([sourceSets.main]) // Only for individual contribution, not recommended for repo having more than 10-15 classes
testSourceSets.set([sourceSets.test]) // Only for individual contribution, not recommended for repo having more than 10-15 classes
targetClasses.set(["com.foo.*","com.bar.*"]) // Ignore if mainSourceSets & testSourceSets is specified
excludedClasses.set(["com.foo1.*","com.bar.IT*"]) // Ignore if mainSourceSets & testSourceSets is specified
pitestVersion.set('1.6.7') // Not needed if you want to use the default version provided by plugin
junit5PluginVersion.set('0.14')
outputFormats.set(['HTML']) // XML is also available
mutators.set(["ROR", "RETURN_VALS", "INVERT_NEGS", "MATH", "VOID_METHOD_CALLS", "NON_VOID_METHOD_CALLS"])
avoidCallsTo.set(["java.util", "java.lang"])
threads.set(4)
timestampedReports.set(false)
mutationThreshold.set(95)
coverageThreshold.set(95)
}
Bind the created task with test
/build
task as required
test {
finalizedBy 'pitest'
}
Intellij idea → Preferences → Plugins → search "PIT mutation testing Idea plugin"
Edit configuration for Pitest Runner and add the needed parameters from below
Flag: --targetClasses
Possible Values: PackagePath
Eg: `--targetClasses com.foo.service.auth*,com.foo.service.product.service.*`
Flag: --sourceDirs
Possible Value: Package ROOT Path
Eg: `--sourceDirs <PATH_TO_BASE_DIR>/service/src/main`
List of formats in which to write mutation results as the mutations are analysed. Supported formats are HTML, XML, CSV.
Flag: --outputFormats
Possible Values: XML,HTML
Eg: `--outputFormats XML,HTML`
Flag: --excludedClasses
Possible Values: REGEX/FileName
Eg: `--excludedClasses *Test`
Flag: --excludedTestClasses
Possible Values: REGEX/FileName
Eg: `--excludedTestClasses *IntegrationTests,*IntegrationTest`
PIT filters are matched against the class names in the compiled binary, not against the source file name.
Flag: --excludedMethods
Possible Values: annotator names
Eg: `--excludedMethods toString,hashCode,equals`
Excluded methods are used to avoid creating mutants within methods that match the supplied list of names.
PIT test will not consider spring annotator like @Service, @Api, @RestController whereas it mutates other value based spring annotations like @Value and also java annotations like @ToString,@EqualsAndHashCode
Flag: --mutators
Possible Values: Mutator type
Eg: `--mutators ROR,RETURN_VALS`
List of Mutators recommended
ROR
RETURN_VALS
INVERT_NEGS
MATH
VOID_METHOD_CALLS
NON_VOID_METHOD_CALLS
If we need to use other mutators, vist official page
Flag: --verbose
export a variable named $CLASSPATH
and add all the paths to the dependency jars and Pitest jars
Pitest jars
* pitest-x.y.z.jar
* pitest-command-line-x.y.z.jar
* pitest-entry-x.y.z.jar
Command:
java -cp $CLASSPATH \
org.pitest.mutationtest.commandline.MutationCoverageReport \
--reportDir <PROJECT_PATH>/build/reports/pittest \
--sourceDirs <PROJECT_PATH>/<MAIN_SRC_PATH> \
--outputFormats XML,HTML \
--mutators ROR,RETURN_VALS,INVERT_NEGS,MATH,VOID_METHOD_CALLS \
--targetClasses "com.foo.service.auth*,com.foo.service.product.*" \
--excludedClasses "*Test" \
--excludedTestClasses "*IntegrationTests,*IntegrationTest"