Pitest (aka PIT) is a state of the art mutation testing system for Java and the JVM.
Read all about it at http://pitest.org
Nothing yet, why not contribute?
- #557 - Issues running on Java 8
- #518 - Experimental BigInteger mutator (thanks @ripdajacker)
- #513 - Sort mutators in html report (thanks @ThLeu)
- #553 - Classic mutators from literature (thanls @LaurentTho3)
Older versions
- #510 - Compute mutation test matrix (thanks @nrainer)
- #519 - Java 11 support
- #500 - Support for large classpaths with new
useClasspathJaroption (thanks @jqhan)
- #446 - Ignore empty directories when aggregating report (thanks @maxgabut)
- #457 - Fix base dir in multi module projects (thanks @sarahBuisson and @giggluigg)
- #471 - Basic Java 11 support
- #477 - Null byte in xml (thanks @maxgabut)
- #480 - Allow aggregation of timestamped reports (thanks @maxgabut)
- #495 - Move summary to end of console output (thanks @pedrorijo91)
- #499 - Fix for equivalent mutants in new return val mutators
- #445 - Raise minimum supported java version to 8
- Switch to ASM 6.1.1
- #448 Aggregate report based on module (thanks @sarahBuisson)
- #462 Escape test names in coverage xml (thanks @maxgabut)
- #433 - Aggregate reports via maven (thanks @rchargel)
- #438 - Filter tests at method level (thanks @ftrautsch)
- #439 - Primitives return mutator doesn't handle bytes
- #380 - Remove reliance on xstream for improved java 9 support
- #414 - Replace xstream history store (thanks @kiftio)
Note when upgrading that history files generated by previous releases are not compatible with 1.3.2 an above.
- #434 - XStream security warning
- #196 - Raise minimum java version to 7
- #425 - Broaden for-each detection
- #428 - Add
excludedTestClassesparameter - #379 - Remove support for mutating static initializers
- #432 - Add export plugin functionality into pitest
- #427 - Better return values mutators
- #429 - Simplify test api
428 changes the existing behaviour of the excludedClasses filter. Previously this excluded classes from both being mutated and run as tests which may require changes for some people when upgrading.
As a result of 429 TestNG tests will no longer be automatically detected - the new testPlugin parameter must be explicitly set to "testng".
- #407 - SCM goal cannot be run from within module (thanks @sbuisson)
- #256 - Inline style violate Content Security Policy (thanks @kiftio)
- #403 - No css in the html report viewed from jenkins (thanks @kiftio)
- #409 - Mutate based on changes across branched (thanks @sbuisson)
- #418 - Avoid for loop increments
- #424 - Avoid for each code
- #389 - Widen matching of implicit null checks
- #397 - Report reasons maven modules skipped
- #389 - Filter junk getClass null checks in lambdas
- #400 - Update to ASM 6.0
- Update to ASM 6 for Java 9 support
- #388 - Mark maven plugin as threadsafe
- #362 - Suppress common equivalent mutant in equals methods
- #367 - Static analysis cannot find core classes in some projects
- #347 - Avoid autogenerated code (e.g by lombok) anotated with @Generated
- #285 - Avoid compiler generated conditionals in switch on string. (thanks @Kwaq)
- #218 - New "naked receiver" / method chain mutator. (thanks @UrsMetz)
- #354/#356 - New extension point plus changes to reduce cost of random access to bytecode for static analysis
- #353 - Improved static initializer code detection
- #364 - Filter infinite loops via static analysis
- #365 - Configuration system for plugins
(Note #347 will not detect javax.annotaion.Generated as it has only source retention.)
- #318 - Separate main process code from code sharing client classpath
- #295 - Ignore abstract TestNG tests
- #215 - Automatic selection of target classes for maven
- #314 - Do not fail for maven modules without tests or code
- #306 - Do not fail when only interfaces are in changeset
- #325 - Anonymous class in Spock test causing error
- #334 - Compatibility with recent TestNG releases
- #335 - Incorrect coverage with powermock
Note - as of this release the maven plugin will automatically determine which classes to mutate instead of assuming that the package names match the group id.
- #269 - Not possible to break build on 1 mutant
- #271 - Include method descriptor in line coverage
- #170 - Exclusion by JUnit runner
- #291 - Handle empty excludes
- #287 - Check class hierarchy for annotations
- #289 - Option to supply classpath from file
- #260 - Initial support for mutating Kotlin code
- #264 - Support for yatspec runner (thanks @theangrydev)
- Break build when more than
maxSurvivingmutants survive
- #132 - Allow analysis of only files touched in last commit (thanks Tomasz Luch)
- #239 - Provide a shortcut to set history files via maven
- #240 - Support for regexes (thanks sebi-hgdata)
- #243 - Use ephemeral ports to communicate with minions
- #196 - Raise minimum java version to 1.6
- #231 - Process hangs
- #10 - Add maven report goal (thanks jasonmfehr)
- #184 - Remove undocumented project file feature
- #219 - Performance improvement for report generation (thanks tobiasbaum)
- #190 - Allow custom properties for plugins
Note this release contains a known issue (#231). Please upgrade.
- Fix for #148 - Stackoverflow with TestNG data providers when using JMockit
- Fix for #56 - Not reporting junit incompatibilities
- Fix for #174 - Invalid linecoverage.xml with static initializers
- Fix for #183 - Can't run GWTMockito tests
- Fix for #179 - Broken
includeLaunchClasspath=falseon Windows - #173 - Read exclusions and groups from maven surefire config
- #157 - Support maven -DskipTests flag (thanks lkwg82)
- Fix for #163 - Should not include test tree in coverage threshold
- #166 - Allow classpath exclusions for maven plugin (thanks TomRK1089)
- #155 - Restore Java 5 compatibility
- Fix for #148 - Issue with JMockit + TestNG (thanks estekhin and KyleRogers)
- Fix for #158 - Tests incorrectly excluded from mutants
- Fix for #153 - SCM plugin broken for maven 2
- Fix for #152 - Does not work with IBM jdk
- Fix for #150 - line coverage under reported
- Block based coverage (fixes 79/131)
- End support for running on Java 5 (java 5 bytecode still supported)
- Skip flag for maven modules (#106)
- Stop declaring TestNG as a dependency
- New parameter propagation mutator (thanks UrsMetz)
- Change scheme for identifying mutants (see hcoles#125)
- Support alternate test apis via plugin system
- Report error when supplied mutator name does not match (thanks artspb)
- Report exit codes from coverage child process (thanks KyleRogers)
- Treat JUnit tests with ClassRule annotation as one unit (thanks devmop)
Please note that any stored history files or sonar results are invalidated by this release.
- Switch version numbering scheme
- Upgrade to ASM 5.0.2
- Fix for #114 - fails to run for java 8 when -parameters flag is set
- #99 Support additionalClasspathElements property in maven plugin (thanks artspb)
- #98 Do not mutate java 7 try with resources (thanks @artspb)
- #109 extended remove conditional mutator (thanks @vrthra)
- Move to Github
- Upgrade of ASM to support Java 8 bytecode (thanks to "iirekm")
- Partial support for JUnit categories (thanks to "chrisr")
- New Remove Increments Mutator (thanks to Rahul Gopinath)
- Minor logging improvements (thanks to Kyle Rogers aka Stephan Penndorf)
- Fix for #92 - broken maven 2 support
- Fix for #75 - incorrectly ignored tests in classes with both @Ignore and @BeforeClass / @AfterClass
- restores java 7 compatibility
- new remove conditionals mutator
- support for mutating static initializers with TestNG
- properly isolate classpaths when running via Ant
- break builds on coverage threshold
- allow JVM to be specified
- support user defined test selection strategies
- support user defined output format
- support user defined test prioritisation
- fix for issue blocking usage with Robolectric
Note, setup for Ant based projects changes in this release. See ant setup for details on usage.
- Maven 2 compatibility restored
- Much faster line coverage calculation
- Fix for #78 - Error when PowerMockito test stores mock as member
This release also changes a number of internal implementation details, some of which may be of interest/importance to those maintaining tools that integrate with PIT.
Mutations are now scoped internally as described in https://groups.google.com/forum/#!topic/pitusers/E0-3QZuMYjE
A new class (org.pitest.mutationtest.tooling.EntryPoint) has been introduced that removes some of the duplication that existed in the various ways of launching mutation analysis.
- Support for parametrized Spock tests
- Support for JUnitParams tests
- Fix for #73 - JUnit parameterised tests calling mutee during setup failing during mutation phase
- Fix to #63 - ant task fails when empty options supplied
- Ability to override maven options from command line
- Ability to fail a build if it does not achieve a given mutation score
- Performance improvement when tests use @BeforeClass or @AfterClass annotations
- Slightly improved scheduling over multiple threads
- Improved maven multi project support
- Integration with source control for maven users
- Incremental analysis (--historyInputLocation and --historyOutputLocation)
- Inlined code detection turned on by default
- Quieter loggging by default
- Improved Java 7 support
- Upgrade of ASM from 3.3 to 4
- Fix for concurrency issues during coverage collection
- Fix for #53 - problems with snapshot junit versions
- Fix for #59 - duplicate dependencies set via maven
- Inlined finally block detection (--detectInlinedCode)
- New experimental switch statement mutator (contributed by Chris Rimmer)
- Do not mutate Groovy classes
- Fix for #33 - set user.dir to match surefire
- Fix for #43 - optionally suppress timestamped folders (--timestampedReports=true/false)
- Fix for #44 - concurrent modification exception when gathering coverage
- Fix for #46 - incorrect setting of flags by ant task
- Smaller memory footprint for main process
- Faster coverage gathering for large codebases
- Faster classpath scanning for large codebases
- Support for JUnit 3 suite methods
- Fixes for incorrect detection of JUnit 3 tests
Known issue - Fix for #33 may not resolve issue for maven 2 users.
Detection of Groovy code has not yet been tested with Groovy 2 which may generate substantially different byte code to earlier versions.
- Much prettier reports
- Now avoids mutating assert statements
- Removed inScopeClasses option - use targetClasses and targetTests instead
- Fix for 100% CPU usage when child JVM crashes
- Fix for #35 #38 - experimental member variable mutator now corrects stack
- Fix for #39 - order of classpath elements now maintained when running from maven
Upgrading users may need to modify their build due to removal of the inScopeClasses parameter
- Ant support
- New experimental mutator for member variables
- Fix for #12 #27 - no longer hangs when code under test launches non daemon threads
- Fix for #26 - now warns when no test library found on classpath
- Fix for #30 - now errors if mutated classes have no line or source debug
- Fix for #32 - now correctly handles of JUnit assumptions
Known issue - The new member variable mutator may cause errors in synchronized errors. The mutator is however disabled by default, and the generated errors are correctly handled by PIT.
- TestNG support (experimental)
- Fix for issue where mutations in nested classes not isolated from each other
- Fix for broken classpath isolation for projects using xstream
- Improved handling of JUnit parametrized tests
- Ability to limit mutations to specific classpath roots (--mutableCodePaths)
- Ability to add non launch classpath roots (--classPath) (experimental)
- Read configuration values from XML (experimental)
- Option to not throw error when no mutations found
- Consistent ordering of classes in HTML report
- Statistics written to console
- Classes no longer loaded during initial classpath scanning
- New syntax to easily enable all mutation operations
- JMockit support
- Option to output results in XML or CSV
- Fix for #11
- Improved INLINE_CONSTS mutator
- Fix for issue 7 - source files not located
- Upgrade of Xstream to 1.4.1 to enable OpenJDK 7 support
- Fix for #5 - corruption of newline character in child processes
- Ability to set child process launch arguments
- Significant performance improvements
- Support for powermock via both classloader (requires PowerMockIgnore annotation) and java agent
- Minor error reporting and usability improvements
- Fix for major defect around dependency analysis
- PIT dependencies no longer placed on classpath when running via maven
- Support for excluding certain classes or tests
- Support for verbose logging
- Limit number of mutations per class
- Upgrade xstream to 1.3.1
- Make available from maven central
- Built in enum methods now excluded from mutation
- Fixed bug around reporting of untested classes
- Support for excluding tests greater than a certain distance from class
- Support for excluding methods from mutation analysis
- Performance improvements
- Removed support for launching mutation reports from JUnit runner
- First public release
Please consult our issue management rules before creating or working on issues.
Pitest is mainly the work of me but has benefited from contributions from many others.
Notable contributions not visible here as they were made before this code was migrated to github include
- Nicolas Rusconi - Ant Task
- Struan Kerr-Liddell - Improvements to html report
- Stephan Pendorf - Multiple improvments including improved mutators
Although PIT does not incorporate any code from the Jumble project (http://jumble.sourceforge.net/), the Jumble codebase was used as a guide when developing some aspects of PIT.
The codebase is checked up on in a few places that give slower feedback than the github hooks.