Cost of ownership of PiTest needed
Opened this issue · 4 comments
Some indication is needed of how long mutation testing could take for a particular testbase. I'd suggest something like:
"PiTest will take about 4x longer than your regular unit test suite duration."
A suggestion too as to how to operationalize it in CI would be cool. Like so:
"Don't put PiTest in your primary CI build. Instead make another job that runs at some frequency (say hourly) that tests everything at HEAD revision then. Take the rollup number from the log, and track that one on a timeline. Perhaps using the Plot plugin https://wiki.jenkins.io/display/JENKINS/Plot+Plugin"
"PiTest will take about 4x longer than your regular unit test suite duration."
I don't think giving numbers here is reliable. The time for mutation testing depends on your hardware and whether you use parallel unit tests already.
Maybe one could add such a warning but couldn't find a good place so the warning is neither too demotivating nor overlooked.
How about ...
"PiTest will take about 1.5x to 20x longer than your regular unit test suite duration."
I agree that it could be discouraging. Maybe we should just close this issue - people can run PiTest where they like.
How about ...
"PiTest will take about 1.5x to 20x longer than your regular unit test suite duration."
I agree that it could be discouraging. Maybe we should just close this issue - people can run PiTest where they like.
How about ...
"PiTest will take about 1.5x to 20x longer than your regular unit test suite duration."
I agree that it could be discouraging. Maybe we should just close this issue - people can run PiTest where they like.