boxed/mutmut

--CI parameter

enalcopolo opened this issue · 5 comments

enalcopolo commented

Where can I find information about the --CI parameter?
I want to integrate mutmut to a jenkins pipeline
Thanks

boxed commented

I would argue against it. Mutation testing makes no sense in CI in my opinion. It's a tool you use to write tests.

enalcopolo commented

For me they help a lot in improving the quality of the code, therefore to have a product to deploy, they can give you a score (killed mutants/total mutants) that can serve as a metric if you compare it with a threshold in a CI. I come back and ask

Why is the --CI parameter in mutmut, what is it for?

Thanks

boxed commented

@maxkahan do you want to answer this? Have you actually used this?

As the original author of mutmut I think this flag is a bad idea, a waste of time and electricity. 🤷

enalcopolo commented

Don't worry, I already understood what CI is for, in my pipeline, when I run
mutmut run
Always returned status code 6 and the build failed, but after running
mutmut run --CI
works perfect
I will manage to get a score with the results.

Thanks

maxkahan commented

Hi @boxed yes I have used this, see for example this GitHub Actions workflow. I don't use it as part of a pipeline, but to validate my tests are sensible when I release larger features (I'm aware that's not the same way you use it).

Also, I've given several conference talks introducing developers to mutation testing, and every time I get questions about running in a CI system from Python developers. Now I have a sensible answer using mutmut with the caveat that you don't want to run mutation tests as part of your release pipeline. There are definitely advantages to being able to run lengthy processes on other machines.

So I think it should stay! It's your project though, so do as you see fit 🙂

@enalcopolo yes that's exactly why I wrote it, glad it's helpful to you too! 🙂