Optional more granular reporting for JUnit reports
Closed this issue · 2 comments
What would you like to be added:
#509 changed the format of the JUnit reports to be more granular. A statement at the end of the Caveats section says
If needed, I could add a knob to make this high-granularity opt-in.in the end
This request is about introducing such knob to allow people to use the previous, less granular reporting format.
Why is this needed:
This potential need was already highlighted in the commit which changed the format, as highlighted above. I'd like to add that in the project I contribute, OpenStack Kubernetes Operators, we are starting to collect the results of KUTTL tests in a consistent way for internal reporting. The way each operator structured its tests is to have to have a basic mapping of 1-1 between tests and features. We don't want nor need to mark each step for reporting. So instead of having to post-process the JUnitXML file, it would be nice if kuttl could produce also (with some flag) the report in the old format.
@tosky would you be able to verify that the knob added in #581 works as expected, by building the binary manually from tip of main?
I did check that with test-level granularity outputs from an example suite looks the same as it used to before #509 , but it wouldn't hurt to have a confirmation.
If that works, I can cut a release next week.
Thank you so much for working on it! I've just tested the kuttl CLI from main and I confirm it new behavior when you use the new switch (either through the config file or through the command line parameter).