This provides two scripts that read Xcode unit test output and print out XML suitable for parsing by Jenkins's JUnit plugin.
- plist2junit, for reading TestSummaries.plist from Xcode <= 10
- xcresult2junit, for reading .xcresult directories from Xcode 11+
- extract_xccovarchive, for working around this bug if you use SonarQube for code coverage
The most popular tools for doing this (xcpretty and trainer) are surprisingly bad at the one thing they are supposed to do: generate test reports.
- xcpretty tries to parse Xcode's output as it runs the tests rather than parsing the nicely formatted summary file. Because tests can run in parallel or output log messages that interrupt the output, this resulted in xcpretty missing 25% of our test results in its report.
- trainer outputs redundant timing summaries. This confuses the Jenkins plugin, causing it to report incorrect aggregate test times
- trainer incorrectly groups tests (it doesn't translate test targets into packages), so if you organize your tests in Xcode into separate targets that all gets wiped away in the report
- trainer depends on fastlane, which is a massive dependency for such a simple task
- Both xcpretty and trainer give up if any tests encountered errors, even if other tests were run successfully. Meaning you get no report, rather than a partial one
- Depends only on standard mac OS utilities (ruby, plutil, xcrun, no gems)
- Simple to run (one file in, prints to stdout)
- Fast
- plist2junit: 0.22s for ~540 tests
- xcresult2junit: 1.5s for ~700 tests
- Includes test errors in output
- Nicely organizes separate testing targets