CICD is not failing when listener is failing
Tattoo opened this issue · 0 comments
Tattoo commented
The CICD currently relies on Robot Framework exit code to know if the test execution failed or not. However, in cases where the Oxygen listener fails, but the test execution otherwise passes, RF will return 0 and the CICD passes.
For example:
==============================================================================
Atest | PASS |
10 critical tests, 10 passed, 0 failed
10 tests total, 10 passed, 0 failed
==============================================================================
[ ERROR ] Calling method 'output_file' of listener 'oxygen.listener' failed: MismatchArgumentException: parse_results expects 3 arguments but got 1
Output: /[...]/output.xml
Log: /[...]/log.html
Report: /[...]/report.html
[python38][robotframework==3.2] Exited with 0
Since Oxygen is, in part, a listener, we need CICD to detect that listener has failed during the test execution