containers/podman-desktop-e2e

Junit reporter reports false positive test cases when there is a failure in before test hook

odockal opened this issue · 0 comments

REf. run: https://github.com/containers/podman-desktop-e2e/actions/runs/8236938501/job/22524543429

Test output:

  FAIL  tests/src/yaml-smoke.spec.ts [ tests/src/yaml-smoke.spec.ts ]
Error: Podman Desktop could not be started correctly with error: Error: electron.launch: Electron failed to install correctly, please delete node_modules/electron and try installing again
 ❯ PodmanDesktopRunner.start tests/src/runner/podman-desktop-runner.ts:70:13
     68|     } catch (err) {
     69|       console.log(`Caught exception in startup: ${err}`);
     70|       throw Error(`Podman Desktop could not be started correctly with …
       |             ^
     71|     }
     72| 
 ❯ tests/src/yaml-smoke.spec.ts:38:10

Here is the visible part of the junit reporter:
Screenshot_20240312_104957

Summary is even more confusing, showing that most cases passed. The truth is that the testing app. did not make it to the even start up. BeforeAll hook for each test has an error.

Screenshot_20240312_105150