vegaprotocol/jenkins-shared-library

Jenkins job didn't archive artifacts and report failed tests to parent

Closed this issue · 0 comments

See thread in slack:
https://vegaprotocol.slack.com/archives/CE5L3T60J/p1695033273842589

I saw one this morning, and @daniel1302 apparently saw a few instances last week where one of the System Test child jobs seemed to fail pretty hard for non-advertised reasons, not do any of the archival steps, and just report "Fail" to the parent job rather than specific tests and doing all our standard failure cleanup. In the instance I saw, the last Jenkins messages you see are actually all talking about what seems to be a successful run of the snapshot tests, so it's really confusing that the job bailed.

End of logging:

[2023-09-15T17:53:58.250Z] Restoring from height 132...
[2023-09-15T17:53:58.250Z] Finished: replay from height 132 has reached height 229 > 182.
[2023-09-15T17:53:58.250Z] Restoring from height 132 successful.
[2023-09-15T17:53:58.250Z] 
[2023-09-15T17:53:58.250Z] Restoring from height 110...
[2023-09-15T17:53:58.250Z] Finished: replay from height 110 has reached height 266 > 160.
[2023-09-15T17:53:58.250Z] Restoring from height 110 successful.
[2023-09-15T17:53:58.250Z] 
[2023-09-15T17:53:58.250Z] Restoring from height 88...
[2023-09-15T17:53:58.250Z] Finished: replay from height 88 has reached height 231 > 138.
[2023-09-15T17:53:58.250Z] Restoring from height 88 successful.
[2023-09-15T17:53:58.250Z] 
[2023-09-15T17:53:58.250Z] Restoring from height 66...
[2023-09-15T17:53:58.250Z] Finished: replay from height 66 has reached height 257 > 116.
[2023-09-15T17:53:58.250Z] Restoring from height 66 successful.
[2023-09-15T17:53:58.250Z] 
[2023-09-15T17:53:58.250Z] Restoring from height 44...
[2023-09-15T17:53:58.250Z] Finished: replay from height 44 has reached height 197 > 94.
[2023-09-15T17:53:58.250Z] Restoring from height 44 successful.
[2023-09-15T17:53:58.250Z] 
[2023-09-15T17:53:58.250Z] Restoring from height 22...
[2023-09-15T17:53:58.250Z] Finished: replay from height 22 has reached height 240 > 72.
[2023-09-15T17:53:58.250Z] Restoring from height 22 successful.
[2023-09-15T17:53:58.250Z] 
[2023-09-15T17:53:58.250Z] percentage of working snapshots: 100.0% (328/328)
[2023-09-15T17:53:58.250Z] block heights of failing snapshots
[2023-09-15T17:53:58.250Z] []
[2023-09-15T17:53:58.250Z] ........................................................................................................................................................................................................................................................................................................................................
[2023-09-15T17:54:56.828Z] Running in /jenkins/workspace/common/system-tests-wrapper/system-tests

consoleText.zip