Flank should record fatal exception outcome and or details from a matrix execution
Opened this issue · 1 comments
AaronMT commented
Hello,
I've noticed that when a Fatal exception occurs on Firebase Test Lab (e.g,
These are simply recorded in the matrix_ids.json
file as {"outcome": "failure"}
. For the purposes of daily reporting on test executions, I would love if Flank were to capture any other keys indicative of crashes when they happen. I know this is possible through the Cloud Tool Results API, but I would like if Flank could do that, as we simply parse artifacts. Even if it's a simple boolean from FailureDetail
({"crashed": true}
) that would be helpful.
https://firebase.google.com/docs/test-lab/reference/toolresults/rest/v1beta3/Outcome#FailureDetail
https://firebase.google.com/docs/test-lab/reference/toolresults/rest/v1beta3/Outcome