xparq/Space_Test

RUN/SHELL error codes get masked by subsequent EXPECT (or other) statements

Opened this issue · 0 comments

xparq commented

Both RUN and SHELL meticulously return any failure codes -- but there's nothing checking them... They are just lines in the test script being sourced,.. nobody's magically also "calling" them...

However...

  • RUN has its own exit-code follow-up logic, to trigger forced fail, so there's at least that, but it still can't trigger ABORT...
  • SHELL doesn't even have that.
  • Anyway, they also save (append) the exit codes to a retval file, so that could still be useful!
  • But anyway, add an ABORTED flag that they can set, and then also check on their next turn.
    ...Umm, but there's no reliable way to recognize and abortive error! :-o The current heuristic in run_test checks for non-empty stderr dump, and that's it... Maybe it can be tightened with EXPECT_ERROR 0, or even ignore/warn? But that's still something a normally failing case could also do... Well, but such test cases could redirect their "kinda expected" stderr to null, or a custom file, if important! That could help. If their stderr was not expected, they can get an ABORT, and then they can refine their error policy if that's not what's expected.