Bug report - Stubbed result / todo:true is not registering.

[ORESoftware]

By bug report I mean "potential bug" report

I am investigating to see why a stubbed test is not being marked as "todo".

I have this output result:

ok 0 bbbbb # STUBBED -
ok 0 zzzzzz # SKIP -
ok 1 mmmm

testpoint: Result { ok: true, id: 0, name: 'bbbbb # STUBBED -' }
testpoint: Result { ok: true, id: 0, skip: '-', name: 'zzzzzz' }
testpoint: Result { ok: true, id: 1, name: 'mmmm' }

So there are two problems I see - the first problem is that two test cases have the same id, that seems wrong. I have 3 test cases and 3 individual Result objects, so having dupe ids seems like a legit problem.

The second problem I see is that the # STUBBED test does not get marked as "todo", and so no stubbed test results are getting outputted.

Any idea what might be wrong? Seems like a legit problem, thanks.

[ORESoftware]

Looking to bump this issue.

I fixed a bug with the library I have and now the ids are incremented properly, but I still see same problem:

testpoint: Result { ok: true, id: 1, name: 'is normal' }
         ✔  'is normal' 

testpoint: Result { ok: true, id: 2, skip: '-', name: 'is skipped' }
         ↪  (skipped) 'foo bar'

testpoint: Result { ok: true, id: 3, name: 'is stubbed # STUBBED -' }
         ✔  'is stubbed # STUBBED -' 

I believe that for testpoint with id 3, that the # STUBBED - bit should get parsed out, and the testpoint object should be marked as todo:true.

If there's anything clearly amiss, given the info provided, please let me know. Otherwise, I will try to figure out what other details I can provide.

[isaacs]

Only # SKIP and # TODO are defined by the TAP specification. # time=[0-9]+ms is a nonstandard addition that this library supports. # STUBBED isn't a thing, sorry. So that just gets lumped into the name of the assertion.

The IDs are the same because they're the same in the TAP stream. This is a parser, it's telling you what it parsed. An argument could be made that it ought to get upset about re-used IDs within a TAP stream, but if so, that's a new feature request. It's a rare enough occurrence, and relatively harmless to allow, so unless it causes some problems elsewhere, I'd be inclined to just keep ignoring improperly reused test point IDs.