aghassemi/tap-xunit

ava todos showing up as failures

Closed this issue · 9 comments

xjamundx commented

Given a test that looks like this:

var test = require('ava')
test('normal test', t => t.pass())
test.todo('todo test')

ava's tap output looks like this:

TAP version 13
# normal test
ok 1 - normal test
# todo test
not ok 2 - todo test # TODO

1..1
# tests 1
# pass 1
# fail 0

that gets converted into xunit output that looks like this:

<?xml version="1.0"?>
<testsuites>
  <testsuite name="normal test" tests="1" failures="0" errors="0">
    <testcase name="#1 normal test"/>
  </testsuite>
  <testsuite name="todo test" tests="1" failures="1" errors="0">
    <testcase name="#2 todo test">
      <failure/>
    </testcase>
  </testsuite>
</testsuites>

I don't think the todos should be failures IMO, but I'm not sure if that's on AVA or this module

aghassemi commented

Agreed. We can treat # TODO and # SKIP the same for the XML output.

  • 👍2
adambiggs commented

Any updates? I just ran into this issue on CircleCI.

aghassemi commented

@adambiggs fixed and published as 1.6.0 now # SKIP and # TODO are treated the same in the XML output.

  • 🎉1
adambiggs commented

Wow that was fast! Thanks @aghassemi!

adambiggs commented

I just tested this and there's one more problem... The XML now looks correct, but the command still exits with a non-zero code (1) so my CircleCI builds are still failing.

willm commented

I think the issue is with this line https://github.com/aghassemi/tap-xunit/blob/master/lib/converter.js#L58 We should check whether assert is not ok and it's not a todo or skip. I'll try to do this when I get a minute.

aghassemi commented

Done and released as 1.7.0. We exit with code 0 now if all failures are skipped or todo.

adambiggs commented

Awesome, thanks again @aghassemi. My builds are now passing 👍

  • 👍1