TestAnything/testanything.github.io

Inclusion of Test::More-style subtests

Eckankar opened this issue · 2 comments

Eckankar commented

I propose that Test::More-style subtests be mentioned in some manner in the specification.

That is, for an example:

1..3
ok 1 - First test
    # Subtest: An example subtest
    1..2
    ok 1 - This is a subtest
    ok 2 - So is this
ok 2 - An example subtest
ok 3 - Third test

Regular Test::Simple/Test::More/... tests usually output their diagnostic messages after a failed test. I.e.:

1..5
ok 1 - new() works
ok 2 - Title() get
ok 3 - Director() get
not ok 4 - Rating() get
#   Failed test 'Rating() get'
#   in t/film.t at line 14.
ok 5 - NumExplodingSheep() get
# Looks like you failed 1 tests of 5

I.e., if a TAP runner wishes to report debug information about the failed test to the programmer, it would usually report what follows the given test. However, for subtests, reporting the output of the subtest itself in addition to the final end result seems like it'd provide the developer with better output than simply reporting what is output after the failed test.

While a TAP-parser can function perfectly fine not knowing about these, someone writing a parser would probably want to know about this, so that they could provide proper support.

mblayman commented

You're in luck! They will be. 😄 The next version of the spec that is under review includes sub-tests. See https://github.com/TestAnything/testanything.github.io/pull/36/files#diff-4501a9b31c6633d6d7ebea185c8f0572R66.

  • 👍2