avajs/ava

Implement .because() modifier for irregular tests

novemberborn opened this issue · 14 comments

Based on discussion with @tommy-mitchell in #3224:

test.failing('does foo', t => {
  // ..
}).because('waiting on some/repo#123')

Should output:

✔ [expected fail] does foo
  - waiting on some/repo#123

This requires changes to https://github.com/avajs/ava/blob/main/lib/create-chain.js so that a chainable object is returned for test.failing(), test.skip() etc. Here we then need to add a because function. The reason should make its way to the reporter where it can be printed beneath the test title.

This should be reflected in the type definitions and documentation. Once we've made some progress on this, work needs to be done with https://github.com/avajs/eslint-plugin-ava to support this new modifier.

It should be a test failure if because() is called with anything but a non-empty string.

As commented in #3224, I also think .because() should come last.

As commented in #3224, I also think .because() should come last.

That'd be easy to miss with a large test implementation / function body.

It's also a new thing to implement, test() doesn't return anything at the moment, and to make trailers work we'd have to change that, todo, failing, skip, and only. Adding a test.because() function which continues the chain is easier.

This could have an ESLint rule as well, requiring a .because() description

test() doesn't return anything at the moment, and to make trailers work we'd have to change that, todo, failing, skip, and only

Both #2979 and #2980 propose trailing modifiers on test(), so all three would benefit from it.

That's fair — it's worth exploring but it's a chunk of work. Will amend the issue description.

@novemberborn If no one has started working on the implementation, I'd like to take a crack at this.

Turned out to more complex than I anticipated. But I am still working on this.

@novemberborn I have made some test and trying to test it. But looks like when I run npm run test it calls the old version of create-chain.js instead of the new version that I have made edits to. Is this an issue you have ever faced ??

npm test calls npx test-ava which uses a different copy of AVA to test AVA with. But these tests are meant to be integration style, so you'd set up a fixture which uses the new syntax and then you observe the behavior.

I haven't worked on this for a while now. And I am facing troubles setting up the the test, so that the tests use the my changes. If some else has more experience doing I'd like some help, or they can take up the this issue entirely.

@adiSuper94 , I can take up this issue if you want. In addition, what troubles were you having setting up the tests?