Document common pitfalls
jamestalmage opened this issue · 33 comments
There are some points that continue to confuse people (especially as it relates to our async behavior).
One in particular is using closure scope variables to try and pass data between beforeEach and individual tests (see #402).
I think we need to create some documentation that addresses each of these types misconceptions so we can quickly link to them. They can be in the readme if they are concise, but maybe separate documents if they require lengthy explanations.
If anyone can think of other common downfalls / misconceptions / misuses - go ahead and add them to this discussion.
I think this is a really great idea. Also, I'd like to appreciate the relevant issue number.
Good idea!
Pitfalls:
- Using global variables and forgetting that tests run concurrently.
console.lognot logging where the test result is printed (I hope we can fix #392, but we should document until we do)
Actually, What if we made this a wiki page and linked to it from the README? That way others could contribute?
What if we made this a wiki page and linked to it from the README?
No. I want to have quality control on the docs. I don't want random strangers to be able to just freely modify the docs. Also: https://plus.google.com/+sindresorhus/posts/QSS2du26Mg4
That way others could contribute?
They already can with a pull request.
Another one: if you run AVA in Docker as part of your CI, you need to forward the appropriate environment variables (see https://github.com/watson/is-ci/blob/master/index.js), or configure the verbose reporter. See #751.
Should we make a common_pitfalls.md in the documentation directory with these?
yep
👍
common_pitfalls.md => common-pitfalls.md
Another pitfall I see often is trying to do an async operation in a normal test and not understand why it's not finishing.
Forgetting to return the promise:
test(t => {
fetch().then(data => {
t.is(data, 'foo');
});
});Or should have used a callback test:
test(t => {
fetch((err, data) => {
t.is(data, 'bar');
});
});Now documented here: https://github.com/avajs/ava/blob/master/docs/common-pitfalls.md
Gonna keep this issue open for discovery reasons.
I've seen people trying to pass CLI flags to their tests: $ ava --user-flag This doesn't work as we don't pass flags to the test processes. Instead, users should use environment variables.
Should we also link to eslint-plugin-ava in the common pitfalls section? We can probably push more people to using it.
@jfmengels Good idea! Can you add it?
Sure, will do that later.
What abut source vs files in the AVA package.json options?
@sotojuan Have you encountered anyone getting confused by those?
Just did on an IRC!
Can you copy paste the conversation for context? Like, what was confusing about it.
Ah, sorry I should've just said everything in the first comment. My friend was just confused about the difference between both (what do they do?)—they do look very similar. Maybe not a pitfall but worth a sentence or two in the readme.
@sotojuan Yeah, definitely. If anything is unclear, we should make it clearer. Wanna do a PR improving it?
When import ing a sass file with webpack and ava, the import fails. Any way to fix that?
@cameronroe http://stackoverflow.com/questions/tagged/ava would be a better place to ask this. Include some more info too, like AVA and Webpack config.
Not sure if I should add this here, or file a separate issue, but one pitfall I ran into:
If your tests do anything memory intensive, you might end up hitting OOM process killers in CI environments because ava runs them in parallel. See this build as an example.
@sindresorhus okay, so no one is alive on Stack overflow.. Any ideas on what's up with ava and webpack? Docs anywhere on the subject?
@davewasmer Open a new issue. I think we simply need to extend https://github.com/avajs/ava/blob/master/docs/common-pitfalls.md#ava-and-connected-client-limits
@cameronroe You didn't include the information I recommended. Just a guess, but you need to use https://github.com/istarkov/babel-plugin-webpack-loaders or precompile your component before testing.
Would love to have a pitfall around the parallel nature of unit tests. I can't tell you how many times I've run into Sinon throwing errors about methods already being wrapped in spies. It's just not something I expected to encounter.
Specifically, illustrating the workflow of before, beforeEach, test, afterEach, after and the fact that beforeEach may run multiple times in a row. I realize the answer has been 'make your tests serial' it's just not something I expect to run into. At least illustrating that if you have to mess with singletons or global objects or static properties, that beforeEach can cause issues.
One thing I'm still not clear on: if I want to write my tests in a parallel way, are the tests run concurrently, or should I be putting the global modifications and subsequent cleanup into the test itself? In which case, running things like sinon.sandbox.restore() can't be done in an afterEach()
I have a separate library named testUtils.js. It's an ES6 file, documented by the fact that my package.json includes "type": "module". Ava seems to recognize that it's an ES6 file; however, when I try to do "import {tname} from './testUtils.js'", I get the error message "Must use import to load ES Module", which is really confusing because I AM using import. I think that Ava is mistakenly transpiling my test.js (it should have checked package.json first), converting my import into a require, which then fails. At least that's my best guess.
When debugging a test with WebStorm (v 2020.1.1), the normal test timeout occurs when paused at a breakpoint and the test fails. Can I disable the timeout or configure WebStorm/AVA differently?
@jmamcallister see #2317. The ava debug command disables the timeout, but the recipe needs to be updated with instructions on how to use that.