jest coverage misses subprocesses
mnmkng opened this issue · 14 comments
Hello,
some time ago I opened an issue with nyc, which is a CLI interface to istanbul, the instrumenter used by jest #729. It is related to getting incorrect coverage out of jest (+ nyc) combination. I'll briefly summarize and hopefully explain why I'm also opening the issue with Jest.
We use Jest as our test platform and would like to also use its --coverage feature. However, even after thorough testing, we weren't able to get coverage output for files executed in a forked subprocess (we spawn servers and hit them with requests to test their API). Parent files would report coverage correctly, while no coverage would be output for child files.
So we introduced nyc to the mix. Surprisingly, nyc would output correct coverage for the child process files, but no coverage for the parent files. Gentlemen at nyc suggested running tests twice and merging the results, but that has proven difficult for our use case.
Expected behavior
Jest should correctly output coverage for forked processes.
Observed behavior
Jest outputs no coverage for forked processes.
Repo that reproduces the issue
https://github.com/mnmkng/nyc-jest-issue
The nice gentlemen at nyc offered coordination in resolution of this issue, so feel free to refer back to the original nyc issue #729 for more details and discussion.
Thank you for your time.
Duplicate of #3190, but I'm fine with a tracking issue, it feels like something that should work.
Huge bummer that the original issue and this one have languished for so long unresolved. Built-in coverage in jest was money in the bank, and was really hoping nyc's method for tracking coverage in subprocesses would've been migrated here. At present the only workaround is to use nyc and pretty much take jest out of the coverage game. Unfortunately that results in some pretty fugly CI setup.
Help very much welcome in landing support for it 🙂
I'm not sure how nyc handles it, so just digging into that and listing what would need to be done in Jest would be awesome
@SimenB looks like nyc is using spawn-wrap (wrapper for your process and automatically wraps child processes), and require hooks to transform code for coverage during runtime.
From what I can tell, it looks like Jest is transforming everything first, then running. So I think Jest would need to switch to the hook-based approach to achieve this
Jest injects its own require which does JIT transpilation and instrumentation. Not sure how that plays with wrapping the spawned child
Hmm, maybe the only work needed is adding spawn-wrap to apply the require override to all processes, including spawned. I'm not very familiar with the Jest codebase but can look into this if it's not a top priority issue for the team
As maintainer of nyc I have to advise against using spawn-wrap, there are edge cases where it fails especially with some shells in Windows. nyc 15 will soon be released and will instead use node-preload by default. My understanding is the next semver-major of jest will require node.js 8 so you will be able to use that module.
Any news?
This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.
activity
This would be hugely helpful for functional test suites
any news on this?
This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.
comment