emarteca/npm-filter

Verbose mode depends on running tests without any instrumentation

Closed this issue · 1 comments

When running tests in "verbose" mode, I am modifying the command line for Jest so that the tests run in band. This is necessary in some execution environments for the tests to run at all (and not just hang). However, in verbose mode, NPM filter runs all of the tests twice: once without the instrumented command line and once with. Hence, it stalls on these configurations.

I guess that there are a few options here:

  1. Add the runInBand flag to the non-instrumented test run (is this even possible?)
  2. Save the information that is needed from the non-instrumented test run when building a container with NPMFilter. Then, when the tests are re-run in the container under different options, there is no need to do the initial non-instrumented test run.

I think that option (2) makes the most sense from a performance perspective - running all of the tests twice to collect the data once seems inefficient. I think that this is now off of my critical path for the next week or two - we will be missing some data as a result of this, but only for ~50 project-resource configurations.

Good catch! You're right -- the uninstrumented test command was running every time, and then if running in verbose mode and there was an instrumented version of the command it would be run after.

Fixed it -- the code at commit a875faa only runs the instrumented version of a test command if running in verbose mode. Hopefully this fixes the issue!