callstack/reassure

[BUG] Node's fetch fails because of the `--jitless` flag

Shywim opened this issue · 3 comments

Shywim commented

Describe the bug
I am unable to run tests which use fetch because it is implemented using WebAssembly, tus it fails with WebAssembly is not defined message.

Source: https://www.reddit.com/r/node/comments/16xnr7s/any_idea_why_fetch_is_implemented_using/

To Reproduce
I forked the examples repo: Shywim/reassure-examples@8919603

  ● Other Component 20

    TypeError: fetch failed



    Cause:
    ReferenceError: WebAssembly is not defined

      10 |
      11 |   const handlePress = () => {
    > 12 |     fetch('https://example.com').then(() => {
         |     ^
      13 |       setCount((c) => c + 1);
      14 |     });
      15 |   };

      at fetch (src/OtherTest.perf-test.tsx:12:5)

Expected behavior
This should run fetch without problem.

I saw the discussion in #421 and I think we should have an option so we can accept the JIT problematic

Additional context

  • Node version: 20.9.0

@Shywim You are right; the --jitless flag clashes with WASM support which is required used by fetch.

Just for clarity, I would recommend not making real network calls in your tests, as they make the tests less predictable and extend their execution time. I assume that you might be using something like MSW to actually mock the fetch though.

The reason for --jitless flag is that we want to remove the usage of optimizing compiler that would negatively impact the stability of measurements through two factors:

  1. VM pausing for non-optimized, optimized compilation
  2. Changed execution characteristics due to usage of compiled code vs previously interpreted code

Each of these would make the execution times much less stable.

I've investigated some alternatives for that, and there is --no-opt --no-sparkplug option that should turn off Turbo Fan optimizing compiler (--no-opt) and sparkplug non-optimizing compiler. In order to use these instead of --jitless, we would have to enable them either conditionally so you could try them or conduct some testing on their impact on tests for enabling them by default.

@Shywim pls check if the --enable-wasm option to reassure resolves the issue for you.

Shywim commented

Works nicely with the --enable-wasm flag, thank you!