Undici 6.x - Request/Response/TextEncoder is not defined (Jest) no longer works

Question

Undici 6.x - Request/Response/TextEncoder is not defined (Jest) no longer works

joel-daros opened this issue a year ago · 28 comments

joel-daros commented a year ago

Prerequisites

I confirm my issue is not in the opened issues
I confirm the Frequently Asked Questions didn't contain the answer to my issue

Environment check

I'm using the latest msw version
I'm using Node.js version 18 or higher

Node.js version

v18.19.0

Reproduction repository

http://

Reproduction steps

Install latest version of Undici (6.x)
Try to run tests

Current behavior

The documentation suggestion to create a jest.polyfills.js no longer works in Undici version 6.x

All tests are returning the same error:

 ● Test suite failed to run

   ReferenceError: ReadableStream is not defined

     27 |   Headers: { value: Headers },
     28 |   FormData: { value: FormData },
   > 29 |   Request: { value: Request },
        |                 ^
     30 |   Response: { value: Response },
     31 | });
     32 |

Expected behavior

An update version of jest.polyfills.js

Answer 1 · 2023-12-07T17:20:47.000Z

I had the same issue, but I've solved it with install web-streams-polyfill and modify jest.polyfills.js as bellow:

const { TextDecoder, TextEncoder } = require('node:util')
const { ReadableStream } = require('web-streams-polyfill/ponyfill/es2018')
 
Object.defineProperties(globalThis, {
  ReadableStream: { value: ReadableStream },
  TextDecoder: { value: TextDecoder },
  TextEncoder: { value: TextEncoder },
})
 
const { Blob, File } = require('node:buffer')
const { fetch, Headers, FormData, Request, Response } = require('undici')
 
Object.defineProperties(globalThis, {
  fetch: { value: fetch, writable: true },
  Blob: { value: Blob },
  File: { value: File },
  Headers: { value: Headers },
  FormData: { value: FormData },
  Request: { value: Request },
  Response: { value: Response },
})

Answer 2 · 2023-12-07T19:41:52.000Z

A polyfill of a polyfill? 😄

I hope there is clear solution.

Answer 3 · 2023-12-11T13:03:58.000Z

I know this solution was frowned upon; but it's been working well for me so far, and it does not require Undici.

Answer 4 · 2023-12-11T19:36:45.000Z

@azangru Thanks for the suggestion. I like it better than installing Undici and dealing with incompatibilities with each new version.

This is my jsdom-extended.js file if anyone wants to use the same approach:

const JSDOMEnvironment = require("jest-environment-jsdom").default; // or import JSDOMEnvironment from 'jest-environment-jsdom' if you are using ESM modules

class JSDOMEnvironmentExtended extends JSDOMEnvironment {
  constructor(...args) {
    super(...args);

    this.global.ReadableStream = ReadableStream;
    this.global.TextDecoder = TextDecoder;
    this.global.TextEncoder = TextEncoder;

    this.global.Blob = Blob;
    this.global.File = File;
    this.global.Headers = Headers;
    this.global.FormData = FormData;
    this.global.Request = Request;
    this.global.Response = Response;
    this.global.Request = Request;
    this.global.Response = Response;
    this.global.fetch = fetch;
    this.global.structuredClone = structuredClone;
  }
}

module.exports = JSDOMEnvironmentExtended;

And then in the jest.config.js file:

  testEnvironment: "<rootDir>/src/test-helpers/jsdom-extended.js",

Answer 5 · 2023-12-16T00:36:10.000Z

A polyfill of a polyfill? 😄

I hope there is clear solution.

I've gotten to the point where I don't personally think it's worth it to migrate to v2 if you're using Jest. Part of the amazing thing about this library on v1 is that it just worked. It was a magical DX when you compared it to the alternatives. But now, I'm highly advising anyone using Jest (which many of us are) to just continue using v1. I've followed the official migration docs with various levels of success but never able to reproduce the just works factor.

The library authors have taken a pretty clear stance that they consider Jest antiquated even though it's currently 20x more popular than vitest according to npm downloads. And for the record, I do respect their right to take the library in that direction as the maintainers.

Answer 6 · 2023-12-17T06:26:45.000Z

I just want to chime in that the proposed solution using web-stream-polyfill did not work for me, but the alternate solution extending JSDOMEnvironment did work... sorta.

I had to modify the solution as such:

const JSDOMEnvironment = require('jest-environment-jsdom').default;

class MyJSDOMEnvironment extends JSDOMEnvironment {
  constructor(...args) {
    super(...args);

    this.global.Request = Request;
    this.global.Response = Response;
    this.global.TextEncoder = TextEncoder; // Had to add this
    this.global.TextDecoder = TextDecoder; // Had to add this
    this.global.fetch = fetch;
    this.global.structuredClone = structuredClone;
  }
}

module.exports = MyJSDOMEnvironment;

Even though my tests did pass, I got a billion of these messages in my console:

(node:49550) MaxListenersExceededWarning: Possible EventTarget memory leak detected. 15 abort listeners
added to [AbortSignal]. Use events.setMaxListeners() to increase limit

As such, I'm gonna stick with using the previous major version of undici until either a more robust solution is presented, or I migrate to vitest.

Answer 7 · 2024-01-05T08:39:37.000Z

@azangru Thanks for the suggestion. I like it better than installing Undici and dealing with incompatibilities with each new version.

This is my jsdom-extended.js file if anyone wants to use the same approach:

const JSDOMEnvironment = require("jest-environment-jsdom").default; // or import JSDOMEnvironment from 'jest-environment-jsdom' if you are using ESM modules

class JSDOMEnvironmentExtended extends JSDOMEnvironment {
  constructor(...args) {
    super(...args);

    this.global.ReadableStream = ReadableStream;
    this.global.TextDecoder = TextDecoder;
    this.global.TextEncoder = TextEncoder;

    this.global.Blob = Blob;
    this.global.File = File;
    this.global.Headers = Headers;
    this.global.FormData = FormData;
    this.global.Request = Request;
    this.global.Response = Response;
    this.global.Request = Request;
    this.global.Response = Response;
    this.global.fetch = fetch;
    this.global.structuredClone = structuredClone;
  }
}

module.exports = JSDOMEnvironmentExtended;

And then in the jest.config.js file:

  testEnvironment: "<rootDir>/src/test-helpers/jsdom-extended.js",

i tried import this to setupTests.js and still doesnt work

Answer 8 · 2024-01-05T16:02:29.000Z

ReadableStream: { value: ReadableStream },

ReadableStream was important to make work my local tests with Jest, I believe it should be included in the documentation at this part: https://mswjs.io/docs/migrations/1.x-to-2.x#requestresponsetextencoder-is-not-defined-jest

Answer 9 · 2024-01-10T17:11:21.000Z

I solve it this way

// jest.polyfills.js

const { TextDecoder, TextEncoder, ReadableStream } = require('node:util');

Object.defineProperties(globalThis, {
  TextDecoder: { value: TextDecoder },
  TextEncoder: { value: TextEncoder },
  ReadableStream: { value: ReadableStream },
});

const { Blob, File } = require('node:buffer');
const { fetch, Headers, FormData, Request, Response } = require('undici');

Object.defineProperties(globalThis, {
  fetch: { value: fetch, writable: true },
  Blob: { value: Blob },
  File: { value: File },
  Headers: { value: Headers },
  FormData: { value: FormData },
  Request: { value: Request },
  Response: { value: Response },
});

Answer 10 · 2024-01-10T19:38:17.000Z

@azangru Thanks for the suggestion. I like it better than installing Undici and dealing with incompatibilities with each new version.
This is my jsdom-extended.js file if anyone wants to use the same approach:
const JSDOMEnvironment = require("jest-environment-jsdom").default; // or import JSDOMEnvironment from 'jest-environment-jsdom' if you are using ESM modules

class JSDOMEnvironmentExtended extends JSDOMEnvironment {
constructor(...args) {
super(...args);
this.global.ReadableStream = ReadableStream;
this.global.TextDecoder = TextDecoder;
this.global.TextEncoder = TextEncoder;

this.global.Blob = Blob;
this.global.File = File;
this.global.Headers = Headers;
this.global.FormData = FormData;
this.global.Request = Request;
this.global.Response = Response;
this.global.Request = Request;
this.global.Response = Response;
this.global.fetch = fetch;
this.global.structuredClone = structuredClone;
}
}

module.exports = JSDOMEnvironmentExtended;

And then in the jest.config.js file:
  testEnvironment: "<rootDir>/src/test-helpers/jsdom-extended.js",
i tried import this to setupTests.js and still doesnt work

this is a test environment configuration, importing that in setup tests won't do anything, so i think that's expected. the test environment solution seems reasonable enough to use though

Answer 11 · 2024-01-24T16:56:05.000Z

Conclusion

You are experiencing issues because Jest+JSDOM take away Node.js globals from you (the structuredClone global function in this case). For more info on this, see this and this.

Solution

There is nothing we can or should do to fix legacy tooling that doesn't embrace the platform. The issue here isn't caused by MSW, so it's not MSW that should be fixing it.

I suggest you migrate to a modern test framework, like Vitest. You are also welcome to tackle this in whichever fashion you choose, using custom Jest environment, downgrading to undici@5 and other workarounds. I will not be recommending those because you don't need them. The only reason you resort to them is because Jest forces you to. Naturally, I recommend migrating away from Jest.

Answer 12 · 2024-01-24T19:57:55.000Z

This is the classic answer: "It’s working on my machine". 😄

Answer 13 · 2024-01-25T11:04:24.000Z

@joel-daros, I cannot fix issues in others' tools. You are encouraged to report this to Jest and JSDOM and to have them address this. You cannot bring a tool that depends on JavaScript into another tool that throws JavaScript out of the window and expect things to work.

Answer 14 · 2024-01-31T21:10:50.000Z

@azangru Thanks for the suggestion. I like it better than installing Undici and dealing with incompatibilities with each new version.

This is my jsdom-extended.js file if anyone wants to use the same approach:

const JSDOMEnvironment = require("jest-environment-jsdom").default; // or import JSDOMEnvironment from 'jest-environment-jsdom' if you are using ESM modules

class JSDOMEnvironmentExtended extends JSDOMEnvironment {
  constructor(...args) {
    super(...args);

    this.global.ReadableStream = ReadableStream;
    this.global.TextDecoder = TextDecoder;
    this.global.TextEncoder = TextEncoder;

    this.global.Blob = Blob;
    this.global.File = File;
    this.global.Headers = Headers;
    this.global.FormData = FormData;
    this.global.Request = Request;
    this.global.Response = Response;
    this.global.Request = Request;
    this.global.Response = Response;
    this.global.fetch = fetch;
    this.global.structuredClone = structuredClone;
  }
}

module.exports = JSDOMEnvironmentExtended;

And then in the jest.config.js file:

  testEnvironment: "<rootDir>/src/test-helpers/jsdom-extended.js",

@joel-daros solution works when I combine it with some information from another issue thread. When I add his testEnvironment file jsdom-extended.js and the following configuration in jest.config.js then it works as expected with Webpack, jest, MSW and SWR.

File: jest.config.js

/** @type {import('ts-jest').JestConfigWithTsJest} */
module.exports = {
    verbose: true,
    preset: 'ts-jest',
    testEnvironment: "<rootDir>/jsdom-extended.js",
    // Next Line(s) is important! https://github.com/mswjs/msw/issues/1786#issuecomment-1782559851
    testEnvironmentOptions: {
        customExportConditions: [''],
    },

File: jsdom-extended.js

const JSDOMEnvironment = require("jest-environment-jsdom").default; // or import JSDOMEnvironment from 'jest-environment-jsdom' if you are using ESM modules

class JSDOMEnvironmentExtended extends JSDOMEnvironment {
    constructor(...args) {
        super(...args);

        this.global.ReadableStream = ReadableStream;
        this.global.TextDecoder = TextDecoder;
        this.global.TextEncoder = TextEncoder;

        this.global.Blob = Blob;
        this.global.File = File;
        this.global.Headers = Headers;
        this.global.FormData = FormData;
        this.global.Request = Request;
        this.global.Response = Response;
        this.global.Request = Request;
        this.global.Response = Response;
        this.global.fetch = fetch;
        this.global.structuredClone = structuredClone;
    }
}

module.exports = JSDOMEnvironmentExtended;

I have written an article on medium about it ⇾ Jest + React + MSW + Webpack — Solution for Errors

Answer 15 · 2024-02-01T09:40:39.000Z

@tobiashochguertel, thanks for putting the custom environment up!

So, because in the environment class' context we're still in regular Node.js, we can grab the globals without having to explicitly import them from anywhere? This is interesting. Sounds like it solves the ReadableStream/structuredClone issue.

My only concern with this is that it's a workaround. I feel uncomfortable recommending workarounds. Granted, what we recommend in the docs right now is also a workaround, albeit with a bit smaller effect area. I'd much prefer to see those changes you propose to be a part of jest-environment-jsdom because without them, that environment is quite broken.

Did you consider raising this as an issue/pull request to Jest? That would be a nice solution for everyone (and a contribution opportunity for you too!).

Answer 16 · 2024-02-01T10:55:11.000Z

So, because in the environment class' context we're still in regular Node.js, we can grab the globals without having to explicitly import them from anywhere? This is interesting. Sounds like it solves the ReadableStream/structuredClone issue.

Yes, it solved the problem with the structuredClone issue after solving the ReadableStream issue. I first worked with the polyfill solution which was provided in another issue and is now also available in the FAQ from MSWjs, but then you run into the issue that structuredClone is not available and there I didn't have a polyfill for - and didn't know from where I can get one.

Did you consider raising this as an issue/pull request to Jest? That would be a nice solution for everyone (and a contribution opportunity for you too!).

@kettanaito I didn't think on that. I don't understand the solution completely, I just was good at puzzling everything together.

I didn't want to migrate to Vite now.

Answer 17 · 2024-02-20T15:19:06.000Z

Just wanted to add here that I'm on node 18, and I had to remove the this.global.File = File from the environment provided above, since apparently that API doesn't exist in node 18. Which for me was fine, since I didn't need the File api to run my tests.

Answer 18 · 2024-02-21T03:56:23.000Z

I tried the jsdom-extended.js solution, it seems to work better than using jest-polyfills.js as mentioned in msw docs for troubleshooting fetch

Answer 19 · 2024-03-07T11:14:09.000Z

It would be nice if docs can reflect this in FAQ https://mswjs.io/docs/faq#requestresponsetextencoder-is-not-defined-jest rather than suggesting only one options of jest polyfills

Answer 20 · 2024-03-18T10:58:08.000Z

Update

Thank you so much everyone for battling your way through this. I think we should publish that custom Jest environment to NPM and recommend it officially for everyone using MSW (and beyond, really).

I've created the jest-fixed-jsdom repo to host that environment. I've invited @joel-daros, @tobiashochguertel, and @JamesZoft. You have the write access to that repo now. Please, push the source for that Jest environment and I will help you get it published to NPM! Will update the docs afterward too. Let's solve these Jest issues once and for all.

Thanks.

Answer 21 · 2024-03-21T01:39:03.000Z

Hi @kettanaito, just wanted to say that as a spectator of this issue for some time, we appreciate your time and effort spent on solving this problem, even though it's not an issue caused by msw. I've enjoyed removing my polyfills and undici. Thank you.

Answer 22 · 2024-03-21T02:50:38.000Z

Thank you for the great effort and time @kettanaito 🚀

Answer 23 · 2024-03-21T11:31:19.000Z

@robbieaverill @piyushchauhan2011 means a lot to me to hear this. I know issues like this suck. I also understand how natural it is to blame the tool that seemingly causes the issue when, in fact, it just helps it surface. I hope jest-fixed-jsdom will provide a better developer experience for everyone blocked by this. I hope even more that we migrate away from JSDOM and test browser code in the browser—that is the actual solution.

Answer 24 · 2024-03-22T10:13:07.000Z

I hope even more that we migrate away from JSDOM and test browser code in the browser—that is the actual solution.

😃

Me too. But what does the landscape of test runners currently look like? Last time I checked, there was Web Test Runner, by the Modern Web project; but I think it is targeting primarily browser-native code, which something like React is not. And there was also Playwright component testing, but it has remained experimental for over a year, and is emphatically not a priority for the Playwright team. Have there been any developments among test runners that give you hope that we can move to testing our UI components, which at this point are predominantly React, in the browser?

Answer 25 · 2024-03-24T10:28:27.000Z

@azangru, there's a Browser mode in Vitest, which looks promising. What Playwright is doing with component-level testing for React is also good and I hope it finds the proper love it needs from the team.

In the RSC era, unless the frameworks expose the rendering/hydration pipelines, there will be no reliable way to test RSC on an integration level. That's concerning. I really hope the frameworks will work on this.

Answer 26 · 2024-07-24T11:27:02.000Z

Importing ReadableStream from node:stream/web fixed my tests.

// jest.polyfills.js

const { TextDecoder, TextEncoder } = require('node:util');
const { ReadableStream } = require('node:stream/web'); // <--- this did the magic

Object.defineProperties(globalThis, {
  TextDecoder: { value: TextDecoder },
  TextEncoder: { value: TextEncoder },
  ReadableStream: { value: ReadableStream },
});

const { Blob, File } = require('node:buffer');
const { fetch, Response, Request, FormData, Headers } = require('undici');

Object.defineProperties(globalThis, {
  fetch: { value: fetch, writable: true },
  Blob: { value: Blob },
  File: { value: File },
  Headers: { value: Headers },
  FormData: { value: FormData },
  Request: { value: Request },
  Response: { value: Response },
});

Answer 27 · 2024-07-24T12:42:36.000Z

Please note that the recommended way forward is to use the https://github.com/mswjs/jest-fixed-jsdom package in Jest. Then you don't have to create the setup file at all. It also correctly remaps the globals on the test environment level, where you don't have to import them from undici or other sources.

Answer 28 · 2024-07-24T21:17:12.000Z

Importing ReadableStream from node:stream/web fixed my tests.

// jest.polyfills.js

const { TextDecoder, TextEncoder } = require('node:util');
const { ReadableStream } = require('node:stream/web'); // <--- this did the magic

Object.defineProperties(globalThis, {
  TextDecoder: { value: TextDecoder },
  TextEncoder: { value: TextEncoder },
  ReadableStream: { value: ReadableStream },
});

const { Blob, File } = require('node:buffer');
const { fetch, Response, Request, FormData, Headers } = require('undici');

Object.defineProperties(globalThis, {
  fetch: { value: fetch, writable: true },
  Blob: { value: Blob },
  File: { value: File },
  Headers: { value: Headers },
  FormData: { value: FormData },
  Request: { value: Request },
  Response: { value: Response },
});

This worked for me. I'm using vitest with jsdom so it seems this issue is not only related to jest.