ESM loader hooks in Workers no longer working since Node.js 22.2
alan-agius4 opened this issue · 7 comments
Version
v22.2.0
Platform
Linux 6.6.15-2rodete2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.6.15-2rodete2 (2024-03-19) x86_64 GNU/Linux
Subsystem
No response
What steps will reproduce the bug?
Since Node.js version 22.2, ESM loader hooks no longer function inside of a worker and the Node.js application becomes unresponsive.
app.js
const { Worker, isMainThread } = require("node:worker_threads");
if (!isMainThread) {
import("./test.mjs").then(console.log).catch(console.error);
} else {
new Worker("./app.js", {
execArgv: [
'--import=data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("./hooks.mjs", pathToFileURL("./"), { data: {} });',
],
});
}hooks.mjs
export async function initialize() {
console.log('init')
}
export async function resolve(specifier, context, nextResolve) {
console.log('resolve: ' + specifier)
return nextResolve(specifier, context)
}$ node app.js
How often does it reproduce? Is there a required condition?
Always
What is the expected behavior? Why is that the expected behavior?
init
Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/test-loader-esm/test.mjs' imported from ///test-loader-esm/app.js
at finalizeResolution (node:internal/modules/esm/resolve:264:11)
at moduleResolve (node:internal/modules/esm/resolve:924:10)
at defaultResolve (node:internal/modules/esm/resolve:1148:11)
at nextResolve (node:internal/modules/esm/hooks:750:28)
at resolve (/test-loader-esm/hooks.mjs:8:10)
at nextResolve (node:internal/modules/esm/hooks:750:28)
at Hooks.resolve (node:internal/modules/esm/hooks:238:30)
at handleMessage (node:internal/modules/esm/worker:199:24)
at Immediate.checkForMessages (node:internal/modules/esm/worker:141:28)
at process.processImmediate (node:internal/timers:478:21) {
code: 'ERR_MODULE_NOT_FOUND',
url: '/test-loader-esm/test.mjs'
}
resolve: ./test.mjs
What do you see instead?
Application becomes unresponsive.
Additional information
I suspect that this is caused by #52706
Installing a game (KH3) via Heroic, got an error regarding Node.js. Should I be concerned, and what should I do? Thanks in advance for any help.
this behaviour changed indeed with #52706. Passing execArgv to new Worker is not supported. Please see this PR as a trial to document the behaviour.
Please add the customization hook to the main thread and this will affect all created workers as described here.
app.js will become:
const { Worker, isMainThread } = require("node:worker_threads");
if (!isMainThread) {
import("./test.mjs").then(console.log).catch(console.error);
} else {
new Worker("./app.js");
}
hooks.mjs will stay the same.
reg.mjs will be:
import { register } from "node:module";
import { pathToFileURL } from "node:url";
register("./hooks.mjs", pathToFileURL("./"));
and the application start will be:
node --import=./reg.mjs app.js
the customization hooks chain will be as mentioned the same on all threads and the output will be something like:
init
resolve: file:///test-loader-esm/app.js
resolve: file:///test-loader-esm/app.js
resolve: ./test.mjs
Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/test-loader-esm/test.mjs' imported from /test-loader-esm/app.js
at finalizeResolution (node:internal/modules/esm/resolve:260:11)
at moduleResolve (node:internal/modules/esm/resolve:920:10)
at defaultResolve (node:internal/modules/esm/resolve:1119:11)
at nextResolve (node:internal/modules/esm/hooks:791:28)
at resolve (file:///test-loader-esm/hooks.mjs:7:10)
at nextResolve (node:internal/modules/esm/hooks:791:28)
at Hooks.resolve (node:internal/modules/esm/hooks:238:30)
at MessagePort.handleMessage (node:internal/modules/esm/worker:255:24)
at [nodejs.internal.kHybridDispatch] (node:internal/event_target:816:20)
at MessagePort.<anonymous> (node:internal/per_context/messageport:23:28) {
code: 'ERR_MODULE_NOT_FOUND',
url: 'file:///test-loader-esm/test.mjs'
}
I assume this is caused by #52706.
There a bug was fixed that every worker created a fresh hooks thread instead of using that one created by the main thread.
As a result calling module.register() in a worker has no effect.
#53006 will improve the docs regarding this.
So you should register your hooks once for all in main thread before starting the worker.
In all fairness, this seems like a breaking change, especially since this feature is in the Release Candidate stage. I wouldn't expect such a drastic change in behavior, particularly in a minor release.
For us, the Angular CLI, this is quite a breaking change because in some of our workers, we want different resolutions, which differs from the main thread and that of other workers.
We could surely do some workarounds, but it was very convenient that workers supported custom ESM hooks separate from the main thread.
If this is the intended and desired behavior, IMHO, it should be implemented in a major release to comply with semantic versioning.
#52706 fixed a longstanding bug: #50752. We knew that this fix might surprise or break people, which is why we were waiting to fix this before making the API stable (and we still haven’t, in case any further issues arise). This is the point of the Release Candidate stage: to identify and fix issues like this.
I’m sorry that this change has inconvenienced you, but this is why we have experimental features and why they need to be allowed to change. If we couldn’t fix this bug until October with Node 23, and then we couldn’t backport the fix to the LTS lines, it would be April 2025 before a stable hooks API would be available on an LTS line. Many users are eager for this API to become stable so that they can rely on it, and stretching it out over years inconveniences them for the dubious benefit of protecting early adopters who knew what they were getting into by relying on an experimental API. We prioritize the larger group of users seeking stability over the early adopters.