Cannot import built-in node modules in worker in electron/webpack build
Opened this issue · 16 comments
https://github.com/jc-lab/threads-plugin-problem
See the repository above.
Error when using module as esnext in tsconfig.
If you use commonjs instead of esnext, threads-plugin will not detect Worker.
> npm run build
...
Child WorkerPluginLoader:
2 assets
Entrypoint 0 = 0.bundle.worker.js 0.bundle.worker.js.map
[6] (webpack)/buildin/harmony-module.js 573 bytes {0} [built]
[15] ./node_modules/ts-loader??ref--4!./src/test.thread.ts + 2 modules 16.7 KiB {0} [built]
| ./node_modules/ts-loader??ref--4!./src/test.thread.ts 730 bytes [built]
| + 2 hidden modules
+ 26 hidden modules
ERROR in ./src/test.thread.ts (./node_modules/ts-loader??ref--4!./src/test.thread.ts)
Module not found: Error: Can't resolve 'fs' in 'D:\temp\webpack-worker-test\src'
@ ./src/test.thread.ts (./node_modules/ts-loader??ref--4!./src/test.thread.ts) 3:0-25 8:22-42
ERROR in ./src/test.thread.ts (./node_modules/ts-loader??ref--4!./src/test.thread.ts)
Module not found: Error: Can't resolve 'zlib' in 'D:\temp\webpack-worker-test\src'
@ ./src/test.thread.ts (./node_modules/ts-loader??ref--4!./src/test.thread.ts) 4:0-29 9:23-41
It works with TypeScript, you just need to set module to esnext as you already wrote. It's also documented like that: See here.
The error above is unrelated to threads.js and the threads-plugin as far as I can tell.
Try npm install --save-dev @types/node.
Feel free to re-open if I missed something. The error above looks very much like something else is fishy, though.
@andywer
If you comment out ThreadsPlugin as shown below, no error will occur.
That's why I think it's an issue of threads-plugin.
// plugins: [new ThreadPlugin()],output:
> npm run build
> webpack --config webpack.config.js -p
Hash: 1fb9dddabef6f1be9dc0
Version: webpack 4.41.5
Time: 1404ms
Built at: 2020-01-06 23:04:43
Asset Size Chunks Chunk Names
..\dist-ts\index.d.ts 47 bytes [emitted]
..\dist-ts\index.d.ts.map 104 bytes [emitted]
..\dist-ts\test.thread.d.ts 53 bytes [emitted]
..\dist-ts\test.thread.d.ts.map 116 bytes [emitted]
bundle.js 7.37 KiB 0 [emitted] main
bundle.js.map 4.43 KiB 0 [emitted] [dev] main
Entrypoint main = bundle.js bundle.js.map
[0] external "threads" 42 bytes {0} [built]
[1] ./src/index.ts 3.26 KiB {0} [built]
Well, if you out-comment the threads-plugin, then webpack won't even know that it should process the worker script and its dependencies, so it won't crash as the worker's code isn't touched by webpack at all 😉
Have you installed the @types/node package?
As shown above, @types/node is installed.
The same code in test.worker.ts works fine with index.ts.
Wow, that seems like a weird issue and requires more time to investigate then.
@andywer Please review. Thank you.
Can you please check if it's fixed in the latest version? #16 might have fixed it.
I have encountered this issue.
solved it by doing:
// webpack.main.js
const NodeTargetPlugin = require('webpack/lib/node/NodeTargetPlugin');
const ThreadsPlugin = require('threads-plugin');
module.exports = function(config) {
// NodeTargetPlugin must be used in order to use nodejs built-in API in workers.
config.plugins.unshift(
new ThreadsPlugin({
plugins: [new NodeTargetPlugin()]
})
);
return config;
};
this works for both main thread and renderer threads
@raz-sinay Cool, thanks!
@jc-lab Can you try and confirm that it works? If so, let's add a target option to the threads-plugin that defaults to the webpack config's main target option and adds the NodeTargetPlugin if the new setting is node.
@andywer @jc-lab I've just discovered that you need to enable nodeIntegrationInWorker when creating a new BrowserWindow for fs to work. See https://www.electronjs.org/docs/tutorial/multithreading for more information.
@pverscha Sure, but you shouldn't… 😅 One little XSS vulnerability in your webapp code and the attacker can take over your whole system at once ;)
I wouldn't use fs in a BrowserWindow context at all, but multithreading should still be possible out of the box - in the web context via web workers and outside of it using worker threads.
(That little security insight is brought to you by someone building cryptocurrency wallets with Electron for a living, among other things ;) )
@raz-sinay Cool, thanks!
@jc-lab Can you try and confirm that it works? If so, let's add a
targetoption to thethreads-pluginthat defaults to the webpack config's maintargetoption and adds theNodeTargetPluginif the new setting isnode.
I met the same problem and the NodeTargetPlugin trick works for me. Maybe you can add this to the documentation?
@arition Hmm, are you using the latest version of the threads-plugin? As you can see in the loader code, the behavior that I outlined before has already been implemented.
So if you set the webpack target option to something containing node, it will automatically add the NodeTargetPlugin to the worker compiler.