How to use other packages within task?

[wxfred]

const PSD = require('psd.js');
const { StaticPool } = require("node-worker-threads-pool");

const pool = new StaticPool({
size: 4,
task: param=> {
const psd = PSD.func(param);
...
},
});

pool.exec(param).then(() => { ... });

ReferenceError: PSD is not defined
at Object.task ([worker eval]:8:17)
at MessagePort. ([worker eval]:29:46)
at MessagePort.emit (events.js:223:5)
at MessagePort.onmessage (internal/worker/io.js:70:8)

ps: PSD object is not support clone, i can't pass it within the param.

[SUCHMOKUO]

Just use require in your task function.

const { StaticPool } = require("node-worker-threads-pool");

const pool = new StaticPool({
  size: 4,
  task: param=> {
    const PSD = require('psd.js');
    const psd = PSD.func(param);
    ...
  },
});

pool.exec(param).then(() => { ... });

[wxfred]

@SUCHMOKUO It's not working.
ReferenceError: webpack_require is not defined
at Object.task ([worker eval]:8:19)
at MessagePort. ([worker eval]:18:46)
at MessagePort.emit (events.js:223:5)
at MessagePort.onmessage (internal/worker/io.js:70:8)

I figure it out that moving my code to a task js file can solve the problem.

[SUCHMOKUO]

@SUCHMOKUO It's not working.
ReferenceError: webpack_require is not defined
at Object.task ([worker eval]:8:19)
at MessagePort. ([worker eval]:18:46)
at MessagePort.emit (events.js:223:5)
at MessagePort.onmessage (internal/worker/io.js:70:8)

I figure it out that moving my code to a task js file can solve the problem.

Looks like you're using webpack to bundle your code?
Maybe you could try setting webpack target to "node". see webpack target

[wxfred]

@SUCHMOKUO not working.
I'm developing an electron app with vue-cli-plugin-electron-builder, following official guide https://nklayman.github.io/vue-cli-plugin-electron-builder/guide/configuration.html#webpack-configuration

module.exports = {
  configureWebpack: {
    // Configuration applied to all builds
  },
  pluginOptions: {
    electronBuilder: {
      chainWebpackMainProcess: (config) => {
        // Chain webpack config for electron main process only
        config.target('node')
        console.log(config.get('target')) // print 'node'
      },
      ...

ReferenceError: webpack_require is not defined.

[SUCHMOKUO]

The problem here is that webpack is transpiling require to __webpack_require__, so the transpiled code running in worker causes ReferenceError.

I've added this.require to worker function to avoid this problem, please try the latest version (1.4.3).

Feel free to comment if there is any problem :)

[chrgue]

Hi there, I have a similar problem and I am pretty sure the problem occurs of my missing knowledge. Hope you can help me out.

const { StaticPool } = require("node-worker-threads-pool");
const wait = require('./wait')

const waitTime = 100

const staticPool = new StaticPool({
    size: 4,
    task: (param) => {
        console.log(__filename, __dirname) // -> "[worker eval] ."
        this.require('./wait.js')(param)
        return 'payload generated by worker thread'
    },
});

async function doBlockingWork() {
    return new Promise((resolve) => {
        wait(waitTime)
        resolve('payload generated by blocking work')
    })
}

async function doBlockingWorkWithWorkers() {
    return staticPool.exec(waitTime)
}

I want to show some kind of side bye side comparison with & without worker threads. So in this module I tried to load the wait module in the context of the main thread and in the context of the worker thread.

But now I get this:

(node:85487) UnhandledPromiseRejectionWarning: Error: Cannot find module './wait'

Do you have an idea on how to fix that?

Thank you

[SUCHMOKUO]

@chrgue What's in your 'wait.js'? Are you using webpack?

[SUCHMOKUO]

Looks like there are some issues with integration with webpack.

[chrgue]

@SUCHMOKUO Thanks for your reply.

No I do not use webpack here:

"dependencies": {
    "cookie-parser": "~1.4.4",
    "debug": "~2.6.9",
    "express": "~4.16.1",
    "morgan": "~1.9.1",
    "node-worker-threads-pool": "^1.4.3"
  }

Here is the content of wait.js:

function now() {
    return new Date().getTime();
}

function wait(millis){
    const waitTill = now() + millis;
    while(waitTill > now()) {} //NOSONAR
}

module.exports = wait

Any idea?

[gentlementlegen]

@chrgue Instead of writing the worker's code within an arrow function, try instead to use it in a separate file and assign the absolute path of your worker.js file to the StaticPool task. The require calls should work properly this way.

Cheers