metarhia/Metarhia

Require for native add-ons in worker_threads

Opened this issue · 6 comments

tshemsedinov commented

Some modules, for example argon2 can't be loaded in worker threads.

Try load.js with the following code:

'use strict';

const { Worker } = require('worker_threads');

const argon2 = require('argon2');
console.dir({ argon2 });

new Worker('./thread.js');

and thread.js with the following code:

'use strict';

const argon2 = require('argon2');

console.dir({ argon2 });

we will get exception:

Error: Module did not self-register.
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:775:18)
    at Module.load (internal/modules/cjs/loader.js:626:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:566:12)
    at Function.Module._load (internal/modules/cjs/loader.js:558:3)
    at Module.require (internal/modules/cjs/loader.js:663:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at bindings (.../node_modules/bindings/bindings.js:112:48)
    at Object.<anonymous> (.../node_modules/argon2/argon2.js:4:37)
    at Module._compile (internal/modules/cjs/loader.js:734:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:745:10)

We need to find solution to use worker thread in impress. @nechaido you can assign this task to developers.

tshemsedinov commented

After resolving this problem we can use solution here: metarhia/impress#1099

lundibundi commented

worker_threads doesn't support native addons as of 11.9.0
"Native addons are not supported yet." therefore it won't be possible to require('argon2') in them as npm argon2 module is just bindings to the C argon library.

Edit: after additional testing, it looks like you can safely run the above code on node 11.7.0+
Edit2: --//-- not in multiple threads though

tshemsedinov commented

It works in 11.9.0 if you require from a single worker, but fails if you require from master process and 1 worker or from more then 1 workers. We need at least: mdsf, websocket, argon2

lundibundi commented

Starting from Node.js 11.11.0 (nodejs/node#26175) it should be possible to use native addons from multiple worker threads as long as they are

  • an N-API addon, or
  • declared as context-aware using NODE_MODULE_INIT() as described above

(https://nodejs.org/api/addons.html#addons_worker_support)

Where the latter basically means that the addon properly manages its global/static state to protect it against race conditions, clean up its data when instance shutdowns, doesn't store persistent JS object references and doesn't access JS objects from the wrong context (not the one they were created in). Therefore using NODE_MODULE_INIT instead of NODE_MODULE tells Node.js that the module respects the given rules.

  • 👍1
  • 🎉1
  • 🚀1