josdejong/workerpool

Question: complex objects to worker

vanodevium opened this issue · 3 comments

vanodevium commented

Hello!
First of all, thank you very much for this incredible library!

I have a question that has probably been asked 100 times, but I haven't found an answer that works for my case.

I have a pool of 10+ workers. Each of them is a function that uses an instance of puppeteer. And I need to somehow make it so that I can transfer exactly one instance of the puppeteer, because at the moment (due to the fact that the pool creates workers very quickly), I cannot achieve a singleton.

My worker file looks like:

const workerPool = require("workerpool");
const Worker = require("./worker");

const worker = new Worker(/* maybe I can make singleton exactly at this place? Or pass instance? No? */);

async function processPage(options) {
  return worker.process(options);
}

workerPool.worker({
  processPage: processPage,
});

Please help. I will be grateful for any answer! Thanks!

josdejong commented

Some ideas:

  • Configure workerpool to create only 1 worker
  • Create one worker where your puppeteer instance is and let the workers communicate with that via a MessageChannel
vanodevium commented

So far, the only way I've come up with is as follows:

I am creating an instance of puppeteer and waiting it by await and only after that I initialize the pool of workers in which I only connect to that instance.

I could not come up with more options.
Thank you for your reply.
Until we close, maybe someone else will recommend some method to me.

Thank you!

josdejong commented

👍

I'll close this issue now, feel free to comment with more options or reopen if needed.

  • 👍1