developit/workerize-loader

Webpack runs out of heap memory, only when using workerize-loader

stoplion opened this issue · 4 comments

Hi,

Wonder if you might know what might be causing this this error.
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

When I take away workerize-loader, the bundle compiles and Node/Webpack does not run out of memory.

Using Workerize-Loader 2.0.0
and Webpack 4.27.1 (via Webpacker Rails)

My worker is pretty simple..

import parseLine from './parse_line';


export function runParseLine(linesStr, varTable) {

  const lines = linesStr.split(/\n/);

  const value = lines.map((lineText, lineNum) => {
    return {
      lineNum,
      lineText: parseLine(lineText, varTable),
    };
  });

  return value;
}

and I'm importing it like this:

import ParseLineWorker from 'workerize-loader!../utils/parser/parse_line_worker';
const parseLineWorker = new ParseLineWorker();

If I comment out workerize-loader! in the import and change the worker to export default, it compiles ok.

When running ./bin/webpack --progress, it get stuck on the parse_line_worker.js file...

69% building 1702/1704 modules 2 active ...d/lib/utils/parser/parse_line_worker.js

until it eventually runs out of memory and fails.

Been Googling this to death for the last three days. Would appreciate any suggestions...

Is it possible you've set up an infinite looping build here?

Workerize-loader creates a webpack "child compiler", which certainly uses more memory. Depending on your configuration, it could potentially double memory consumption.

Not sure, I'll check and report back

yup that is it

TeoTN commented

Sorry for unearthing this issue, but do you have any hints on how to debug it?