Loading other libraries in a worker thread

[cskiwi]

Current Behavior

hi,

I'm trying to offload some excel processing to a separate worker thread so the main thread isn't blocked
I got the worker working, but I need access to another library @badman/backend-database however this is

Expected Behavior

That it's able to resolve the library

GitHub Repo

No response

Steps to Reproduce

All files below are in the library /libs/backend/ranking/

1. have a worker.ts:

async function run() {
  if (isMainThread) {
    throw new Error('This script should be run as a worker thread');
  }
  const app = await NestFactory.createApplicationContext(ProcessRankingModule);
  const updateRankingService = app.get(UpdateRankingService);
  updateRankingService.run()
}

run();

2. config.ts

import path from 'path';
const workerThreadFilePath = path.resolve(process.cwd(), 'dist/libs/backend/ranking/src/worker/process-ranking.js');
export default workerThreadFilePath;

3. call the worker:

const worker = new Worker(workerThreadFilePath, {
  workerData: {
    // data
  },
});

worker.on('message', () => {
  this._logger.verbose('Done');
});
worker.on('error', (e) => {
  return this._logger.error('on error', e, e.message);
});
worker.on('exit', (code) => this._logger.log('on exit', code));

the UpdateRankingService includes some models from @badman/backend-database

this throws:

[Badman] - 12/13/2024, 2:52:15 PM   ERROR [Cannot find module '@badman/backend-database'
Require stack:
- <repo>\dist\libs\backend\ranking\src\services\calculation\calculation.service.js
- <repo>\dist\libs\backend\ranking\src\services\calculation\index.js
- <repo>\dist\libs\backend\ranking\src\services\index.js
- <repo>\dist\libs\backend\ranking\src\worker\process-ranking.js] on error - {
  stack: [
    {
      code: 'MODULE_NOT_FOUND',
      requireStack: [
        '<repo>\\dist\\libs\\backend\\ranking\\src\\services\\calculation\\calculation.service.js',
        '<repo>\\dist\\libs\\backend\\ranking\\src\\services\\calculation\\index.js',
        '<repo>\\dist\\libs\\backend\\ranking\\src\\services\\index.js',
        '<repo>\\dist\\libs\\backend\\ranking\\src\\worker\\process-ranking.js'      
      ]
    }
  ],
  appname: 'api',
  version: '6.174.5'
} +2s

Nx Report

NX   Report complete - copy this into the issue template

Node           : 21.7.2
OS             : win32-x64
Native Target  : x86_64-windows
bun            : 1.1.34

nx                 : 20.1.3
@nx/js             : 20.1.3
@nx/jest           : 20.1.3
@nx/eslint         : 20.1.3
@nx/workspace      : 20.1.3
@nx/angular        : 20.1.3
@nx/devkit         : 20.1.3
@nx/eslint-plugin  : 20.1.3
@nx/nest           : 20.1.3
@nx/node           : 20.1.3
@nx/playwright     : 20.1.3
@nx/vite           : 20.1.3
@nx/web            : 20.1.3
@nx/webpack        : 20.1.3
typescript         : 5.5.4
---------------------------------------
Registered Plugins:
@nx/eslint/plugin
@nx/jest/plugin
@nx/playwright/plugin
@nx/webpack/plugin
---------------------------------------
Community plugins:
@auth0/auth0-angular : 2.2.3
apollo-angular       : 7.2.1
ng-apexcharts        : 1.12.0
ng-process-env       : 16.0.6
ngxtension           : 3.5.5

Failure Logs

Package Manager Version

No response

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

I tried defining an additional entry point in my project.json. but that didn't make any difference

[cskiwi]

Did some further investigation and found that working with normal tsconfig paths it does work

Working example: https://github.com/cskiwi/nestjs-worker-thread-example
Not working example: https://github.com/cskiwi/nestjs-nx-worker-thread-example

[cskiwi]

Fixed by doing this: #7123 (comment)