Externals packaging prioritises yarn.lock lookup

[Dutzu]

Describe the bug
When packing externals, the Utils/findUpIO willl recursively search up the directory structure for a given file and the function findProjectRoot calls this function sequentially looking for yarn.lock, pnpm-lock.yaml and package-lock.json.

In case you are using pnpm or npm instead of yarn BUT you have a yarn.lock somewhere up the directory structure, it will cause a fail-ure.

To Reproduce

1. Setup a basic serverless with esbuild project, define any externals. Use npm with a package-lock.json file in your project
2. Somewhere, anywhere up the folder structure create a yarn.lock file.
3. Execute sls package.

Expected behavior
The build should work....

Screenshots or Logs

lifecycle:command:invoke:hook: [16] < before:package:createDeploymentArtifacts
Compiling to node18 bundle with esbuild...
Compiling with concurrency: Infinity
Compiling completed.
plugin:serverless-esbuild: Trying to create packager: npm
plugin:serverless-esbuild: Packager created: npm
lifecycle:command:invoke:hook: [16] > before:package:createDeploymentArtifacts
process: handle error
process: finalize { error:
   { Error: ENOENT: no such file or directory, open '/Users/anonymous/package.json'
       at Object.openSync (node:fs:600:3)
       at Object.readFileSync (node:fs:468:35)
       at readFileSync (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/utils/fs/read-file-sync.js:7:24)
       at Utils.readFileSync (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/classes/utils.js:78:12)
       at EsbuildServerlessPlugin.packExternalModules (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless-esbuild/dist/pack-externals.js:212:51)
       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
       at async before:package:createDeploymentArtifacts (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless-esbuild/dist/index.js:78:17)
       at async PluginManager.runHooks (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/classes/plugin-manager.js:530:9)
       at async PluginManager.invoke (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/classes/plugin-manager.js:563:9)
       at async PluginManager.run (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/classes/plugin-manager.js:604:7)
       at async Serverless.run (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/serverless.js:179:5)
       at async /Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/scripts/serverless.js:834:9
     [stack]:
      "Error: ENOENT: no such file or directory, open '/Users/anonymous/package.json'\n    at Object.openSync (node:fs:600:3)\n    at Object.readFileSync (node:fs:468:35)\n    at readFileSync (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/utils/fs/read-file-sync.js:7:24)\n    at Utils.readFileSync (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/classes/utils.js:78:12)\n    at EsbuildServerlessPlugin.packExternalModules (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless-esbuild/dist/pack-externals.js:212:51)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async before:package:createDeploymentArtifacts (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless-esbuild/dist/index.js:78:17)\n    at async PluginManager.runHooks (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/classes/plugin-manager.js:530:9)\n    at async PluginManager.invoke (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/classes/plugin-manager.js:563:9)\n    at async PluginManager.run (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/classes/plugin-manager.js:604:7)\n    at async Serverless.run (/Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/lib/serverless.js:179:5)\n    at async /Users/anonymous/Projects/theProjectInQuestion/node_modules/serverless/scripts/serverless.js:834:9",
     [message]: "ENOENT: no such file or directory, open '/Users/anonymous/package.json'",
     errno: -2,
     syscall: 'open',
     code: 'ENOENT',
     path: '/Users/anonymous/package.json' },
  shouldBeSync: undefined,
  telemetryData: undefined,
  shouldSendTelemetry: undefined }

Versions (please complete the following information):

• OS: Mac
• Serverless Framework Version: 3.32.2
• Plugin Version: 1.46.0

[Dutzu]

I thought of 2 possible solutions:

1. Since you already know you the packer you will use, you could narrow down the search and only look for the lock file specific to your packer.
   a. In my case the issue was clear: it found a location of a yarn.lock file and then tried to call npm run beforePackage on it.
2. Instead of triggering the search multiple times for each individual lock file name, search only 1 time using all of them as valid options for a match.

Instead of:

const findUpIO = (name: string, directory = process.cwd()): IOO.IOOption<string> =>
  pipe(path.resolve(directory), (dir) =>
    pipe(
      safeFileExistsIO(path.join(dir, name)),
      IO.chain((exists: boolean) => {
        if (exists) return IOO.some(dir);
        if (isPathRoot(dir)) return IOO.none;
        return findUpIO(name, path.dirname(dir));
      })
    )
  );

/**
 * Forwards `rootDir` or finds project root folder.
 */
export const findProjectRoot = (rootDir?: string) =>
  pipe(
    IOO.fromNullable(rootDir),
    IOO.fold(() => findUpIO('yarn.lock'), IOO.of),
    IOO.fold(() => findUpIO('pnpm-lock.yaml'), IOO.of),
    IOO.fold(() => findUpIO('package-lock.json'), IOO.of),
    IOO.toUndefined
  )();

Have something like:

const findUpIO = (**names: string[]**, directory = process.cwd()): IOO.IOOption<string> =>
  pipe(path.resolve(directory), (dir) =>
    pipe(
      **array.some(names.map((name) => safeFileExistsIO(path.join(dir, name))))**,
      IO.chain((exists: boolean) => {
        if (exists) return IOO.some(dir);
        if (isPathRoot(dir)) return IOO.none;
        return findUpIO(name, path.dirname(dir));
      })
    )
  );

/**
 * Forwards `rootDir` or finds project root folder.
 */
export const findProjectRoot = (rootDir?: string) =>
  pipe(
    IOO.fromNullable(rootDir),
    IOO.fold(() => findUpIO(['yarn.lock', 'pnpm-lock.yaml', 'package-lock.json'), IOO.of),
    IOO.toUndefined
  )();

I know this is a crude solution, and I haven't given much thought and effort into how it would fit into the overall code, but perhaps it will help speed things up and shorten the time it takes to fix this.

[deanc]

I can confirm this issue. We spent some time debugging it, and the cause was looking up the directory tree and finding a package-lock.json outside of the repo.

[github-actions]

🎉 This issue has been resolved in version 1.48.5 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀