Unable to update workspace package-lock.json after `version` on npm@7

[bigtimebuddy]

I manage two mono-repo projects (PixiJS and PixiJS Filters) which are both experiencing flavors of the same issue after upgrading to npm@7 workspaces. The package-lock file is not updated after calling lerna version. This creates an unclean git environment when CI attempts to publish resulting in a blocked publish (see example).

Expected Behavior

After doing lerna version, the package-lock.json should be updated to reflect the version number bumps in the packages and included in the tag.

Current Behavior

package-lock.json is not updated, and version numbers on workspace packages reflect the previous versions. Subsequent npm installs create a change in the package-lock.json.

Possible Solution

I tried unsuccessfully add a hook to bump the lock:
"postversion": "npm i --package-lock-only"
But this errored Failed to exec postversion script

Maybe having a pretag lifecycle hook would help to add this?

Steps to Reproduce (for bugs)

1. git clone git@github.com:pixijs/filters.git
   2.npm install
2. npm run release -- --no-push --force-publish (notice no package-lock.json changes in tag)
3. npm install (notice package-lock.json is updated)

lerna.json

<!-- Please paste your `lerna.json` here -->

lerna-debug.log

<!-- If you have a `lerna-debug.log` available, please paste it here -->
<!-- Otherwise, feel free to delete this <details> block -->

Context

Your Environment

Executable	Version
lerna --version	v4.0.0 & v3.13.4
npm --version	v7.11.2
yarn --version	n/1
node --version	v12.18.1

OS	Version
macOS BigSur	11.2.3

[bigtimebuddy]

Sharing my workaround. This, however, should probably be Lerna's responsibility to play well with npm 7.

Before

lerna version

After

# Ignore Lerna's tag and push
lerna version --no-push --no-git-tag-version

# Get the current tag version
tag=v$(node -e "process.stdout.write(require('./lerna.json').version)");

# Update the lock file here
npm i --package-lock-only

# Auto-tag and auto-commit like usual
git commit --all -m ${tag}
git tag -a ${tag} -m ${tag}
git push --tags
git push

[andyjy]

Same issue, I was able to come up with what felt like a 'cleaner' workaround:

Just add a version npm script to the root package.json:

  "scripts": {
     ...
     "version": "npm install --package-lock-only && git add package-lock.json"
   }

..turns out this (npm run version) is called automatically by Lerna "AFTER bumping the package version, but BEFORE commit"

(which I initially discovered by reading the lerna version source code, before realising is actually documented in the "Lifecycle scripts" section of the version command 🤦‍♂️ ).

Thanks @bigtimebuddy for the original issue and workaround above - which gave me a strong head-start to understand what was up 😃

(Looks like this could be a duplicate of #2879)

[bigtimebuddy]

Looking through the post-4.0.0 release commits, I can see this project also hit this issue here c019815 to manually sync workspace versions.

[bigtimebuddy]

@andyjy after some testing, I found that some npm versions are temperamental when using that approach. There seems to be a change introduced here npm/cli@v7.20.2...v7.20.3 which breaks when using --package-lock-only with unpublished-private packages.

✅ Worked

• 7.17.0
• 7.18.1
• 7.19.1
• 7.20.0
• 7.20.2

❌ Failed

• 7.20.3
• 7.20.6
• 7.24.2
• 8.1.2

[andyjy]

Oh, curious. To be fair I'm probably guilty of cargo-culting --package-lock-only. That said, npm install --package-lock-only is "working for me" with npm v8.1.0, where I can best define "working for me" as:

The new v2 of package-lock.json used by npm >= 7 now includes a bunch of extra detail about dependencies - specifically including nested version information for the packages within the monorepo - which lerna doesn't know to update during lerna version. Using "version": "npm install --package-lock-only && git add package-lock.json" fixes by successfully bumping the version numbers for the monorepo-contained packages within the single commit created during lerna version.

Example:

1. lerna version-powered commit where it didn't update package-lock.json (no change to package-lock.json)
2. subsequent manual update (from running npm install)
3. fix implemented
4. successful release including updated package-lock.json

(I'm using (only) relative file:packages/... specifiers for my monorepo packages in package.json, in case that makes a difference somehow)

[bigtimebuddy]

Thanks, I'm trying to create a minimal reproduction to see if it's actually an npm issue, but no success yet. Will post if I find something.

Update: created a reproduction: https://github.com/bigtimebuddy/npm-bug-package-lock-only

Seems like an npm regression, will report there.

[andyjy]

@bigtimebuddy so I realised I'm not hitting your error because i'm not (yet) including a private package as a devDependency. Which seemed like a thing I'll wish to do shortly, so I took a look at your repro repo.

Notwithstanding it indeed looks like an npm regression you've hit, it looks like everything(?) works if you use a relative file: specifier instead of the version string when including your private devDependency.

I.e. in your example packages-a/package.json: "devDependencies": ["@tools/a": "file:../tools-a"]

• 👎  creates a hardcoded dependency on package folder structure
• 👍  smaller release diffs with fewer version lines to update
• 🤷  versioned dependencies for private packages in same monorepo don't seem meaningful anyhow

Does this make sense to you? Anything I'm missing re: utilising relative file: version specifiers for private devDependencies?

[bigtimebuddy]

Nice @andyjy. All this makes sense and seems like file: sounds like a workaround. I know in the past when I've used internal versioning like *, Lerna blew that away with the fuzzy/exact versions. Do you know if Lerna leaves file: paths intact?

[andyjy]

Huzzah! Yes it appears Lerna does leave file: paths intact; in fact I see it listed as a highlighted feature for devDependencies (lerna link convert) - and in fact recommended..

[JamesHenry]

Hi Folks 👋

You may or may not know that lerna is now under the stewardship of Nrwl (announcement here #3121), a company with a long history of not just producing valuable open-source software (OSS), but also backing others (at the time of writing, Nrwl has donated over $50,000 to OSS it hasn't created, see https://opencollective.com/nx for full details).

Quite simply, Nrwl ❤️ OSS, and is committed to making lerna the best it can be. We use it ourselves.

We hope you will continue to be a part of this community as we look to take things forward from here!

Please see #3140 for more details on our plans for 2022.

In the case of this specific issue, I'm please to say that a fix for it was just merged in #3091 and will go out in the next release of lerna.

Many thanks 🙏