[BUG] Code in `node_modules` not updated when branch in dependency changes

[JustAnotherJavaProgrammer]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

When the branch name in the package.json is changed for a package installed from a git repository and npm install is run, the code in node_modules for that package is not updated to that from the different branch.

Expected Behavior

After updating the package.json file, running npm install causes the code for that package in node_modules to originate from the changed branch.

Steps To Reproduce

1. In an npm project, install a package from a git repository, like so (here: from a hosted Gitlab instance using a Group Access Token):

npm install git+https://group_000_bot_username:glpat-token@gitlab.example.com/project/sdk#dev

The package.json in the project/sdk repo contains a line like:

{
  "name": "@proj/source",
  ...

2. A dependency was added to the package.json of the current project, looking like this:

"dependencies": {
    "@proj/source": "git+https://group_000_bot_username:glpat-token@gitlab.example.com/project/sdk#dev"
    ...

And the source code from branch dev can be found in node_modules/@proj/source.
3. Change the dependency in the package.json to

"dependencies": {
    "@proj/source": "git+https://group_000_bot_username:glpat-token@gitlab.example.com/project/sdk#main"

(and save the file).
4. Run npm i (or npm install). The output will look something like this:

up to date, audited XXX packages in 1s

XXX packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

5. See the line in package-lock.json, reflecting the changed branch:

...
"dependencies": {
        "@proj/source": "git+https://group_000_bot_username:glpat-token@gitlab.example.com/project/sdk#main"
        ...

And see the files in node_modules/@proj/source. Those have not changed and still reflect the dev branch, not main.

Workarounds

I have discovered the following two workarounds:

1. Instead of editing package.json, run npm install again, but with the changed branch name:

npm install git+https://group_000_bot_username:glpat-token@gitlab.example.com/project/sdk#main

This changes package.json, package-lock.json and the files in node_modules/@proj/source are updated to their versions from the main branch.
2. Change the package name (and referenced branch) in the package.json (of your current project), e.g. like this:

"dependencies": {
    "@proj/sources": "git+https://group_000_bot_username:glpat-token@gitlab.example.com/project/sdk#main"
    ...

(Note the added s to @proj/source)
Now, run npm install. node_modules/@proj/source is replaced with node_modules/@proj/sources.

Then, change the package name back (not the branch):

"dependencies": {
    "@proj/source": "git+https://group_000_bot_username:glpat-token@gitlab.example.com/project/sdk#main"
    ...

Now, run npm install. node_modules/@proj/sources is replaced with node_modules/@proj/source. The files in node_modules/@proj/source now originate from the main branch.

Environment

• npm: 10.2.0
• Node.js: 20.8.0
• OS Name: MacOS 14.0 23A344 x86_64
• System Model Name: MacBookPro16,1
• npm config:

; "builtin" config from /usr/local/lib/node_modules/npm/npmrc

prefix = "/usr/local" 

; node bin location = /usr/local/Cellar/node/20.8.0/bin/node
; node version = v20.8.0
; npm local prefix = /Users/myname/git/tiny-server-mock
; npm version = 10.2.0
; cwd = /Users/myname/git/my-project
; HOME = /Users/myname
; Run `npm config ls -l` to show all defaults.

[ljharb]

This isn’t a reasonable expectation. npm doesn’t hook into git, so when you change branches, it’s your responsibility to run npm install or npm ci to ensure the proper dependencies are installed.

[JustAnotherJavaProgrammer]

Since I'm changing the package.json and then running npm install, I would expect npm to detect the change in the version of the dependency (doesn't correspond with package-lock.jsonanymore) and install the version from the other branch.

I don't expect it to automatically run npm install for me when I change the file or update the code from the branch in node_modules when there are new changes on the remote.

[ljharb]

Ah, ok so this doesn’t have anything to do with git, it’s that you’re manually making a change to package.json and not the lockfile, which confuses npm?

Why are you manually making a change to package.json instead of letting npm do it?

[JustAnotherJavaProgrammer]

Yes, I only change package.json, nothing else and then running npm i.

In this particular case, I manually changed package.json because it seemed simpler to just edit a line in a file, type letters in the terminal and hit enter than to type npm i , copy part of the line from package.json and then change the last few characters of that only to get the same result (which I would have expected).
Apart from that, a similar situation could occur when I have a project checked out and switch branches, each one depending on a different branch. I would expect npm install or at least deleting node_modules and then installing to bring package.json and the installed branch in sync, but it doesn't.

I have also just created this demonstration repository with simple instructions to reproduce in the readme file: https://github.com/JustAnotherJavaProgrammer/NpmBugTestRepo

[ljharb]

I agree that just switching branches between two pairs of in-sync package.jsons and lockfiles should require nothing more than an npm install or npm ci to get proper deps installed.

[wraithgar]

Are the version numbers the same? Between the two sources? This is the signal npm uses to decide if something in the tree is correct. If they match, how would npm know the contents of node_modules differed?

[JustAnotherJavaProgrammer]

Yes, the version number of the required package is the same on both branches.

In that case, I would have hoped for deleting node_modules to fix this. But instead, the same files are installed again.

[ljharb]

@wraithgar I would assume npm does something smarter than just checking the version, like hashing the package.json contents or something.

[wraithgar]

No, npm only ever uses the version. That has historically been the atomic identifier for a given package.

[wraithgar]

Going to close this as "working as intended" Unless npm sees the version is different than what your tree wants, it will consider what's in there to be valid.