gyp: binding.gyp not found (...) while trying to load binding.gyp

Question

gyp: binding.gyp not found (...) while trying to load binding.gyp

PowerSupply opened this issue 2 years ago · 34 comments

npm i @parcel/watcher fails with the error in the title when trying to install right after npm init. I'm sure this has something to do with my environment but I do not understand what is wrong.

node: 18.15.0 (installed via nvm)
npm: 9.5.0
Python: 3.10.4
OS: macOS Ventura 13.4
Computer: Apple M1 Macbook Pro, 16-inch 2021

Full error:

npm ERR! code 1
npm ERR! path /Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.3.1
npm ERR! gyp info using node@18.15.0 | darwin | arm64
npm ERR! gyp info find Python using Python version 3.10.4 found at "/Users/myuser/.pyenv/versions/3.10.4/bin/python3"
npm ERR! gyp info spawn /Users/myuser/.pyenv/versions/3.10.4/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/myuser/Library/Caches/node-gyp/18.15.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/Users/myuser/Library/Caches/node-gyp/18.15.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/Users/myuser/Library/Caches/node-gyp/18.15.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher) while trying to load binding.gyp
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:325:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Darwin 22.5.0
npm ERR! gyp ERR! command "/Users/myuser/.nvm/versions/node/v18.15.0/bin/node" "/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher
npm ERR! gyp ERR! node -v v18.15.0
npm ERR! gyp ERR! node-gyp -v v9.3.1
npm ERR! gyp ERR! not ok

Answer 1 · 2023-09-13T16:58:17.000Z

Having the same issue since at least Aug 16:
node: v18.17.1
npm: v9.6.7
Tried downgrading to node 16.15 (npm 8.11), same error.
Also tried forcing @parcel/watcher version to 2.1 and 2.3.

Answer 2 · 2023-09-15T10:56:24.000Z

In my case @parcel/watcher is a direct dependency of another package. Solved by fixing version on that dependency:

"overrides": {
"package-that-uses-parcel": {
"@parcel/watcher": "2.1.0"
}
}

Answer 3 · 2023-09-21T07:50:13.000Z

@devongovett

Hi, I use nuxt which uses this package.

I'm using nuxt behind a proxy and in a node18-alpine3.17 docker container.

Since version 2.2.0, the "npm install" command, when it arrives at the @parcel/watcher module, tries to compile using python and gyp which itself tries for an unknown reason to download node despite the fact that it is already available locally since I'm using a node container.

My proxy refuses the download.

The problem is also present in all other docker images, whether node20-alpine or node20-debian.

The temporary solution is r-r-lima's answer, to do an ovverides to version 2.1.0

Is it possible to resolve this problem?

Answer 4 · 2023-09-28T14:15:03.000Z

I have the same exact error when installing Nuxt.

Answer 5 · 2023-10-02T07:39:57.000Z

So, is anything happening with this?

Answer 6 · 2023-10-02T07:51:10.000Z

Someone needs to investigate why the prebuilds (e.g. @parcel/watcher-darwin-arm64) are not working sometimes. I can't reproduce it on my machine

In https://unpkg.com/browse/@parcel/watcher@2.3.0/index.js, print out the err thrown by binding = require(name);.

Answer 7 · 2023-10-03T19:43:31.000Z

I have the same exact error when installing Nuxt.

Answer 8 · 2023-10-06T12:35:48.000Z

This looks suspiciously like the infamous npm lockfile bug - see npm/cli#2606 - possible workaround in npm/cli#2606 (comment)

Answer 9 · 2023-10-07T11:21:05.000Z

I've had similar problems. I had to use "@parcel/watcher": "~2.1.0" in my project devDependencies to avoid problems with 'gyp: binding.gyp not found' on both Windows 10 with npm 18 or 16 and Python 3.11, and MacOS 12.6.5 with npm 18 or 16 and Python 3.10. The problems seem to start in "@parcel/watcher": "~2.2.0" with its "node-addon-api": "^7.0.0" dependency, whereas "@parcel/watcher": "~2.1.0" with node-addon-api: ^3.2.1 and node-gyp-build: ^4.3.0 dependencies works.

Answer 10 · 2023-10-13T21:14:58.000Z

Solution temp

package.json

"overrides": {
    "@parcel/watcher": "2.1.0"
}

Answer 11 · 2023-12-01T14:00:15.000Z

Experiencing the same while attempting an npm install on a Nuxt 3 project. A few others in my team are also experiencing this but some are not.

Answer 12 · 2023-12-08T16:56:29.000Z

+1 npx nuxi init (Nuxt 3 app generator) fails due this error. Package override works as a temp fix.

Answer 13 · 2023-12-24T12:34:33.000Z

+1 npm install -D parcel

node: 21.4.0 (installed via apk)
npm: 10.2.5
Python: 3.11.6
OS: Alpine Linux 3.19 x86-64

Answer 14 · 2024-01-26T08:12:02.000Z

+1 Updated package.lock in nuxt 3 and thought it’s the infamous npm cache permissions bug. But even after chowning and reinstalling node, the error stayed. Turns out it was due to this package.

Tested overrides with all sorts of in-between versions. The first version to work is 2.1.0

Env:
Node 21.6
OS macOS Sonoma
Arch arm64 (M1 Pro)

Answer 15 · 2024-01-28T13:45:48.000Z

Can confirm that this causes fresh nuxt installations to fail when using npx nuxi init.
Quite annoying for people that are new to the framework since they won't know they have to override the package.json to fix it

Answer 16 · 2024-01-28T19:21:23.000Z

Can reproduce this issue as well when running npx nuxi@latest init.

$ node --version
v20.10.0
$ node-gyp --version
v10.0.1
$ npm --version
10.4.0
$ uname -a
Linux 6.7.1-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 21 Jan 2024 22:14:10 +0000 x86_64 GNU/Linux

Running with the override worked as expected.

Answer 17 · 2024-01-29T10:07:50.000Z

I ran npm ls @parcel/watcher and it looks like nitropack is using a dependency which is causing the issue.

  ├── @parcel/watcher@2.1.0 overridden
  └─┬ nitropack@2.8.1
    └─┬ listhen@1.6.0
      └── @parcel/watcher@2.1.0 deduped

Answer 18 · 2024-01-31T20:01:45.000Z

Same here. My workaround:

  "overrides": {
    "@parcel/watcher": "~2.1.0"
  }

Answer 19 · 2024-02-03T23:02:12.000Z

"@parcel/watcher": "2.1.0"

First time trying Nuxt today, and I get this :(

Answer 20 · 2024-02-05T12:56:20.000Z

Someone needs to investigate why the prebuilds (e.g. @parcel/watcher-darwin-arm64) are not working sometimes. I can't reproduce it on my machine

In https://unpkg.com/browse/@parcel/watcher@2.3.0/index.js, print out the err thrown by binding = require(name);.

@mischnic Here are 2 Dockerfiles for reproducing the error.

Failing Dockerfile

FROM ubuntu:latest

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  apt-get update \
  && apt-get install -y --no-install-recommends \
    curl \
    gnupg \
    ca-certificates \
    wget \
    g++ \
  && curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
  && apt-get update \
  && apt-get install -y --no-install-recommends nodejs \
  && npm install -g npm@latest \
  && rm -rf /var/lib/apt/lists/*

WORKDIR /home

Build and run:

docker build -t npm-20.
docker run -it --rm npm-20

Then, inside the running container:

npm i -g @parcel/watcher

❌ Error:

npm ERR! gyp: binding.gyp not found (cwd: /usr/lib/node_modules/@parcel/watcher) while trying to load binding.gyp

Working Dockerfile

FROM ubuntu:latest

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  apt-get update \
  && apt-get install -y --no-install-recommends \
    curl \
    gnupg \
    ca-certificates \
    wget \
    g++ \
  && curl -fsSL https://deb.nodesource.com/setup_18.x | bash - \
  && apt-get update \
  && apt-get install -y --no-install-recommends nodejs \
  && rm -rf /var/lib/apt/lists/*

WORKDIR /home

Build and run:

docker build -t npm-18 .
docker run -it --rm npm-18

Then, inside the running container:

npm i -g @parcel/watcher

✔️ Result:

added 13 packages in 3s

Answer 21 · 2024-02-05T13:16:34.000Z

I tried some more combinations of Node and npm:

	Node 18.19.0	Node 20.11.0
npm v10.2.3	✔️	✔️
npm v10.2.4	✔️	✔️
npm v10.2.5	✔️	✔️
npm v10.3.0	✔️	✔️
npm v10.4.0	❌	❌

Answer 22 · 2024-02-05T14:45:44.000Z

Can reproduce this with 'npm install' in a SolidJS project.

$ node –version
v20.11.0
$ node-gyp –version
v10.0.1
$ npm –version
10.4.0
$ uname -a
Linux 6.7.3-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 01 Feb 2024 10:30:35 +0000 x86_64 GNU/Linux

[host@myhost site]$ npm install
npm ERR! code 1
npm ERR! path /home/host/site/node_modules/@parcel/watcher
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@10.0.1
npm ERR! gyp info using node@20.11.0 | linux | x64
npm ERR! gyp info find Python using Python version 3.11.6 found at "/sbin/python3"
npm ERR! gyp info spawn /sbin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/host/site/node_modules/@parcel/watcher/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/host/.cache/node-gyp/20.11.0/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/home/host/.cache/node-gyp/20.11.0',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/home/host/.cache/node-gyp/20.11.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/home/host/site/node_modules/@parcel/watcher',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /home/host/site/node_modules/@parcel/watcher) while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: gyp failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess. (/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:271:18)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:518:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
npm ERR! gyp ERR! System Linux 6.7.3-arch1-1
npm ERR! gyp ERR! command "/home/host/.nvm/versions/node/v20.11.0/bin/node" "/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/host/site/node_modules/@parcel/watcher
npm ERR! gyp ERR! node -v v20.11.0
npm ERR! gyp ERR! node-gyp -v v10.0.1
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /home/host/.npm/_logs/2024-02-05T14_27_16_860Z-debug-0.log

Answer 23 · 2024-02-06T23:08:47.000Z

I tried some more combinations of Node and npm:
Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

As found out by vincesp, downgrading npm to v10.3.0 on Node 18.17.1 did help!
I ran npm install -g npm@10.3.0 for the downgrade just fyi.

Thanks!

Answer 24 · 2024-02-07T10:19:05.000Z

There is another combination that does not work: I did not find any npm version that can install @parcel/watcher successfully when using Artifactory 6 as an intermediate proxy repository. So if you have an .npmrc file like the following:

❌ Fails (even with npm v10.3.0):

registry=https://myartifactory6server.mycompany.com/artifactory/api/npm/NPM-REMOTE/
always-auth=true

We have to switch off the proxying explicitly for namespace @parcel but that means we lose the record in our own repo.

✔️ Works (with npm v10.3.0):

registry=https://myartifactory6server.mycompany.com/artifactory/api/npm/NPM-REMOTE/
@parcel:registry=https://registry.npmjs.org/
always-auth=true

Answer 25 · 2024-02-11T20:52:25.000Z

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

Confirming the downgrade to npm v10.3.0 fixed the issue for me while using Node v20.11.0

Was on npm v10.4.0 and trying to set up Analog.js using npm install @analogjs/platform --save-dev and ran into this error:

npm ERR! code 1
npm ERR! path /Users/<redacted>/Projects/<redacted>/website/node_modules/@parcel/watcher
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@10.0.1
npm ERR! gyp info using node@20.11.0 | darwin | x64
npm ERR! gyp info find Python using Python version 3.7.3 found at "/Library/Developer/CommandLineTools/usr/bin/python3"

After downgrading, the install completed successfully.

Answer 26 · 2024-02-14T06:02:25.000Z

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

It works thanks!

Answer 27 · 2024-02-17T20:07:17.000Z

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

node 21.6.1
npm 10.3.0

It worked for me

Answer 28 · 2024-02-20T12:15:32.000Z

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

node: 20.11.0
npm: 10.2.4, 10.4.0
Mac m1 Max: Sonoma 14.3.1

Didn't work for me.

What worked for me is specifying 2.1.0 as a version. 2.2.0, 2.3.0 and 2.4.0 didn't work at all.

Answer 29 · 2024-02-23T00:36:45.000Z

I don't understand what we are supposed to do about this issue. Appears to be common but I don't think it's our fault? Seems like a bug in npm? Is that the consensus here? Has anyone reported this to npm yet? Does it work with other package managers (e.g. yarn/pnpm)?

Answer 30 · 2024-02-23T06:43:48.000Z

I don't understand what we are supposed to do about this issue. Appears to be common but I don't think it's our fault? Seems like a bug in npm? Is that the consensus here? Has anyone reported this to npm yet? Does it work with other package managers (e.g. yarn/pnpm)?

Unfortunately, our solution was to switch to watcher (just watcher without "@parcel/"). But that only worked because @parcel/watcher is not the dependency of a dependency.

Answer 31 · 2024-02-23T08:59:11.000Z

@devongovett PR #144 removed the install script from package.json as part of v2.2.0 however the npm registry still reports an install script:

$ npm show @parcel/watcher@2.1.0 --json | grep install
    "install": "node-gyp-build",
$ npm show @parcel/watcher@2.2.0 --json | grep install
    "install": "node-gyp rebuild"
$ npm show @parcel/watcher@latest --json | grep install
    "install": "node-gyp rebuild"

I think this erroneous install script is being injected at npm publish time because npm sees a binding.gyp file (thanks to the logic in @npmcli/node-gyp), however this file isn't published.

A few possible solutions, from hardest to easiest:

Fix @npmcli/node-gyp so it checks to see if binding.gyp will actually be published (e.g. look in files or .npmignore).
Move all the src and related build files, including binding.gyp, out of the root directory of this repo so it no longer looks like a native module (I did this for sharp, it works a treat, removes a whole class of error in most Electron tooling as well).
Remove the binding.gyp file from the filesystem just before running npm publish.

npm v10.4.0 introduced npm/run-script#192 which completely changes the way that package.json files are parsed at npm install time and is likely to have exposed this issue.

Answer 32 · 2024-02-23T15:38:57.000Z

Great info @lovell! Thanks so much for sharing. It's interesting because this seems to have been happening even before npm 10.4 Perhaps that's actually a different issue (with lockfiles?) though.

I updated our publishing workflow to copy the files needed into a separate directory and run npm publish from there. That way binding.gyp doesn't get added by npm (even though it already wasn't included in package.json files).

In a related change, I also added better error handling to the package so that if an error occurs while requiring the native module, we only continue trying fallback locations if the error was a module not found error. Other errors (e.g. incompatibilities with OS versions) are re-thrown so they are reported to users.

These changes are published in v2.4.1. I tested locally with npm 10.4.0 and was able to install the package successfully. Please let me know if you continue to run into issues.

Answer 33 · 2024-03-11T18:53:06.000Z

only V.10.40 worked for me

Answer 34 · 2024-07-24T16:41:34.000Z

For anyone experiencing this issue on Ubuntu 22.04 ARM64 and Node 16.20.1 installed as a snap (I need to re-package legacy stuff, don't ask), 2.2.0 is the first version I found builds fine:

// Add this to package.json
"overrides": {
    "@parcel/watcher": "2.2.0"
}