fails to invoke npm when used in run-scripts

Question

fails to invoke npm when used in run-scripts

electriquo opened this issue 2 years ago · 32 comments

Environment

Platform: MacOS
Docker Version: 20.10.16
Node.js Version: 16.15.1
Image Tag: node:16.15.1-alpine

Expected Behavior

Given the following package.json

{
  "name": "test",
  "scripts": {
    "ver": "npm --version"
  },
  "devDependencies": {
    "cowsay": "^1.5.0"
  }
}

$ npm run ver

> ver
> npm --version

8.12.1

The same behavior should be replicated within a container runtime

Current Behavior

When running within a container runtime, npm fails and returns 243 exit code

$ docker run --rm -v $PWD:/app --workdir /app --entrypoint npm node:16.15.1-alpine run ver

> ver
> npm --version


npm notice
npm notice New minor version of npm available! 8.11.0 -> 8.12.1
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v8.12.1>
npm notice Run `npm install -g npm@8.12.1` to update!
npm notice

$ echo $?
243

When using a different image, it works as expected

$ docker run --rm -v $PWD:/app --workdir /app --entrypoint npm node:14-alpine run ver

> test@ ver /app
> npm --version

6.14.17

It works on on some images (note the --user node:node option)

$ docker run --rm -v $PWD:/app --workdir /app --entrypoint npm node:16.15.0-alpine3.15@sha256:bb776153f81d6e931211e3cadd7eef92c811e7086993b685d1f40242d486b9bb run ver

> ver
> npm --version

8.5.5
npm notice
npm notice New minor version of npm available! 8.5.5 -> 8.12.1
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v8.12.1>
npm notice Run `npm install -g npm@8.12.1` to update!
npm notice

$ docker run --rm -v $PWD:/app --workdir /app --entrypoint npm --user node:node node:18-alpine run ver

> ver
> npm --version

8.11.0

when the node user is used, then there a permission issue when mount volumes are used.

$ docker run --rm -it -v $PWD:/src -v node_modules:/src/node_modules --workdir /src --user node:node --entrypoint /bin/sh node:lts-alpine -c 'npm install && npm run ver'
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /src/node_modules/ansi-regex
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/src/node_modules/ansi-regex'
npm ERR!  [Error: EACCES: permission denied, mkdir '/src/node_modules/ansi-regex'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/src/node_modules/ansi-regex'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/node/.npm/_logs/2022-06-09T07_10_27_665Z-debug-0.log

$ docker run --rm -it -v $PWD:/src -v node_modules:/src/node_modules --workdir /src --user root:root --entrypoint /bin/sh node:lts-alpine -c 'npm install && npm run ver'

added 41 packages, and audited 42 packages in 2s

3 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
npm notice
npm notice New minor version of npm available! 8.11.0 -> 8.12.1
npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.12.1
npm notice Run npm install -g npm@8.12.1 to update!
npm notice

> ver
> npm --version

Steps to Reproduce

Use the package.json configuration above and repeat the commands in this post.

Answer 1 · 2022-06-07T20:43:12.000Z

Seeing the same issue in my Jenkins pipeline with node:16.15-alpine3.15. I cannot reproduce the issue locally though.

Answer 2 · 2022-06-08T09:58:23.000Z

we are facing the same issue.

on k8s we're setting the pod security context to run with an arbitrary user id, after deleting this in the config the pod starts again.

we are setting the tag / sha of images to node:16.14 for now, to keep our pod settings.

also see npm/cli#4996

Answer 3 · 2022-06-08T14:20:31.000Z

Also seeing this issue, hard coding to node:16.15.0-alpine in the meantime.

Answer 4 · 2022-06-10T19:56:55.000Z

we are facing the same issue.

on k8s we're setting the pod security context to run with an arbitrary user id, after deleting this in the config the pod starts again.

we are setting the tag / sha of images to node:16.14 for now, to keep our pod settings.

also see npm/cli#4996

This is our circumstance as well. We use an arbitrary non-privileged user ID which worked fine until v16.15.1.

Answer 5 · 2022-06-14T06:36:49.000Z

Maybe @nschonni knows what's up

Answer 6 · 2022-06-15T10:04:17.000Z

Facing the same issue.

Was using node:16-slim which brought in version 16.15.1-slim. So reverted back to previous minor version as a temporary fix by specifying node:16.14-slim

Answer 7 · 2022-06-22T18:41:13.000Z

Facing the same issue.

Was using node:16-slim which brought in version 16.15.1-slim. So reverted back to previous minor version as a temporary fix by specifying node:16.14-slim

For us this was specifically with the 16.15.1 patch release, 16.15.0 works in our case.

Answer 8 · 2022-06-27T17:39:21.000Z

I'm having a similar issue. We're seeing it when we try to do an npm install from the Docker command, when we set the user for the container. If you set the node cache to a non-existent path, it seems to work.

Can replicate it using a simple package.json file with a dependency. This example assumes the package.json is owned by user 1001.

> docker run --rm -it -v $PWD:/app -u=1001 -w /app node:16.15.1-alpine sh -c 'npm i'
> echo $?
243

When setting --cache=nope, the install works:

> docker run --rm -it -v $PWD:/app -u=1001 -w /app node:16.15.1-alpine sh -c 'npm i --cache=nope'
... added some packages
> echo $?
0

EDIT:

It appears (at least in part) to do with the home directory for the user it's running as. Running as user 1001 (which doesn't exist in the container) fails to run npm at all:

> docker run --rm -it -u 1001 node:16.15.1-alpine sh
> npm -v

> echo $?
243

If you add the user with that ID, it creates the home directory and works:

> docker run --rm -it node:16.15.1-alpine sh
> adduser -D -u 1001 test
> su test
> npm -v
8.11.0
> echo $?
0

If you add the user with that ID, but without the home directory, you get the same error:

> docker run --rm -it node:16.15.1-alpine sh
> adduser -D -H -u 1001 test
> su test
> npm -v

> echo $?
243

Answer 9 · 2022-06-28T09:05:59.000Z

It seems that npm exec and other command-invoking command such as npx executes commands as the owner of the current working directory, regardless of the user that these commands executed.

user:~$ npm exec -- /usr/bin/whoami
user
user:~$ sudo npm exec -- /usr/bin/whoami
user
user:~$ cd /
user:/$ npm exec -- /usr/bin/whoami
user
user:/$ sudo npm exec -- /usr/bin/whoami
root

Edit: here overrides uid if root

https://github.com/npm/cli/blob/9f94049f058687b916da726ea625b5fa68d0829d/node_modules/%40npmcli/promise-spawn/lib/index.js#L13

Answer 10 · 2022-06-29T13:41:34.000Z

Probably the same problem with docker image of 18.2-alpine, npm version 8.9.0.

{
  "name": "test-app",
  "version": "0.0.0",
  "scripts": {
    "test": "npm --version"
  },
  ...
}

The issue here seems npm doesnt want to run as root. Example when container is run as root and when switched to user node using su:

1aae0dcc8624:/workspace/app# whoami
root
1aae0dcc8624:/workspace/app# npm run test

> test-app@0.0.0 test
> npm --version


1aae0dcc8624:/workspace/app# su node
1aae0dcc8624:/workspace/app$ whoami
node
1aae0dcc8624:/workspace/app$ npm run test

> test-app@0.0.0 test
> npm --version

8.9.0
1aae0dcc8624:/workspace/app$

Running the container as node fixes the issue.

Answer 11 · 2022-06-29T14:59:49.000Z

@SmallhillCZ

Running the container as node fixes the issue.

Not fully correct. It depends whether you are using the container in conjunction with mount/data volumes as node_modules directory. When it is used, the user must be root in most cases (depends on the inner implementation of how mount volumes work across different operating systems)

Answer 12 · 2022-07-06T05:28:21.000Z

The issue exists in other images too and seems like the maintainers/contributors are not a part of the thread discussion.

Do we know what is the cause for the issue?
Is anyone working on a fix?

Answer 13 · 2022-07-10T19:18:12.000Z

@foolioo
NPM overrides uid if current user is root in the following line.

https://github.com/npm/cli/blob/9f94049f058687b916da726ea625b5fa68d0829d/node_modules/%40npmcli/promise-spawn/lib/index.js#L13

This was removed before, but reintroduced now.

npm/promise-spawn#6

Answer 14 · 2022-07-10T19:28:44.000Z

@yuki-js Thanks. Do you have any suggestion how to resolve the issue?

Answer 15 · 2022-07-10T19:35:21.000Z

@foolioo There seems no fundamental solution for now. Adjust uid of all files, or use yarn instead.

Answer 16 · 2022-07-10T19:40:22.000Z

For me, problems disappear when I add USER node command before invoking my npm run ... command inside my Dockerfile.

Example:

# Multi-stage dockerfile:
# https://docs.docker.com/develop/develop-images/multistage-build/

# Base stage
FROM node:16.15.1-slim AS base
EXPOSE 8080
RUN mkdir -p /app && chown -R node:node /app
WORKDIR /app
USER node

# Builder stage
FROM base AS builder
COPY --chown=node:node . .
RUN npm ci
RUN npm run build

# Prod stage
FROM base AS prod
COPY --from=builder --chown=node:node /app/dist ./dist
CMD ["node", "dist"]

Answer 17 · 2022-07-10T19:40:46.000Z

@yuki-js Have you tried adding a user to match the mounted file permissions, and using that to use npm? Obviously it isn't ideal, but it worked for our use case. (See my comment above). Can see how it works with our setup here: https://github.com/7digital/mysql2-timeout/blob/dbc820bcdd7a638e1f02bf983beb6bb52059d07f/docker-compose.yml#L12

Answer 18 · 2022-07-10T19:48:54.000Z

@scooper91 I know, but unfortunately it doesn't fit my use case.

Answer 19 · 2022-07-13T08:41:33.000Z

The issue still persists with Ubuntu:20.04 image and Node 16.x LTS installed.

Answer 20 · 2022-07-14T09:38:01.000Z

In my case, I use a docker-compose.yml as is:

version: "3"
services:
  npm:
    build: ./docker/node/16
    entrypoint: npm
    user: node
    volumes:
      - .:/home/node
    working_dir: /home/node

And I had this "243" error (which a few months ago with the exact same docker-compose setup I hadn't).

First I tried to upgrade to node 18, but the error was the same. Then I tried to specify the group of the user from "node" to "node:node". Same error.

Then I remembered sometimes on my local there would be an error about writing on the ".npm" folder because of lack of permissions. So I figured I'd had the volume so the npm script would not have to write it on the host folder system (the one on Github must have reduced folder permissions for security purposes).

version: "3"
services:
  npm:
    build: ./docker/node/16
    entrypoint: npm
    user: node:node
    volumes:
      - .:/home/node
      - ./docker-data/npm:/.npm # <--
    working_dir: /home/node

Then the CI on Github started to be more eloquent, this time it would not manage to write a log file on the ".npm/_logs" folder. Getting close:

npm WARN logfile could not be created: Error: EACCES: permission denied, open '/home/node/.npm/_logs/2022-07-14T09_18_36_207Z-debug-0.log'

So I decided to also create the "_logs" folder on my "docker-data/npm" folder, so the npm script would not have to create the folder as well. Here is my folder structure:

my-app/
├── docker/
│   └── 18/
│       └── Dockerfile
└── docker-data/
    ├── npm/
    │   └── _logs/
    │       └── .gitignore
    └── .gitignore

And I also had to remove the "user: node:node" by the way (not sure if I put it back it will still work):

version: "3"
services:
  npm:
    build: ./docker/node/16
    entrypoint: npm
    volumes:
      - .:/home/node
      - ./docker-data/npm:/.npm
    working_dir: /home/node

With this setup, running this command will work without 243 errors

docker-compose run --rm npm outdated

As well as installing dependencies:

docker-compose run --rm npm ci

But compiling my assets will fail (this uses Laravel Mix behind the scene)

Run docker-compose -f app/docker-compose.yml run npm run prod
  docker-compose -f app/docker-compose.yml run npm run prod
  shell: /usr/bin/bash -e {0}
Creating app_npm_run ... 
Creating app_npm_run ... done

> prod
> npm run production

npm notice 
npm notice New minor version of npm available! 8.12.1 -> 8.14.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v8.14.0>
npm notice Run `npm install -g npm@8.14.0` to update!
npm notice 

Error: Process completed with exit code 243.

Edit

I managed to fix all permission issues by starting from Ubuntu:22.04 instead of Node:18-alpine on my Dockerfile

I recap for folks that want to do the same

docker-compose.yml

npm:
    build: ./docker/node/18
    entrypoint: npm
    volumes:
      - .:/home/node
    working_dir: /home/node

docker/node/18/Dockerfile

FROM ubuntu:22.04

RUN apt-get update && \
    apt-get upgrade --yes && \
    apt-get install --yes \ // Remove g++ and others dependencies if you don't need them, I needed them to compile some NPM deps
    g++ \
    make \
    curl \
    bash && \
    curl -fsSL https://deb.nodesource.com/setup_18.x | bash - && \
    apt-get install --yes nodejs

Now all theses commands on my CI works without perm issues

docker-compose run npm ci
docker-compose run npm run prod
docker-compose run npm outdated

Answer 21 · 2022-07-19T07:31:01.000Z

I've added an entrypoint script to all project that suffer from this issue.
It solved the permissions issues of a Docker mount volumes and NPM itself

$ cat docker-compose.yaml
version: '3.9'
services:
  node:
    container_name: node
    image: node:lts-alpine
    user: root:root
    entrypoint: /src/entrypoint.sh node
    volumes:
      - .:/src
      - node_modules:/src/node_modules
    working_dir: /src
    environment:
      - HOST_UID=${HOST_UID:-1000}

volumes:
  node_modules:

$ cat entrypoint.sh
#!/usr/bin/env sh

set -eu

apk add --update --no-cache --no-progress --quiet shadow
usermod -u 1000 node >/dev/null 2>&1
usermod -u "$HOST_UID" node >/dev/null 2>&1
chown node:node node_modules

CMD="$*"
su node -c "$CMD"

And have an environment variable named HOST_UID that holds the current user ID. For instance export HOST_UID=$(id -u)

Answer 22 · 2022-08-23T06:07:25.000Z

Hi!
What's the status, is the problem fixed on latest node:16-alpine image?

Answer 23 · 2022-09-14T12:46:30.000Z

The simple solution is to change the nodejs version. I was facing the same issue, i fixed it by changing node version from 16 to 14 and this worked

Answer 24 · 2022-09-16T16:12:42.000Z

Simply adding user: node to my docker-compose.yaml fixed it in my case...

Answer 25 · 2022-10-10T10:20:56.000Z

This is still happening on 16.17.1 docker images.

Answer 26 · 2022-10-10T19:35:06.000Z

Actually, this is not only an issue with run-scripts. node:16-alpine docker image is completely broken as a base build image in these cases:

Environments where image is run by root user.
Read-only root filesystem environments, e.g., Kubernetes container with readOnlyRootFilesystem: true.
Environments where one could not modify a running user, e.g., on Jenkins pipelines which by default run under Jenkins user and the UID is most probably not the same as UID 1000 which is used in the image.

The source of these issues is that npm_config_prefix is set to /usr/local and npm_config_userconfig is not set at all, so NPM is trying to use HOME folder which in the docker image is set to /. Again, that folder is not writable by any of the users except root, but remember, NPM ignores root user due to https://github.com/npm/cli/blob/37bf6d31bd2f68e661928744833b57dcabbb0d1a/node_modules/@npmcli/promise-spawn/lib/index.js#L13

The only way I found to fix this on Jenkins is set NPM_CONFIG_CACHE to something like /tmp/jenkins/.npm. GitLab/GitHub runners can be fixed in similar manner.

I guess the final fix should be to create .npm cache folder somewhere writable by any user on the docker image, so at least it works by default on CI pipelines. Maybe create some kind of documented folder specified via npm_config_userconfig so anyone can map it and modify the behaviour of a running user.

P.S. Why NPM tries to spawn a process according to permission on current folder in case it is run by the root user is still a mistery to me. It goes against all commons. Programs should not modify running user.

Answer 27 · 2022-10-10T19:36:45.000Z

https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#non-root-user

Answer 28 · 2022-10-10T19:40:25.000Z

https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#non-root-user

Yes, I saw that, but this assumes that Docker image can be modified. In a lot of cases, you cannot do that. Free GitLab runners are one such example.

Answer 29 · 2022-10-11T08:55:20.000Z

I confirm, the issue is still exist in latest 16.17.1 docker images when we run them on Openshift/Kubernetes.
We need to run latest node 16 version images because of security policy we have!
Node 16.15.0 Docker image worked before for us just fine. Now something is changed/broken on later releases.

Seems issue is so old already on nowbody cares to fix it 👎

Answer 30 · 2022-10-11T08:58:01.000Z

cares ro fix it

cares to fix it

Answer 31 · 2023-02-06T17:51:52.000Z

So I encountered what I believe is this problem (although not sure of the fix yet). Essentially experiencing the exact same behavior. It's probably the most confounding problem I've ever had to troubleshoot because (and I mean this sincerely) "the compose-up command was working with the image last week, and not today", with literally nothing changed in the dockerfile or compose file .. I started trying to re-build the image as fresh as possible (--no-cache, etc) .. and then just started deleting images off docker desktop as verbose build logging still showed a few layers being "CACHED" but I might misunderstand that as it was referencing the node:18.13-alpine image itself.
Anyway.. I was able to get it working both inside the docker desktop GUI (play button/no args) (working btw for us means that NestJS runs the 'start --watch' command (via 'npm run start:dev') starting a NestJS dev server). And then (again without changing a single thing except the final CMD to be "bin/sh" instead of what it was (working with) last week which was ["npm", "run", "start:dev" ] then running 'npm install' again which updated 0 things (as there is already that command further up in the dockerfile.. but just to be sure ran it again .. then ran 'npm run start:dev' and lo and behold it started the nest dev server like it did last week.. without changing anything other than deleting old images using node:18.13-alpine off my system

To add more context as one person mentioned in their reply: The docker compose that ran (and worked last week but not today) did create a volume to the pwd (the app directory) and so I assume there would be node_modules in that mount as my local machine is the 'root' for this project's initial build and docker development environment creation

So here I am more confused than at the start as to what in the heck is going on here.. and can't wait for one of the other devs to have the same problem so I can say "oh yeah .. just start deleting things and starting them and clicking.. it eventually works again"

Answer 32 · 2023-05-02T04:13:55.000Z

I also encountered this issue when building an Angular+nginx image. What worked for me is I included the package-lock.json to the Dockerfile.

# Set the working directory
WORKDIR /app

# Add the source code to app
COPY package.json .
COPY package-lock.json .

# Install all the dependencies
RUN npm cache clean --force

# Generate the build of the application

RUN npm ci --legacy-peer-deps