How to install further tools
kakila opened this issue · 13 comments
Some time ago I was running publish with tree in a gitlab runner.
While updating to image 7.2.0, the script stop working because I cannot install tree or update the pkg manager. The error is
$ apt-get update
Reading package lists...
E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1
How can one install additional tools in the octave images?
Thx.
Ok, I checked with older mages and the same happens (back to 6.2.0). Are the images somehow configured to prevent sudo access?
Maybe this is an issue with the gitlab.com runners ... checking...
Further investigation seems to indicate that the issue is the image:
Using docker image sha256:192fe8f5da730ae51c2b9b3b259e9dfbafd465e23959eb5ac971f6e2356f448f for gnuoctave/octave:7.2.0 with digest gnuoctave/octave@sha256:eb28ca9a4477fc95af628101e3db975d3f73e134117b4fc51aef9909d037eaca ...
Entered start.sh with args: sh -c if [ -x /usr/local/bin/bash ]; then
exec /usr/local/bin/bash
elif [ -x /usr/bin/bash ]; then
exec /usr/bin/bash
elif [ -x /bin/bash ]; then
exec /bin/bash
elif [ -x /usr/local/bin/sh ]; then
exec /usr/local/bin/sh
elif [ -x /usr/bin/sh ]; then
exec /usr/bin/sh
elif [ -x /bin/sh ]; then
exec /bin/sh
elif [ -x /busybox/sh ]; then
exec /busybox/sh
else
echo shell not found
exit 1
fi
Executing the command: sh -c if [ -x /usr/local/bin/bash ]; then
exec /usr/local/bin/bash
elif [ -x /usr/bin/bash ]; then
exec /usr/bin/bash
elif [ -x /bin/bash ]; then
exec /bin/bash
elif [ -x /usr/local/bin/sh ]; then
exec /usr/local/bin/sh
elif [ -x /usr/bin/sh ]; then
exec /usr/bin/sh
elif [ -x /bin/sh ]; then
exec /bin/sh
elif [ -x /busybox/sh ]; then
exec /busybox/sh
else
echo shell not found
exit 1
fi
$ whoami
jovyan
I was expecting to see gitlab-runner as the output of whoami
Hi @kakila ,
Thanks for your report. In general the Docker images are root-less, but can be started granting elevated privileges adding the two parameters --user root --env="GRANT_SUDO=yes". This logic is copied from how Jupyter-Stack images are working. For example:
docker run -it --user root --env="GRANT_SUDO=yes" --rm gnuoctave/octave:7.2.0 bash
Entered start.sh with args: bash
Granting jovyan passwordless sudo rights!
Running as jovyan: bash
jovyan@daa2178a1ff1:~$ sudo apt-get update
Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:3 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [4,644 B]
Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [436 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [114 kB]
Get:6 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [393 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [99.8 kB]
Get:8 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [354 kB]
Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1,792 kB]
Get:11 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:12 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [767 kB]
Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [534 kB]
Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [8,056 B]
Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [438 kB]
Get:17 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [7,267 B]
Get:18 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [3,175 B]
Fetched 23.2 MB in 7s (3,446 kB/s)
Reading package lists... Done
Does this help you and can you apply those options in your GitLab setup?
@siko1056 Thanks, I need to check how to pass these parameters... I am on it.
However, I need to point out that others images in dockerhub do not require this to work correctly in the gitlab CI/CD. Image 6.2.0 was indeed working without issues, a few years ago :D.
In the output i gave, all that printout about start.sh happes only with gnuoctave images. Can it be there is some customization that is not completely portable?
It seems there is no solution to this issue: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2750#note_1066546283
So it seems other containers run root as default. It sounds to me better to offer a almighty root in the container and let users use a different user if they want. This seems to be the case with any other image I have tried!
I am trying to use the workaround in https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2750#note_157155236
Seems other communities have faced this and gitlab is not the only service with the deficiency https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2750#note_184865038
I understand the situation. There are other ways to enable non-root containers with a less strict mechanism than Jupyter is using. I switch end of this week to that more simple logic and probably you no longer face this issue. Is that delay acceptable?
Absolutely! I cannot push you around! Thanks!
@kakila in case its helpful, here is one of my CI files (GitHub not GitLab sorry) that worksaround the jovyan stuff: https://github.com/cbm755/octsympy/blob/main/.github/workflows/main.yml
I remember it was a bit frustrating and kind of fragile so I'd also welcome some changes (but I'm not enough of a Podman/Docker expert to know what those changes are, sorry @siko1056)
Greetings @kakila and @cbm755 ,
Finally there have been major updates of the Dockerfiles which are built and online now.
I agree to you, that trying to enforce a non-root user the "Jupyter way" was not the best solution (admittedly a bit exotic) and reduced the universality of all those images. The only image to be done is "octave:jupyterlab", but the rest works fine. Just tested in Octave packages.
Your original workflow on GitLab should be back to normal @kakila ? You are root by default and basically no further options are necessary.
Also, I finally figured out who "jovyan" is! Being of Jupiter: I feel a bit sheepish but I in my defense I had no idea this Octave image was somehow related to Jupyter ;-)
@kakila I had to adopt to this change by changing references to /home/jovyan to /workdir and then I could also drop some docker ... -u root --env="GRANT_SUDO=yes" stuff I was using to make sudo work inside the image. See changes here for details: cbm755/octsympy#1241