Chrome - Failed to move to new namespace.
xcellardoor opened this issue · 104 comments
I keep getting this error when trying to run the Chrome image. If I run with the --no-sandbox argument for Chrome, it then complains about running as the root user and dies immediately. What's the solution?
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
The arguments to run in my dotfiles work for me
On Wednesday, September 23, 2015, xcellardoor notifications@github.com
wrote:
I keep getting this error when trying to run the Chrome image. If I run
with the --no-sandbox argument for Chrome, it then complains about running
as the root user and dies immediately. What's the solution?Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permitted—
Reply to this email directly or view it on GitHub
#65.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
You should not be using --no-sandbox
On Wednesday, September 23, 2015, xcellardoor notifications@github.com
wrote:
I keep getting this error when trying to run the Chrome image. If I run
with the --no-sandbox argument for Chrome, it then complains about running
as the root user and dies immediately. What's the solution?Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permitted—
Reply to this email directly or view it on GitHub
#65.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
Nope sorry still not working. I did a 'make' on your dotfiles and that all seems to have installed okay. Tried re-launching your Chrome and it failed with the same PID issue. Here is the end of installing the dotfiles and then starting the your chrome.
sudo systemctl daemon-reload
╭─cellardoor@glow ~/github/docker/dotfiles ‹master›
╰─$ sudo systemctl daemon-reload
╭─cellardoor@glow ~/github/docker/dotfiles ‹master›
╰─$ docker run -it --net host --cpuset-cpus 0 --memory 512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY --device /dev/snd jfrazelle/chrome 1 ↵
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
^C
hmmm gotta be something weird w your setup :/
On Wed, Sep 23, 2015 at 11:15 AM, xcellardoor notifications@github.com
wrote:
Nope sorry still not working. I did a 'make' on your dotfiles and that all
seems to have installed okay. Tried re-launching your Chrome and it failed
with the same PID issue. Here is the end of installing the dotfiles and
then starting the your chrome.sudo systemctl daemon-reload
╭─cellardoor@glow ~/github/docker/dotfiles ‹master›
╰─$ sudo systemctl daemon-reload
╭─cellardoor@glow ~/github/docker/dotfiles ‹master›
╰─$ docker run -it --net host --cpuset-cpus 0 --memory 512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY --device /dev/snd jfrazelle/chrome 1 ↵
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
^C—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
It's a relatively standard Arch Linux machine, nothing particularly weird about it unfortunately :(
You're not running as a special user are you? Online I've read it's possible to use --no-sandbox to remove the PID error but of course you're forgoing basically everything useful about sandboxing when you do that :| It then says I'm trying to run it as Root too, if using sandboxing. Are you invoking any commands to have it run as a regular user? Is your Docker daemon itself configured differently to stock perhaps?
Thanks.
i use everything in my dotfiles its a standard docker run
On Wed, Sep 23, 2015 at 11:26 AM, xcellardoor notifications@github.com
wrote:
It's a relatively standard Arch Linux machine, nothing particularly weird
about it unfortunately :(You're not running as a special user are you? Online I've read it's
possible to use --no-sandbox to remove the PID error but of course you're
forgoing basically everything useful about sandboxing when you do that :|
It then says I'm trying to run it as Root too, if using sandboxing. Are you
invoking any commands to have it run as a regular user? Is your Docker
daemon itself configured differently to stock perhaps?Thanks.
—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
I tried this here for Chrome and it's working (with --no-sandbox, which is evil I know). The extra steps he does is to put a user in the container and then run as that user, it's got to be something to do with that. Just FYI really in case it's ever useful. http://fabiorehm.com/blog/2014/09/11/running-gui-apps-with-docker/
Can confirm that UID and GID is necessary. Also /dev/dri mapping is also essential otherwise you will have Chrome crashing every couple of tabs.
Would you mind sharing how you passed the UID and GID through? What's the exact command you're using to startup this container? Thanks.
Yeah, I don't mind at all. I actually forked someone's else project and introduce a bunch of fix and features I thought he was missing. You can find the project here:
https://github.com/hurricanehrndz/docker-browser-box
Let me know if you encounter any issues, actually just open one up. Also the readme should be pretty self explanatory. I have not updated the README to point to my docker repo yet, but in case your wondering my username is hurricane on the hub. Feel free to fork though and make pull request.
@hurricanehrndz Sorry to abuse Jess' thread, but I can't figure out how to raise an issue with docker-browser-box on your page. Have you turned anything off? Getting some errors now. Could you let me know how to message/raise an issue so I can help resolve the problem. Thanks.
@xcellardoor issues are now open. My apologies go @jfrazelle and many thanks because without your hard work none of this would be possible!
I'm getting this error on arch linux too. I tried to strace it and here's my understanding of the problem:
It thinks that it's able to create network namespaces and PID namespaces, but it fails when it tries to fork with clone()
it fails with the following error:
[pid 20] clone(child_stack=0, flags=CLONE_NEWPID|CLONE_NEWNET|SIGCHLD) = -1 EPERM (Operation not permitted)
I'll try to reproduce the error by writing a small C program that does just the clone part.
its gotta be distro specific, i use this image everyday :/ even right now
The question is what does Arch do that Debian (which Jess uses) doesn't. I know Debian tends to value stability and older versions of packages... perhaps there is a complication with a package which cutting-edge Arch uses which Debian etc may soon hit too. I look forward to seeing what @vikstrous finds but if I can test anything for anyone, please ask and I'll be happy to.
After reading the man page for clone
I found out that chrome needs CAP_SYS_ADMIN
to use the CLONE_NEWNET
flag. I added --cap-add SYS_ADMIN
and I got past the Operation not permitted
error. I still get this though:
[1:1:1006/032221:ERROR:nacl_fork_delegate_linux.cc(314)] Bad NaCl helper startup ack (0 bytes)
All of this seems very wrong I do not add that cap what user are you
running as in the container, do you have apparmor or selinux installed?
On Monday, October 5, 2015, Viktor Stanchev notifications@github.com
wrote:
After reading the man page for clone I found out that chrome needs
CAP_SYS_ADMIN to use the CLONE_NEWNET flag. I added --cap-add SYS_ADMIN
and I got past the Operation not permitted error. I still get this though:[1:1:1006/032221:ERROR:nacl_fork_delegate_linux.cc(314)] Bad NaCl helper startup ack (0 bytes)
—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
Mine creates the chrome sandbox just fine I just straced it
On Monday, October 5, 2015, Jessica Frazelle me@jessfraz.com wrote:
All of this seems very wrong I do not add that cap what user are you
running as in the container, do you have apparmor or selinux installed?On Monday, October 5, 2015, Viktor Stanchev <notifications@github.com
javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:After reading the man page for clone I found out that chrome needs
CAP_SYS_ADMIN to use the CLONE_NEWNET flag. I added --cap-add SYS_ADMIN
and I got past the Operation not permitted error. I still get this
though:[1:1:1006/032221:ERROR:nacl_fork_delegate_linux.cc(314)] Bad NaCl helper startup ack (0 bytes)
—
Reply to this email directly or view it on GitHub
#65 (comment)
.Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu
http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
I don't have either one. Arch doesn't have them by default.
Here's the c file to test with:
https://gist.github.com/vikstrous/151b4c74fc0ab4c10d85
Does this work on your system without --cap-add SYS_ADMIN
?
Thank you very much @vikstrous for writing the test. Here are the results:
cellardoor at glow in [~/Temp]
12:01:56 › gcc test.c
cellardoor at glow in [~/Temp]
12:02:02 › ./a.out
Cloning...
Cloning failed with errno 1: Operation not permitted
cellardoor at glow in [~/Temp]
12:02:05 › sudo ./a.out
Cloning...
Parent running.
Child running.
Test successful.
There is a problem with my user being unable to clone the process, but running with sudo as a whim to bypass whatever protection is in place worked.
Running 'unshare' will also clone just fyi
On Tuesday, October 6, 2015, xcellardoor notifications@github.com wrote:
Thank you very much @vikstrous https://github.com/vikstrous for writing
the test. Here are the results:cellardoor at glow in [~/Temp]
12:01:56 › gcc test.c
cellardoor at glow in [~/Temp]
12:02:02 › ./a.out
Cloning...
Cloning failed with errno 1: Operation not permittedcellardoor at glow in [~/Temp]
12:02:05 › sudo ./a.out
Cloning...
Parent running.
Child running.
Test successful.There is a problem with my user being unable to clone the process, but
running with sudo as a whim to bypass whatever protection is in place
worked.—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
this is the exact command i run, and i am posting this comment from the chrome in a container :p
docker run -d \
--memory 3gb \
--net host \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e DISPLAY=unix$DISPLAY \
-v $HOME/Downloads:/root/Downloads \
-v $HOME/Pictures:/root/Pictures \
-v $HOME/Torrents:/root/Torrents \
-v $HOME/.chrome:/data \
-v /dev/shm:/dev/shm \
-v /etc/hosts:/etc/hosts \
--device /dev/snd \
--device /dev/dri \
--device /dev/video0 \
--group-add audio \
--group-add video \
--name chrome \
jess/chrome --user-data-dir=/data --force-device-scale-factor=1 \
--proxy-server="$proxy" --host-resolver-rules="$map" "$args"
@vikstrous are u in the office tommorrow we can debug at lunch or something because it seems weird to me
also what kernel are you on?
and what docker version?
Yes, --net host is off. This was on my personal laptop and I don't have it here, but I think the kernel is around 4.2.2 and the docker version is 1.8.2.
On October 6, 2015 10:40:57 AM PDT, Jess Frazelle notifications@github.com wrote:
and what docker version?
Reply to this email directly or view it on GitHub:
#65 (comment)
Sent from my Android device with K-9 Mail. Please excuse my brevity.
I tried Ubuntu in a VM. Here's what I found:
- I confirmed that Chrome on Ubuntu starts without any special capabilities and without turning off the sandbox.
- I confirmed that my test binary fails even though Chrome starts with the same config.
- I don't think
--net host
changes anything WRT the sandbox - I have no idea how X11 does authentication on Ubuntu because clearly it doesn't use
.Xauthority
- I found that I couldn't
strace
inside the container without--privileged
which is kind of weird. This is a bit off topic and it's a kernel issue, but it should be allowed IMO. - Arch Linux's kernel is compiled without user namespaces, but Ubuntu has them on. The issue seems to be that
CLONE_NEWPID
andCLONE_NEWNET
are allowed only withCLONE_NEWUSER
.
Chrome under ubuntu does a test for them from what I can see:
clone(child_stack=0, flags=CLONE_NEWUSER|SIGCHLD) = 21
On arch this test fails:
clone(child_stack=0, flags=CLONE_NEWUSER|SIGCHLD) = -1 EINVAL (Invalid argument)
Then on Ubuntu chrome does:
clone(child_stack=0x7fff9c88d7c0, flags=CLONE_NEWUSER|CLONE_NEWPID|CLONE_NEWNET|SIGCHLD) = 22
And on arch it tries:
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f9d90f60d10) = 20
Now I know why this behaviour happens. The next question is if there is anything we can do about it.
I see two options that are currently possible for Arch users:
- Using a custom kernel on Arch
- Turning off the sandboxing on Arch
I see a few long term options:
- Figure out how to make Chrome/Chromium fail more elegantly in this scenario and contribute a patch.
- Get user namespaces enabled in the Arch kernel - https://bugs.archlinux.org/task/36969 - might take a year until things are more settled down and they are comfortable with the security of the feature.
- I don't know if this is possible, and it's a kernel change, but maybe allow network/pid namespacing without CAP_SYS_ADMIN.
Should chrome be allowed to even do clone with CLONE_NEWPID | CLONE_NEWNET | CLONE_NEWUSER
? I'm not that familiar with how user namespaces are implemented, so I need to read more before I can figure out if this is normal.
Edit, I think this is indeed normal. This article cleared things up for me: http://lwn.net/Articles/528078/
Yeah all that makes sense and you definitely need CLONE_NEWUSER lol I
didn't realize that was missing from the kernel
So mine is using chromiums new namespace sandbox and yours is using the old setuid sandbox which is not as great :p
On Wednesday, October 7, 2015, Viktor Stanchev notifications@github.com
wrote:
Should chrome be allowed to even do clone with CLONE_NEWPID |
CLONE_NEWNET | CLONE_NEWUSER? I'm not that familiar with how user
namespaces are implemented, so I need to read more before I can figure out
if this is normal.—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
Guessing you guys are on a roll in pinpointing the problem :)
@jfrazelle Tested your command this morning, turned off the -d flag so I could see what is put out on the CLI. Same problem as before so it definitely looks to be a sandboxing/namespaces issue on Arch. I could take a look into popping a custom kernel on here, it takes a few minutes so I don't mind.
Can either of you recommend any community built kernels that you know wouldn't implement these protections? I'm taking a look at Zen Kernel at the moment, for which Arch has a repo package. So far my Google-fu hasn't got a definitive answer. To make sure I understand, I'm after user-namespaces ON, but a greater priority is sandboxing OFF?
If it helps, my versions are:
Kernel - 4.2.2-1-ARCH
Docker - 1.8.2
cellardoor at glow in [~]
10:05:08 › docker run \
--memory 3gb \
--net host \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e DISPLAY=unix$DISPLAY \
-v $HOME/Downloads:/root/Downloads \
-v $HOME/Pictures:/root/Pictures \
-v $HOME/Torrents:/root/Torrents \
-v $HOME/.chrome:/data \
-v /dev/shm:/dev/shm \
-v /etc/hosts:/etc/hosts \
--device /dev/snd \
--device /dev/dri \
--device /dev/video0 \
--group-add audio \
--group-add video \
--name chrome \
jess/chrome --user-data-dir=/data --force-device-scale-factor=1 \
--proxy-server="$proxy" --host-resolver-rules="$map" "$args"
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
Using a custom kernel is your best bet here, also I use custom kernels :p
Turning off sandboxing probably isn't the best idea
I tried submitting a patch to chromium once and nothing happened granted
it wasn't for this but still... Idk
You could submit a patch to arch kernel of make your own arch package with
a different kernel config
No, they would never allow cloning new net namespaces or new pid namespaces
without CAP_SYSADMIN
On Tuesday, October 6, 2015, Viktor Stanchev notifications@github.com
wrote:
I tried Ubuntu in a VM. Here's what I found:
- I confirmed that Chrome on Ubuntu starts without any special
capabilities and without turning off the sandbox.- I confirmed that my test binary fails even though Chrome starts with
the same config.- I don't think --net host changes anything WRT the sandbox
- I have no idea how X11 does authentication on Ubuntu because clearly
it doesn't use .Xauthority- I found that I couldn't strace inside the container without
--privileged which is kind of weird. This is a bit off topic and it's
a kernel issue, but it should be allowed IMO.- Arch Linux's kernel is compiled without user namespaces, but Ubuntu
has them on. The issue seems to be that CLONE_NEWPID and CLONE_NEWNET
are allowed only with CLONE_NEWUSER.Chrome under ubuntu does a test for them from what I can see:
clone(child_stack=0, flags=CLONE_NEWUSER|SIGCHLD) = 21
On arch this test fails:
clone(child_stack=0, flags=CLONE_NEWUSER|SIGCHLD) = -1 EINVAL (Invalid argument)
Then on Ubuntu chrome does:
clone(child_stack=0x7fff9c88d7c0, flags=CLONE_NEWUSER|CLONE_NEWPID|CLONE_NEWNET|SIGCHLD) = 22
And on arch it tries:
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f9d90f60d10) = 20
Now I know why this behaviour happens. The next question is if there is
anything we can do about it.I see two options that are currently possible for Arch users:
- Using a custom kernel on Arch
- Turning off the sandboxing on Arch
I see a few long term options:
- Figure out how to make Chrome/Chromium fail more elegantly in this
scenario and contribute a patch.- Get user namespaces enabled in the Arch kernel -
https://bugs.archlinux.org/task/36969 - might take a year until things
are more settled down and they are comfortable with the security of the
feature.- I don't know if this is possible, and it's a kernel change, but
maybe allow network/pid namespacing without CAP_SYS_ADMIN.—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
I'll have a pop at Zen Kernel, because why not?!
If not yeah I'll have to build my own Kernel. That will be a nice throwback to Gentoo :|
FYI you do NOT want to see what your pulseaudio build does when I try to run it for Skype as per your blog... I might open up an issue over there sometime. It's probably caused again by running on Arch.
Making your own package on arch for a kernel w a different config could be
really cool, I hear they have nice packaging tools obviously debians are
better ;) but yeah
On Wednesday, October 7, 2015, xcellardoor notifications@github.com wrote:
I'll have a pop at Zen Kernel, because why not?!
If not yeah I'll have to build my own Kernel. That will be a nice
throwback to Gentoo :|FYI you do NOT want to see what your pulseaudio build does when I try to
run it for Skype as per your blog... I might open up an issue over there
sometime. It's probably caused again by running on Arch.—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
Already done that, very easy :)
Yes the PKGBUILD system is very easy to use, and I remember using the Debian tools when I did a few packages for Ubuntu. Definitely prefer Arch's system :P You should try it sometime.
Failed with ZenKernel by the way... Next step is probably compiling my own.
any luck with this lately?
Nah, I'm planning to fork the AUFS kernel in the AUR and add user namespaces and call it the "docker" kernel. The only thing stopping me is that I need an easy way to make nvidia packages for it too and that's more effort. I'm hoping someone else will do it first.
Can't we just pass puid and pgid like it is in some other containers? Or is this much more than that?
On Fri, Nov 13, 2015 at 7:42 PM -0800, "Viktor Stanchev" notifications@github.com wrote:
Nah, I'm planning to fork the AUFS kernel in the AUR and add user namespaces and call it the "docker" kernel. The only thing stopping me is that I need an easy way to make nvidia packages for it too and that's more effort. I'm hoping someone else will do it first.
—
Reply to this email directly or view it on GitHub.
Chrome wants to do its own namespacing within the container, but it doesn't have privileges to do that for obvious reasons. User namespaces allow it create its own namespaces even though it's running within Docker. You have to completely disable Chrome's sandbox if you want it to run within Docker without user namespaces support. Since Chrome needs x11 access, it's not a good idea to disable its sandbox. With the sandbox disabled it's easier to compromise the Chrome within the container and if that happens it can use the x11 access to do keylogging, screenshots, keyboard emulation and other bad things. Running chrome as non-root within the container doesn't make a difference in this case.
What is it about arch that makes this so much more difficult/complex vs the
likes of jess' favorite Debian?
On Sat, Nov 14, 2015, 12:07 AM Viktor Stanchev notifications@github.com
wrote:
Chrome wants to do its own namespacing within the container, but it
doesn't have privileges to do that for obvious reasons. User namespaces
allow it create its own namespaces even though it's running within Docker.
You have to completely disable Chrome's sandbox if you want it to run
within Docker without user namespaces support. Since Chrome needs x11
access, it's not a good idea to disable its sandbox. With the sandbox
disabled it's easier to compromise the Chrome within the container and if
that happens it can use the x11 access to do keylogging, screenshots,
keyboard emulation and other bad things. Running chrome as non-root within
the container doesn't make a difference in this case.—
Reply to this email directly or view it on GitHub
#65 (comment)
.
The kernel config, honestly compiling your own kernel is the easiest solution here you can even do it in a container
Ha, if I only knew how... Time to Google
On Sat, Nov 14, 2015, 11:10 AM Jess Frazelle notifications@github.com
wrote:
The kernel config, honestly compiling your own kernel is the easiest
solution here you can even do it in a container—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Hi again,
I'm fine with building my own Kernel - that's easy! ;) but could someone tell me the key functionalities I have to ensure are built into the Kernel? Then I'll compile and report back :)
FYI by key functionalities I meant the ones Docker needs to do it's thing!
Check the docker source code, the check config is there under contrib, if I
remember correctly.
On Nov 15, 2015 7:49 AM, "Sam Cater" notifications@github.com wrote:
FYI by key functionalities I meant the ones Docker needs to do it's thing!
—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Thanks @hurricanehrndz - I found https://github.com/docker/docker/blob/master/contrib/check-config.sh - and from that I believe that the following are the flags I need to set, which I shall now do. The big one which stood out to me was 'USER_NS'.
USER_NS
EXT3_FS EXT3_FS_XATTR EXT3_FS_POSIX_ACL EXT3_FS_SECURITY
EXT4_FS EXT4_FS_POSIX_ACL EXT4_FS_SECURITY
AUFS_FS
BTRFS_FS
BLK_DEV_DM DM_THIN_PROVISIONING
OVERLAY_FS
It's make time!
My messy screenshot shows we have liftoff.
http://i.imgur.com/dadMx4m.png
In the end the key change to the Kernel was USER_NS. I also added some EXT3_FS_SECURITY that wasn't enabled before and a couple of other options, but USER_NS was the big one. With that support, launching Chrome through Docker works just fine.
So what is the best way to proceed... Post my config.gz to a Git repo for people to use when building their own Kernel? (That would go out of date quickly though). A blog post with the instructions? Let me know your thoughts.
Job done with a blog post - https://blog.samcater.com/docker-arch-linux-and-user-namespaces/
Jess I guess you can close the issue now. Thanks everyone for the help in working out what needs to be done.
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
I'm getting the same issue.
It appears on docker-machine AND kitematic.
OS: Wins 10, VirtualBox.
It's not exactly an issue read above
On Thursday, March 3, 2016, Charlie Kuharski notifications@github.com
wrote:
Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permittedI'm getting the same issue.
It appears on docker-machine AND kitematic.
OS: Wins 10, VirtualBox.—
Reply to this email directly or view it on GitHub
#65 (comment)
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
Hi Jess,
I would just like to append a bit more info here since I started facing this issue yday after a docker update.
A bit of usage context - till yesterday, I was using default fedora docker (1.9) on Fedora 23
Linux hostname 4.4.2-301.fc23.x86_64 #1 SMP Tue Feb 23 19:00:38 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
And chrome docker was working to my satisfaction.
Y'day, for using latest docker-compose features, I ended up updating my docker to 1.10 from docker repo.
After updating to above docker, my chrome docker is failing with..
[anshup@mouthwa ~]$ docker run -it --net host --cpuset-cpus 0 --memory 512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY -v $HOME/Downloads:/root/Downloads -v $HOME/.config/google-chrome/:/data --rm -v /dev/shm:/dev/shm jess/chrome
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
So, there was no change in kernel, but an update in docker version. Am on F23.
[anshup@mouthwa ~]$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: Fedora
Description: Fedora release 23 (Twenty Three)
Release: 23
Codename: TwentyThree
I then updated my kernel to latest, and I still ve the same issue. I am thinking this is a docker related issue rather than kernel issue. Thoughts?
[anshup@mouthwa ~]$ uname -a
Linux hostname 4.4.5-300.fc23.x86_64 #1 SMP Thu Mar 10 17:54:44 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Use my custom seccomp profile in my dot files repo
On Thursday, March 17, 2016, Anshu Prateek notifications@github.com wrote:
Hi Jess,
I would just like to append a bit more info here since I started facing
this issue yday after a docker update.
A bit of usage context - till yesterday, I was using default fedora docker
(1.9) on Fedora 23Linux hostname 4.4.2-301.fc23.x86_64 #1
#1 SMP Tue Feb 23 19:00:38
UTC 2016 x86_64 x86_64 x86_64 GNU/LinuxAnd chrome docker was working to my satisfaction.
Y'day, for using latest docker-compose features, I ended up updating my
docker to 1.10 from docker repo.After updating to above docker, my chrome docker is failing with..
[anshup@mouthwa ~]$ docker run -it --net host --cpuset-cpus 0 --memory
512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY -v
$HOME/Downloads:/root/Downloads -v $HOME/.config/google-chrome/:/data --rm
-v /dev/shm:/dev/shm jess/chrome
Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permittedSo, there was no change in kernel, but an update in docker version. Am on
F23.[anshup@mouthwa ~]$ lsb_release -a
LSB Version:
:core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: Fedora
Description: Fedora release 23 (Twenty Three)
Release: 23
Codename: TwentyThreeI then updated my kernel to latest, and I still ve the same issue. I am
thinking this is a docker related issue rather than kernel issue. Thoughts?[anshup@mouthwa ~]$ uname -a
Linux hostname 4.4.5-300.fc23.x86_64 #1
#1 SMP Thu Mar 10 17:54:44
UTC 2016 x86_64 x86_64 x86_64 GNU/Linux—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#65 (comment)
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
@jfrazelle awesome, it worked, thanks :)
@jfrazelle this is amazing. How did you managed to build that out?
Unreal. Fantastic work.
I've got the same in Ubuntu 16.04 with Docker v1.11:
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
you need user namespaces
On Wed, May 4, 2016 at 2:08 PM, Thiago Rider Augusto <
notifications@github.com> wrote:
I've got the same in Ubuntu 16.04 with Docker v1.11:
Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permitted—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#65 (comment)
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
@thiagorider If Ubuntu 16.04 doesn't have User Namespaces enabled in the Kernel then you're going to have to compile and install your own I'm afraid. I had this problem in Arch and wrote the basics of what you have to enable in the Kernel for this to work
@xcellardoor @jfrazelle When I was using Ubuntu 14.04 and Docker 1.9 it worked. I should find where to enable user namespaces in my kernel. It is different than Arch :-(
Was this option enabled in past kernels by default? Or my past Docker version(1.9) didn't required it?
It's chrome that needs it for the sandbox
On Thursday, May 5, 2016, Thiago Rider Augusto notifications@github.com
wrote:
@xcellardoor https://github.com/xcellardoor @jfrazelle
https://github.com/jfrazelle When I was using Ubuntu 14.04 and Docker
1.10 it worked. I should find where to enable user namespaces in my
kernel. It is different than Arch :-(
Was this option enabled in past kernels by default? Or my past Docker
version(1.9) didn't required it?—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#65 (comment)
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
@jfrazelle Is it related to the last Chrome release?
It's been in chrome for awhile now
On Thursday, May 5, 2016, Thiago Rider Augusto notifications@github.com
wrote:
@jfrazelle https://github.com/jfrazelle Is it related to the last
Chrome release?—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#65 (comment)
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
@thiagorider you need to run it with:
docker run ... --security-opt seccomp:/path/to/chrome.json ...
The chrome.json file you can get it from: https://raw.githubusercontent.com/jfrazelle/dotfiles/master/etc/docker/seccomp/chrome.json
It seems to grant special privileges to the container. I am also using Ubuntu 16.04, and Docker 1.11, and there is no need to recompile the kernel.
@bmustiata Thank you!
How to get sound to work with Kitematic on Mac:
for those trying to get this working I've used the following to launch chrome successfully but can't get sound working as of yet... hopefully soon with someone's help here:
docker run -d \
--memory 3gb \
--net host \
-v /etc/localtime:/etc/localtime:ro \
-e DISPLAY=192.168.99.1:0 \
-v $HOME/Downloads:/root/Downloads \
-v $HOME/Pictures:/root/Pictures \
-v $HOME/Torrents:/root/Torrents \
-v $HOME/.chrome:/data \
-v /dev/shm:/dev/shm \
-v /etc/hosts:/etc/hosts \
--security-opt seccomp:/path/to/chrome.json \
--group-add audio \
--group-add video \
--name chrome \
jess/chrome --user-data-dir=/data --force-device-scale-factor=1
Also used this:
moby/moby#8710 (comment)
Also another oddity is browsing works fine but the "Sign-in" button just "spins"
Closing as needed to compile a new kernel
Just wanted to thank you for your effort to unleash containers on the Desktop! I used this thread to help solve a problem with another Docker-Chrome project.
So far, creating a new user and use it to launch Chromium is a working workaround on Arch. Except that for some reasons I haven't pinpointed out yet, without the --privileged
flag the same error message "Failed to move to new namespace..." keeps appearing. Which devices Chromium needs to start? I'm wondering it.
It's because arch doesnot have user namespaces enabled.
On Mon, Aug 22, 2016 at 1:15 PM, Thuan Hu notifications@github.com wrote:
So far, creating a new user and use it to launch Chromium is a working
workaround on Arch. Except that for some reasons I haven't pinpointed out
yet, without the --privileged flag the same error message "Failed to move
to new namespace..." keeps appearing. Which devices Chromium needs to
start? I'm wondering it.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#65 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABYNbBjzDBJ9NyVC6bVo5rPi9YsrVCxkks5qigNPgaJpZM4GCdXL
.
Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3
Confirmed. The ticket says it all. Thanks @jfrazelle for this lightning fast answer.
Thank you Jessie for an inspiring lead.
I'm excited to migrate from Mac OS to containerized Linux but I'm stuck at this step.
I'm getting the following error message when I try to run Google Chrome in the container:
# google-chrome
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
Illegal instruction (core dumped)
My host is Debian Sid running in VirtualBox on Mac OS. I believe that NS_USER
is enabled by default in the kernel already:
$ grep USER_NS /boot/config-$(uname -r)
CONFIG_USER_NS=y
I'm disabling seccomp
in my docker run:
docker run \
--env DISPLAY \
--interactive \
--rm \
--security-opt seccomp=unconfined \
--tty \
--volume /tmp/.X11-unix:/tmp/.X11-unix \
my-debian
Here's what strace
shows (but I don't know how to interpret it):
access("/opt/google/chrome/chrome-sandbox", F_OK) = 0
stat("/opt/google/chrome/chrome-sandbox", {st_mode=S_IFREG|S_ISUID|0755, st_size=14464, ...}) = 0
access("/opt/google/chrome/chrome-sandbox", X_OK) = 0
pipe([12, 13]) = 0
close(13) = 0
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f58d1ecad90) = 43
rt_sigprocmask(SIG_SETMASK, [], ~[KILL STOP RTMIN RT_1], 8) = 0
close(12) = 0
close(11) = 0
recvmsg(10, Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
{msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="", iov_len=13}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 0
--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPN, si_addr=0x5605fae8bd9b} ---
+++ killed by SIGILL (core dumped) +++
Illegal instruction
I'm keeping my Dockerfile
simple:
FROM debian:sid
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get upgrade --yes
RUN apt-get install --yes \
wget
RUN wget \
https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg --install google-chrome-stable_current_amd64.deb; \
apt-get install --fix-broken --yes
Chrome works fine outside of the container.
Does anyone have any thoughts?
Thanks for the tip, that fixed the problem!
Here's an article I found describing the background of why it's not enabled by default: Controlling access to user namespaces and here's how to enable it (from Enable user namespaces in Debian kernel):
echo 'kernel.unprivileged_userns_clone=1' > /etc/sysctl.d/00-local-userns.conf
service procps restart
My host OS is Ubuntu 16.10. I'm getting the namespace exception when I attempt to run jess/chrome. I've set the kernel parameter both in the container and outside and have restarted procps but I'm still getting the exception.
I know this might be late but running the container with --privileged option and CMD "google-chrome", "--no-sandbox" , "--user-data-dir" solves the issue on arch linux.
My version: Linux version 4.8.13-1-ARCH (builduser@tobias) (gcc version 6.2.1 20160830 (GCC) ) #1 SMP PREEMPT Fri Dec 9 07:24:34 CET 2016
I do NOT have user namespaces enabled:
lxc-checkpoint prints :
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: missing
Network namespace: enabled
Sorry to jump into the party, but I've been hit with this issue with latest CoreOS and docker 1.12.x. This worked perfectly with CoreOS shipping docker 1.11.x:
[root@13ab34c36c82 /]# /opt/google/chrome/chrome
Failed to move to new namespace: PID namespaces supported,
Network namespace supported,
but failed: errno = Operation not permitted
Aborted (core dumped)
Now, I have to either run the container with --cap-add=SYS_ADMIN
or --privileged
. More on this link:
http://serverfault.com/questions/824809/chrome-under-docker-cap-sys-admin-vs-privileged
@jsosic run this on the CoreOS box - zgrep USER_NS /proc/config.gz
Does it come back with CONFIG_USER_NS=y or CONFIG_USER_NS=n ?
I am also having this issue on CoreOS. I tried using the --security-opt seccomp:/path/to/chrome.json and it is complaining about not opening display. I need to figure out how to update my ssh_config to enable X11Forwarding to see what happens next.
This is my error after using the --security-opt which gets past the namespacing issue. I think once I enable X11 I will be good, but there is no editor on my docker image to make this quick. Will update this post when I edit ssh_config.
[7:7:0112/183421:ERROR:browser_main_loop.cc(271)] Gtk: cannot open display:
@xcellardoor what output are you expecting from zgrep USER_NS /proc/config.gz? I assume it is CONFIG_USER_NS=y.
@jsosic how did exec_linux.go help you? Where did you execute it? On the docker container or the coreos machine hosting the docker engine?
edit: nevermind, --cap-add=SYS_ADMIN worked for me, thanks @jsosic
"Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted"
Had the same problem. Went through the entire thread but unable to find a solution.
I am using ubuntu 16.04 with kernel 4.6. User name spaces are enabled.
cat /boot/config-4.6.0-040600-generic | grep CONFIG_USER_NS
CONFIG_USER_NS=y
Was anyone able to solve this problem?
Yes that did solve the problem for me, but isn't it potentially dangerous to give the SYS_ADMIN cap to a container? That defeats the whole purpose of running it in container right? Correct me if I'm wrong because I'm not sure (My speculation is based upon the article here https://lwn.net/Articles/486306/)
and then use the custom seccomp profile https://github.com/jessfraz/dockerfiles/blob/master/chrome/stable/Dockerfile#L17
Hi i need run google chrome with the --headless --disabled-gpu
flags but dont works and the error is the same.
Error:
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
This is my Dockerfile , can help me, please ?
https://github.com/ripper2hl/jenkinswithchrome/blob/master/Dockerfile
regrards
use the custom seccomp profile https://github.com/jessfraz/dockerfiles/blob/master/chrome/stable/Dockerfile#L17
You should build a bot that replies with "use the custom seccomp profile" every time someone comments in this issue 🤖
Apologies for closed thread necro.
I'm on Xubuntu 16.04 with user namespace enabled and seccomp profile downloaded.
Some sanity checking of assumptions/requirements:
➜ ~ id $(whoami)
uid=1000(nick) gid=1000(nick) groups=1000(nick),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),113(lpadmin),128(sambashare),1001(docker)
➜ ~ uname -r
4.8.0-52-generic
➜ ~ docker --version
Docker version 17.05.0-ce, build 89658be
➜ ~ sudo cat /boot/config-$(uname -r) | grep CONFIG_USER_NS
CONFIG_USER_NS=y
➜ ~ ls -la ~/chrome.json
-rw-rw-r-- 1 nick nick 36373 May 27 11:01 /home/nick/chrome.json
➜ ~ echo $DISPLAY
:0.0
➜ ~ ls -la /tmp/.X11-unix
total 8
drwxrwxrwt 2 root root 4096 May 27 10:48 .
drwxrwxrwt 11 root root 4096 May 27 11:17 ..
srwxrwxrwx 1 root root 0 May 27 10:48 X0
Not using a custom Dockerfile, just cloned your repo modified the run command:
➜ ~ docker run -it --net vpn --cpuset-cpus 1 --memory 512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY -v $HOME/Downloads:/root/Downloads -v $HOME/.config/google-chrome-docker/:/data --security-opt seccomp=$HOME/chrome.json --device /dev/snd -v /dev/shm:/dev/shm --name chrome jess/chrome
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
No protocol specified
[1:1:0527/152604.775741:ERROR:browser_main_loop.cc(279)] Gtk: cannot open display: unix:0.0
Same with --cap-add
➜ ~ docker rm chrome
chrome
➜ ~ docker run -it --cap-add=SYS_ADMIN --net vpn --cpuset-cpus 1 --memory 512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY -v $HOME/Downloads:/root/Downloads -v $HOME/.config/google-chrome-docker/:/data --security-opt seccomp=$HOME/chrome.json --device /dev/snd -v /dev/shm:/dev/shm --name chrome jess/chrome
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
No protocol specified
[1:1:0527/152648.866742:ERROR:browser_main_loop.cc(279)] Gtk: cannot open display: unix:0.0
I'm not sure where to take it from here, advise would be welcome.
Thanks!
fixed with xhost +
I'll circle back to make that more secure later, but for now, it's working beautifully.
Thanks for all your container work, and for the fast reply.
It is better to disable sandboxing than disable selinux and running as root. It doesn't make sense to remove security of your entire host, just to run chrome with sandboxing. If your container is running as non-root and with selinux isolating it from the rest of the system, you don't have to worry about chrome IMO. Especially if sole purpose of container is testing.
@jessfraz , fixed where? If you mean fixed in chrome (to not use forbidden features) I would agree. But the other fixes I read over the internet all involve reducing container security to unknown state.
If container runtime has put restrictions over what you can do from inside the container, chances are the big and experienced security team of said container runtime knew why these restrictions are set better than you. By randomly removing restrictions, this is asking for trouble. Containers provide a false sense of security, but without the proper restrictions, you can easily break out of them.
I would say that removing container restrictions and compiling custom kernels is 99% more likely to be a bad idea than running chrome without sandboxing. Would make sense only in certain use cases. Especially for the average no-clue-about-security developer.
Additional evidence is OpenSSH and privilege separation. It is mandatory in latest version but they patched it such that it skips protections that are not possible in a restricted container environment.
Too bad Chrome is not at that yet. Somebody should file a bug I guess.
Or if you know how to make Chrome run with sandboxing and without compromising security of my OpenShift cluster, let me know.
If container runtime has put restrictions over what you can do from inside the container, chances are the big and experienced security team of said container runtime knew why these restrictions are set better than you.
You know I literally wrote a lot of those "container runtime" defaults you speak of.
I didn't want to question your competency to decide what level of protection is ok for your own use cases. I'm just saying that the advice I see above is wrong for the average (and even above average) users without clear explanation of the pros and cons in different use cases.
Or maybe I'm thinking about production use cases where security is more important, e.g. compared to just run some test stuff without messing up my machine with experimental libs and software.
I'm leaving it here though. I can't argue against no arguments. Everybody should check what guarantees the underlying container runtime is providing and make sure breaking those is acceptable for him/her.
Nice thread!
FWIW, I was able to get Chrome running on a CentOS image with a couple of minor changes to the default.json file (adding "clone" and "unshare").
@vikstrous Apologies for resurrecting an old thread, but I'm wondering about your comment re: --no-sandbox
option:
With the sandbox disabled it's easier to compromise the Chrome within the container and if that happens it can use the x11 access to do keylogging, screenshots, keyboard emulation and other bad things.
This is understood. However, my use-case is very narrow (headless SVG to PNG conversion), and I can't use @jessfraz's seccomp
profile in my environment. My only other option is adding CAP_SYS_ADMIN
which is clearly worse than --no-sandbox
. Given all of the above, I fail to see any practical security concerns with using --no-sandbox
, but feel like I'm missing something, so would really appreciate any comment and/or advice. Thanks!
Headless usage and disabling chrome sandboxing
@ebr if the x11 server that chrome is connecting to is containerized along with chrome, I can see how --no-sandbox might be a good solution. (If chrome doesn't need x11 in that mode, even better.) When I said it's a bad idea to run it with --no-sandbox I was specifically referring to the case where you are using it as a browser to handle untrusted input and giving it access to your x11 user session. In your case the input is probably untrusted, but you don't have the same level of exposure because there's no x11 user session. If you can keep the x11 server chrome is using containerized in the same container or a sidecar container, you can isolate the potential damage. You would be relying on docker's containerization instead of chrome's in this case and all the usual docker hardening approaches can be used.
TLDR: IMO it's not an issue because you'd still have docker's sandboxing. Just don't give it access to your user's x11 session.
@vik-y have you tried
--cap-add=SYS_ADMIN
when starting container?
Thanks so much I tried a lot of things but this was the only thing that worked!