NVIDIA/enroot

Shadow hook segmentation faults when starting recent images

rcoacci opened this issue · 15 comments

When using recent images from Docker hub for Debian, Ubuntu and Archlinux, I'm getting an error in /etc/enroot/hooks.d/10-shadow.sh, upon start.

Steps to reproduce:

enroot import  docker://ubuntu:rolling
enroot create  ubuntu+rolling.sqsh 
enroot start -r -w ubuntu+rolling

Result of start:

/etc/enroot/hooks.d/10-shadow.sh: line 70: 2840366 Broken pipe             yes 2> /dev/null
     2840367 Segmentation fault      (core dumped) | pwck -R "${ENROOT_ROOTFS}" "${pwddb#${ENROOT_ROOTFS}}" /etc/shadow > /dev/null 2>&1

Even debian:stable fails.

Using enroot version 3.4.1

3XX0 commented

I can't reproduce, is it only doing that with those images?
You can try to debug the script see what makes pwck segfaults in your environment.
You can also try to see if it reproduces manually with something like:

enroot-unshare --user --mount --remap-root pwck -R ~/.local/share/enroot/ubuntu+rolling /etc/passwd /etc/shadow

I can't reproduce, is it only doing that with those images?

Those are the images I tried. Strangely enough ubuntu:latest works.
I coudn't run your test because it seems I don't have enroot-unshare , or at least it's not in my PATH.

I don't know how relevant it is but I'm running on RHEL 8.

3XX0 commented

Sorry it's been renamed enroot-nsenter in the new releases.

It seems the problem is yes :

$ enroot-nsenter --user --mount --remap-root pwck -R /scr01/enroot-data/user-6479/ubuntu+rolling /etc/passwd /etc/shadow
duplicate password entry
delete line 'root:x:0:0:root:/root:/bin/bash'?

$ enroot-nsenter --user --mount --remap-root yes | pwck -R /scr01/enroot-data/user-6479/ubuntu+rolling /etc/passwd /etc/shadow
Segmentation fault (core dumped)

However "yes" works by itself, and it works as yes | tee test.txt

Is there anything else?

Also now that ubuntu:latest was updated point to nobel/rolling, it's not working in latest either.
I can reproduce using mantic or latest/nobel/rolling (doesn't work) but works correctly with ubuntu:jammy

3XX0 commented

Um, can you reproduce with the following?

enroot-nsenter --user --mount --remap-root bash -c '(trap "" PIPE; yes 2> /dev/null || :) | pwck -R /scr01/enroot-data/user-6479/ubuntu+rolling /etc/passwd /etc/shadow'

Yes, same error. Investigating more, it seems that problem might be in RHEL 8.9 version of pwckafter all. But its really weird that it works for some images but not others: docker://redhat/ubi8:latest works, but docker://redhat/ubi9:latest doesn't work.

I'm assuming it's using the host's pwckright?

3XX0 commented

Could be this: https://bugzilla.redhat.com/show_bug.cgi?id=2021339
Yes, it probably depends on whether or not pwck needs to do some changes in the shadow files which is why you don't always see it

One more information: it seems that copying over /etc/passwd and /etc/shadow from an working image (e.g. ubuntu:jammy) to the non-working image (e.g. ubuntu:rolling) after enroot create stops pwck from segfaulting. I diff'ed the files but couldn't find anything obviously wrong with them.

SELinux/AppArmor?

SELinux/AppArmor?

I fail to understand. Can you elaborate?
Do you mean that the images have SELinux/AppArmor? Or the RHEL 8 host?

Just guessing that these might have a play in that.
I know that RHEL is adamant about a lot of /etc/... files having the right security context, maybe something like that happens with recent ubuntu?
It just has the smell of that category of problems

Another workaround: it seems that deleting /etc/shadow avoids the issue. Perhaps that would be better than trying to fix it with pwck when the container starts?

3XX0 commented

Enroot hooks are designed to be configurable, so you can edit them as you see fit.

This is an issue with your distro tooling, not really enroot. Same crash would happen on your host, we aren't doing anything special here.
So feel free to edit your configuration to workaround it if you can't backport a shadow-utils fix.