Pre start hook example does not work in ubuntu 16.04

Question

Pre start hook example does not work in ubuntu 16.04

sinneduy opened this issue 5 years ago · 5 comments

For some reason, this pre start hook example does not work in ubuntu 16.04, but works in ubuntu 18.04

In order to test out shutdown behavior, i used the following script as the "process" to make sure that it follows the expected behavior in passing the kill signal to the child process, and letting the child process decide with the container dies:

#!/bin/bash
trap "catch" SIGTERM
catch() {
  echo "An error has occurred but we are going to eat it!!"
  sleep 60
}
echo "Before bad command"
while true;
do
    sleep 5
done
echo "After bad command"

I build and run the following Dockerfile:

FROM ubuntu:16.04
RUN apt update -y && apt install curl -y
RUN curl -L -o dumb-init.deb https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_amd64.deb && dpkg -i dumb-init.deb && rm dumb-init.deb


RUN echo '#!/bin/bash\n\
trap "catch" SIGTERM\n\
catch() {\n\
  echo "An error has occurred but we are going to eat it!!"\n\
  sleep 60\n\
}\n\
echo "Before bad command"\n\
while true;\n\
do\n\
    sleep 5\n\
done\n\
echo "After bad command"\n'\
>> test.sh

RUN chmod +x test.sh

ENTRYPOINT ["/usr/bin/dumb-init", "--"]

CMD ["bash", "-c", "ls && ./test.sh"]

But this unfortunately dies immediately:

$ docker run -it f7bd5149639a
bin   dev	     etc   lib	  media  opt   root  sbin  sys	    tmp  var
boot  entrypoint.sh  home  lib64  mnt	 proc  run   srv   test.sh  usr
Before bad command
$

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
5bac5fa3f3d7        f7bd5149639a        "/usr/bin/dumb-init …"   4 seconds ago       Up 4 seconds                            compassionate_carson
$ docker exec -it 5bac5fa3f3d7 bash
root@5bac5fa3f3d7:/# kill 1
root@5bac5fa3f3d7:/# $

However, if I remove the ls (the pre start part):

FROM ubuntu:16.04
RUN apt update -y && apt install curl -y
RUN curl -L -o dumb-init.deb https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_amd64.deb && dpkg -i dumb-init.deb && rm dumb-init.deb


RUN echo '#!/bin/bash\n\
trap "catch" SIGTERM\n\
catch() {\n\
  echo "An error has occurred but we are going to eat it!!"\n\
  sleep 60\n\
}\n\
echo "Before bad command"\n\
while true;\n\
do\n\
    sleep 5\n\
done\n\
echo "After bad command"\n'\
>> test.sh

RUN chmod +x test.sh

ENTRYPOINT ["/usr/bin/dumb-init", "--"]

CMD ["bash", "-c", "./test.sh"]

$ docker run -it 4fc5ea247111
Before bad command
Terminated
An error has occurred but we are going to eat it!!

^C$

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
905030ceef9e        4fc5ea247111        "/usr/bin/dumb-init …"   3 seconds ago       Up 2 seconds                            intelligent_gould
$ docker exec -it 905030ceef9e bash
root@905030ceef9e:/# kill 1
root@905030ceef9e:/#

However, if I try to use the pre start example in ubuntu 18.04, this also works:

FROM ubuntu:18.04
RUN apt update -y && apt install curl -y
RUN curl -L -o dumb-init.deb https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_amd64.deb && dpkg -i dumb-init.deb && rm dumb-init.deb


RUN echo '#!/bin/bash\n\
trap "catch" SIGTERM\n\
catch() {\n\
  echo "An error has occurred but we are going to eat it!!"\n\
  sleep 60\n\
}\n\
echo "Before bad command"\n\
while true;\n\
do\n\
    sleep 5\n\
done\n\
echo "After bad command"\n'\
>> test.sh

RUN chmod +x test.sh

ENTRYPOINT ["/usr/bin/dumb-init", "--"]

CMD ["bash", "-c", "ls && ./test.sh"]

$ docker run -it 297416091abc
bin   dev	     etc   lib	  media  opt   root  sbin  sys	    tmp  var
boot  entrypoint.sh  home  lib64  mnt	 proc  run   srv   test.sh  usr
Before bad command
Terminated
An error has occurred but we are going to eat it!!

^C$

$ docker exec -it 677110cf1233 bash
root@677110cf1233:/# kill 1
root@677110cf1233:/#

Answer 1 · 2019-11-08T01:05:23.000Z

with -x and -v I see:

$ docker run --rm -ti testxen
[dumb-init] Detached from controlling tty, ignoring the first SIGHUP and SIGCONT we receive.
[dumb-init] Child spawned with PID 6.
[dumb-init] Received signal 1.
[dumb-init] Ignoring tty hand-off signal 1.
[dumb-init] Received signal 18.
[dumb-init] Ignoring tty hand-off signal 18.
[dumb-init] setsid complete.
bin   dev  home  lib64	mnt  proc  run	 srv  test.sh  usr
boot  etc  lib	 media	opt  root  sbin  sys  tmp      var
+ trap catch SIGTERM
+ echo 'Before bad command'
Before bad command
+ true
+ sleep 5
+ true
+ sleep 5
+ true
+ sleep 5
[dumb-init] Received signal 15.
[dumb-init] Forwarded signal 15 to children.
Terminated
[dumb-init] Received signal 17.
[dumb-init] A child with PID 6 was terminated by signal 15.
[dumb-init] Forwarded signal 15 to children.
[dumb-init] Child exited with status 143. Goodbye.

pretty sure bash just changed how it handles signals?

Answer 2 · 2019-11-08T01:19:55.000Z

pretty sure bash just changed how it handles signals?

Sorry, are you saying that trap basically just doesn't work in ubuntu 16.04? I'm not sure I follow you...

Answer 3 · 2019-11-08T05:02:16.000Z

the process that is being signaled is the bash -c '...' process, not test.sh (you'd have to exec ./test.sh -- which is precisely what the example does)

I recall some changes to bash between xenial and bionic about how signals are handled when given a controlling tty, they probably explain the difference here

If you use exec test.sh it should work in both

and if I adjust your original dockerfile:

--- Dockerfile.orig	2019-11-07 21:01:53.976892760 -0800
+++ Dockerfile	2019-11-07 20:58:50.080990759 -0800
@@ -3,7 +3,7 @@
 RUN curl -L -o dumb-init.deb https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_amd64.deb && dpkg -i dumb-init.deb && rm dumb-init.deb
 
 
-RUN echo '#!/bin/bash\n\
+RUN echo '#!/bin/bash -x\n\
 trap "catch" SIGTERM\n\
 catch() {\n\
   echo "An error has occurred but we are going to eat it!!"\n\
@@ -19,6 +19,6 @@
 
 RUN chmod +x test.sh
 
-ENTRYPOINT ["/usr/bin/dumb-init", "--"]
+ENTRYPOINT ["/usr/bin/dumb-init", "-v", "--"]
 
-CMD ["bash", "-c", "ls && ./test.sh"]
+CMD ["bash", "-c", "ls && exec ./test.sh"]

I get this as expected:

$ docker run --rm -ti test
[dumb-init] Detached from controlling tty, ignoring the first SIGHUP and SIGCONT we receive.
[dumb-init] Child spawned with PID 6.
[dumb-init] Received signal 1.
[dumb-init] Ignoring tty hand-off signal 1.
[dumb-init] Received signal 18.
[dumb-init] Ignoring tty hand-off signal 18.
[dumb-init] setsid complete.
bin   dev  home  lib64	mnt  proc  run	 srv  test.sh  usr
boot  etc  lib	 media	opt  root  sbin  sys  tmp      var
+ trap catch SIGTERM
+ echo 'Before bad command'
Before bad command
+ true
+ sleep 5
+ true
+ sleep 5
[dumb-init] Received signal 15.
[dumb-init] Forwarded signal 15 to children.
Terminated
++ catch
++ echo 'An error has occurred but we are going to eat it!!'
An error has occurred but we are going to eat it!!
++ sleep 60
^C[dumb-init] Received signal 17.
[dumb-init] A child with PID 6 was terminated by signal 2.
[dumb-init] Forwarded signal 15 to children.
[dumb-init] Child exited with status 130. Goodbye.

🎉

Answer 4 · 2019-11-08T08:07:45.000Z

I see. Thanks for the detailed explanation, I really appreciate it.

If this is regarded as a limitation of bash in xenial, should this be documented in the pre start hook example in the README.md? Alternatively, are there any changes to dumb-init that could potentially account for this?

Answer 5 · 2019-11-08T09:02:27.000Z

in my opinion, no and no -- this was a bug in bash in a particular set of versions of bash and you're not following the example