Update docker in production to something more recent

Question

Update docker in production to something more recent

iby opened this issue 9 years ago · 24 comments

Not sure if this is the right place to ask, I've contacted the support on this recent, but it went silent. Please consider updating docker to something more recent. Circle right now runs 1.4.1, the most recent version is 1.6.2 which is light years ahead.

kimh commented 9 years ago

yup!

Answer 1 · 2015-06-08T14:24:51.000Z

Sorry about that. We are planning to update pre-instlaled Docker version very soon. Meanwhile, you can install 1.6.2 binary as part of the build by adding the following steps to circle.yml

machine:
  pre:
    - sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci' && sudo chmod 0755 /usr/bin/docker

[You may need to remove docker from the machine.services section]

Let me know how it goes.

Answer 2 · 2015-06-08T14:33:39.000Z

Not much luck, if I do just that I end up with this problem. If I add docker -d it all goes down with fire:

sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
sudo chmod 0755 /usr/bin/docker
sudo docker -d
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

100 14.7M  100 14.7M    0     0  28.6M      0 --:--:-- --:--:-- --:--:-- 29.0M
INFO[0000] +job init_networkdriver()                    
INFO[0000] +job serveapi(unix:///var/run/docker.sock)   
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock) 
WARN[0000] Running modprobe bridge nf_nat failed with message: , error: exit status 1 
INFO[0000] -job init_networkdriver() = OK (0)           
WARN[0000] mountpoint for memory not found              
FATA[0000] Shutting down daemon due to errors: Error loading docker apparmor profile: exit status 243 (Warning from profile docker-default (docker) ptrace rules not enforced
Warning from profile docker-default (docker) signal rules not enforced
Warning from profile docker-default (docker) mount rules not enforced
Unable to open /sys/kernel/security/apparmor/.replace - Permission denied
/sbin/apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?
Warning failed to create cache: docker

sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
sudo chmod 0755 /usr/bin/docker
sudo docker -d
 returned exit code 1

Action failed: sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
sudo chmod 0755 /usr/bin/docker
) sudo docker -d

Answer 3 · 2015-06-08T14:35:34.000Z

Though. That's strange. It seems the error is also with downloading / replacing the docker? When I ran this for the first time it didn't argue:

sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
sudo chmod 0755 /usr/bin/docker
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

100 14.7M 100 14.7M 0 0 29.9M 0 --:--:-- --:--:-- --:--:-- 30.4M

Answer 4 · 2015-06-08T14:37:05.000Z

I believe docker will already be auto-started as part of the apt-get/deb install - so you don't need to invoke it directly yourself.

Also, in CircleCI, we only support btrfs storage and lxc execution engine; so the command is sudo docker -d -e lxc -s btrfs (but it will bail out probably if the service is up already).

Answer 5 · 2015-06-08T14:50:05.000Z

Did the trick:

machine:
  pre:
    - |
      sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
      sudo chmod 0755 /usr/bin/docker
      sudo start docker

Answer 6 · 2015-06-08T14:51:44.000Z

Thanks! I'll keep it in mind next I time I suggest it! Let us know how else we can help!

Answer 7 · 2015-06-08T14:54:22.000Z

Sadly updating it didn't help. I think it's the lxc execution engine you've mentioned. I'm trying to debug the container with docker exec -ti and get:

FATA[0000] Error response from daemon: Unsupported: Exec is not supported by the lxc driver

Assume there's no workaround for that? Is there any other way?

Answer 8 · 2015-06-08T15:46:43.000Z

I'm afraid that docker exec isn't supported by lxc - and we cannot support native (a.k.a. libcontainer) yet. We are planning to support docker exec - but for the time-being you can use the workaround documented at https://circleci.com/docs/docker#docker-exec in CircleCI builds . Would that work you?

Answer 9 · 2015-06-08T18:56:53.000Z

That is perfect, thanks so much for pointing this out! I can attach in the same way as with docker exec -ti … bash and do what I need. There's a typo in the docs, missing use:

To work around this, you can use the following command, customized for your container name and the command you want to run, using LXC directly:

Answer 10 · 2015-07-05T07:28:55.000Z

I need a more recent version too (to specify an alternate Dockerfile). This ticket helped me with setting it up. The image building is different from our development situation vs our online environments, mostly mounting local paths at runtime vs embedding artifacts in the images.

Docker 1.6 offer the features we currently need.

Our circle.yml starts like this:

machine:
  pre:
    - |
      test $(docker --version | awk '{ print $3 }' | head -c3) != "1.6" && {
        sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci';
        sudo chmod 0755 /usr/bin/docker;
        sudo start docker;
      }
# Restore this once docker >1.6.2 is available.
#  services:
#    - docker

Builds start failing once 1.6.x becomes available on Circle, in which we restore the services section.

Answer 11 · 2015-07-05T19:20:34.000Z

@Dynom fwiw - you can change it to make it work every after >1.6.2 becomes available (planned for the next few days)

machine:
  pre:
    - |
      test $(docker --version | awk '{ print $3 }' | head -c3) != "1.6" && {
        sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci';
        sudo chmod 0755 /usr/bin/docker;
      }
    - sudo service docker start
# Restore this once docker >1.6.2 is available.
#  services:
#    - docker

Answer 12 · 2015-07-07T07:39:02.000Z

Hi @notnoopci,

Thanks for the comment! I want it to fail because then I know when I can cleanup my circle.yml and notify my team that you've updated Docker on your hosts. So it was intentional this time, thanks for the tip though! (-:

Answer 13 · 2015-07-09T11:53:22.000Z

At voila, Circle supports Docker 1.6 now! Fixing my Circle file (-;

Answer 14 · 2015-08-23T18:53:10.000Z

Just pushed a docker 1.8.1 version - and could use someone testing it first. To enable it you can run with:

machine:
  pre:
    - |
      sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.8.1-circleci'
      sudo chmod 0755 /usr/bin/docker
      sudo service docker start

(but without machine.services.docker line)

Answer 15 · 2015-08-23T19:04:15.000Z

I can confirm that this solves an issue I was having with docker-compose and docker build.

Answer 16 · 2015-08-25T21:26:19.000Z

I can't get docker-compose to start with this version in conjunction with docker-compose 1.4.0

Traceback (most recent call last):
File "", line 3, in
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 39, in main
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 27, in dispatch
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 59, in perform_command
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 407, in start
File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 198, in start
File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 132, in start
File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 110, in containers
File "/code/build/docker-compose/out00-PYZ.pyz/docker.client", line 202, in containers
File "/code/build/docker-compose/out00-PYZ.pyz/docker.clientbase", line 86, in _get
File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 477, in get
File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 465, in request
File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 573, in send
File "/code/build/docker-compose/out00-PYZ.pyz/requests.adapters", line 433, in send
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=120)

Answer 17 · 2015-08-27T18:03:50.000Z

I can confirm this also works for me using docker-compose 1.3.3

Answer 18 · 2015-08-27T18:17:20.000Z

Spoke too soon:

docker-compose up && [ -f ./docker-test/test.txt ]
Pulling test1 (nginx:latest)...
latest: Pulling from library/nginx













Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 39, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 27, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 59, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 495, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 265, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 369, in execute_convergence_plan
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 440, in start_container
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.container", line 136, in start
  File "/code/build/docker-compose/out00-PYZ.pyz/docker.utils.decorators", line 20, in wrapped
  File "/code/build/docker-compose/out00-PYZ.pyz/docker.client", line 797, in start
  File "/code/build/docker-compose/out00-PYZ.pyz/docker.clientbase", line 128, in _post_json
  File "/code/build/docker-compose/out00-PYZ.pyz/docker.clientbase", line 83, in _post
  File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 508, in post
  File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 465, in request
  File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 573, in send
  File "/code/build/docker-compose/out00-PYZ.pyz/requests.adapters", line 433, in send

requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60) docker-compose up && [ -f ./docker-test/test.txt ] returned exit code 255

Answer 19 · 2015-09-01T21:29:44.000Z

Having the same issue.

requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60) (cd test/e2e && docker-compose up -d) returned exit code 1

Answer 20 · 2015-09-10T15:42:34.000Z

Hi @notnoopci

When using this version (Docker version 1.8.1-circleci, build 076fb94)
The following errors occur when building my image:

...
---> caa0020ada3f
Step 1 : MAINTAINER Florian Schrag <florian@schrag.org>
 ---> Running in 0ea869e2fe39
 ---> 831642999a7e
Error removing intermediate container 0ea869e2fe39: Cannot destroy container 0ea869e2fe391fa9cb79db137e918d5b52ba9b33f512cec1f084c9553acde6ea: Driver btrfs failed to remove root filesystem 0ea869e2fe391fa9cb79db137e918d5b52ba9b33f512cec1f084c9553acde6ea: Failed to destroy btrfs snapshot: operation not permitted
Step 2 : VOLUME /imports
 ---> Running in 650c7eb0128a
 ---> b6faca93db81
Error removing intermediate container 0ea869e2fe39: no such id: 0ea869e2fe391fa9cb79db137e918d5b52ba9b33f512cec1f084c9553acde6ea
...

Answer 21 · 2015-09-10T21:43:50.000Z

I'm having the same issue with docker-1.8.1-circleci and docker-compose 1.4.0. I can get the default circle-ci docker bin (v 1.6.1) and docker-compose 1.3.3 to work, but not the latest combination.

@gitu I believe that is 'ok' behavior here. There is a note somewhere in the circle-ci docker documentation about lxc not being allowed to delete intermediate containers... but that shouldn't prevent a successful build

Answer 22 · 2015-11-10T11:41:20.000Z

Is v1.9.0 ready?

Answer 23 · 2015-11-11T01:38:57.000Z

@kimh 👍