Possible to make the docker image smaller?
spandan13 opened this issue · 26 comments
I tried to build the container using python:3-slim-buster as the base image to try and make the image smaller but running the container this way maxed out cpu and ram and crashed the system. Any idea why this might be happening? Is there any other way to make the image smaller?
I'm not familiar with slim buster. What's the difference compared to the one I'm running currently?
Also, what device are you testing on? @spandan13
The slim image is a paired down version of the full image. This image generally only installs the minimal packages needed to run python.
So far I tested this on an x86 and also on an arm machine. Both are running in the cloud. I found same issue on both. Max cpu and ram after running and hence system crashed.
I am not very familiar with the details so will have to see if anyone else can figure out something. If not then will have to continue using the full buster image.
Okay I'll have a look later as well. Just to confirm, the regular buster image is working fine?
Yes, the regular image works completely find just that it results in almost a 1GB image whereas if it was possible to make it work with the slim image then the size would come down to under 300MB
@DartLazer Just found something interesting, I tried to build using 'slim-buster' instead of 'buster' but from my old fork i.e. version 0.17 and it's working fine! So I guess the issue with it maxing out the cpu and ram is due to some changes that were added in version 0.18 and above?
Very interesting.
This might be a big potential in disk space savings (costly in raspberry pi's indeed) and compiling/downloading time etc.
The original python-buster image setup consumed the following:
The new setup with python-slim-buster:
A difference af about 0.5 GB. Makes me think this might be worthwhile implementing.
Are you aware of any downsides using the slim image? @spandan13
What about you @babanomania any suggestions on this topic?
Edit
As you can see in my setup though the unique size of the slim-buster one is higher.
In my case the slim buster adds more space because I already use the python-3 image for other containers (which are running) and I think it shares the space that way (hence the high share spaced value). Might be worth considering too.
The way I'm looking at it is,
I build the image, then push it to docker hub, and then delete the whoishome and python images locally. And then finally I redeploy the container by directly pulling the image from docker hub. In this way the python image is never actually downloaded to the user's machine which also results in a much quicker download/deployment.
The image sizes you saw in my attached image above were by deploying the images directly from docker hub.
Also, for those building locally, there's no need to actually keep the python image once the container is up and running right?
Hmm I'd actually have to research that it sounds you have a more thorough understand of how docker handles files locally than I do @spandan13.
I'm gonna think it through a bit more, see what @babanomania had to see and see what we do!
i'm testing on a Pi Zero 2W (ARMv7) I tried building the docker image using python:3-slim-buster in Dockerfile, the build crashed saying 'Error: pg_config executable not found.', i'm trying to build it using a specific version of the buster image
the thought of having a slimmer docker image does sound exiting to me
@babanomania Hehe that's not a lot of hardware indeed.
I'm used the following dockerfile for testing (I had to install procps as well since it's not included in the slim version):
FROM python:3-slim-buster
ENV PYTHONUNBUFFERED=1
RUN mkdir /mysite
WORKDIR /mysite
COPY mysite /mysite/
COPY requirements.txt /mysite/
RUN pip install -r requirements.txt
RUN apt update && apt install -y net-tools arp-scan procps
okay, i'm gonna check with python:3-alpine also, just for comparison
i have some good news with alpine image, below are my changes
Below is my Dockerfile
FROM python:3-alpine
ENV PYTHONUNBUFFERED=1
RUN mkdir /mysite
WORKDIR /mysite
COPY mysite /mysite/
COPY requirements.txt /mysite/
RUN apk add --no-cache --virtual .build-deps \
ca-certificates gcc linux-headers musl-dev \
libffi-dev jpeg-dev zlib-dev net-tools arp-scan \
&& pip install -r requirements.txt \
&& find /usr/local \
\( -type d -a -name test -o -name tests \) \
-o \( -type f -a -name '*.pyc' -o -name '*.pyo' \) \
-exec rm -rf '{}' + \
&& runDeps="$( \
scanelf --needed --nobanner --recursive /usr/local \
| awk '{ gsub(/,/, "\nso:", $2); print "so:" $2 }' \
| sort -u \
| xargs -r apk info --installed \
| sort -u \
)" \
&& apk add --virtual .rundeps $runDeps \
&& apk del .build-deps
Below is my docker-compose.yml
version: "3.8"
services:
web:
network_mode: "host"
build: .
command: /bin/sh -c "python mysite/manage.py migrate && python mysite/manage.py runserver 0.0.0.0:${WHOIH_PORT:-8000}"
environment:
- DJANGO_TZ=${WHOIH_TZ}
restart: always
volumes:
- .:/mysiteBelow is my requirements.txt, i removed the dependency for postgres driver since it was causing build issues
Django>=3.0,<4.0
django-background-tasks>=1.2.5
django-crispy-forms==1.11.2
requests~=2.26.0
discord==1.7.3
discord.py==1.7.3
The image size is now 94MB on my PiZero2W (Armv7), the image size using the buster image was around 800MB.
Next is slim-buster
below are my observations slim-buster image, below are my changes
Below is my Dockerfile
FROM python:3-slim-buster
ENV PYTHONUNBUFFERED=1
RUN mkdir /mysite
WORKDIR /mysite
COPY mysite /mysite/
COPY requirements.txt /mysite/
RUN apt update && apt install -y procps net-tools arp-scan gcc
RUN pip install -r requirements.txt
Below is my requirements.txt, i removed the dependency for postgres driver since it was causing build issues
Django>=3.0,<4.0
django-background-tasks>=1.2.5
django-crispy-forms==1.11.2
requests~=2.26.0
discord==1.7.3
discord.py==1.7.3
The image size is now 280MB on my PiZero2W (Armv7), the image size using the buster image was around 800MB.
@DartLazer in terms of image size on ARMv7 the alpine image is the clear winner
should we try a multi-arch build to check the image size on different cpu architectures ?
We should keep the slim-buster and the alpine images running for some time and observe for some high CPU or memory usage.
We should keep the slim-buster and the alpine images running for some time and observe for some high CPU or memory usage.
Thanks do that please and keep me up to date! This is looking promising =D
One small update, the alpine image seems to have an older version of arp-scan, the command line is a bit different.
Might take a little time for the code changes
Hmm I remember some other users having issues as well in #1 with the regular python build on raspberry pi's. This is why I ended up using buster.
Are you experiencing the same now?
slim-buster runs fine till now, no crashes on PiZero2W,
For alpine, sure the size will be much slimmer, but some work is needed since the arp-scan package is older and the flags are a bit different
@DartLazer the arp-scan package provided by alpine seems to be quite limited in feature, it doesn't even support scan by network range, you can do the below command to get the scan work
arp -a -i wlan0
my suggestion is to stay with slim-buster
Slim-buster has been working fine on my RPi 4 and also tested on x86 machine
Can you guys share the docker file you've been using it with please? 😁 @babanomania @spandan13
I believe mine is same as the one you had mentioned earlier
FROM python:3-slim-buster ENV PYTHONUNBUFFERED=1 RUN mkdir /mysite WORKDIR /mysite COPY mysite /mysite/ COPY requirements.txt /mysite/ RUN pip install -r requirements.txt RUN apt update && apt install -y net-tools arp-scan procps
In that case I'll be confident to swap to this later on today
below are my observations slim-buster image, below are my changes
Below is my Dockerfile
FROM python:3-slim-buster ENV PYTHONUNBUFFERED=1 RUN mkdir /mysite WORKDIR /mysite COPY mysite /mysite/ COPY requirements.txt /mysite/ RUN apt update && apt install -y procps net-tools arp-scan gcc RUN pip install -r requirements.txtBelow is my requirements.txt, i removed the dependency for postgres driver since it was causing build issues
Django>=3.0,<4.0 django-background-tasks>=1.2.5 django-crispy-forms==1.11.2 requests~=2.26.0 discord==1.7.3 discord.py==1.7.3The image size is now 280MB on my PiZero2W (Armv7), the image size using the buster image was around 800MB.
@DartLazer this is what I've been using, only difference is I've removed the postgres driver from requirement.txt to cut down on build time and other issues