etix/mirrorbits

Dockerfile improvement suggestion (multistage build) + docker-compose

ashledombos opened this issue · 6 comments

ashledombos commented

Hello,

I'd like to suggest improvements for the Dockerfile.

If we use it as is, the size of the resulting docker image is huge (mine was 1.307GB)

So I first made a multistage build

FROM golang:latest as build

LABEL maintainer="etix@l0cal.com"

ADD . /go/mirrorbits

RUN apt-get update -y && \
    DEBIAN_FRONTEND=noninteractive apt-get install -y pkg-config zlib1g-dev protobuf-compiler libprotoc-dev rsync && \
    apt-get clean

RUN go get -u github.com/maxmind/geoipupdate2/cmd/geoipupdate && \
    go install -ldflags "-X main.defaultConfigFile=/etc/GeoIP.conf -X main.defaultDatabaseDirectory=/usr/share/GeoIP" github.com/maxmind/geoipupdate2/cmd/geoipupdate

RUN  cd /go/mirrorbits && \
    make install PREFIX=/usr

FROM debian:latest

# Set your own GeoIP settings
ENV GEOIP_ACCOUNT_ID 000000
ENV GEOIP_LICENSE_KEY 0000000000000000

RUN mkdir /usr/share/mirrorbits /var/log/mirrorbits /srv/repo

COPY --from=build /usr/bin/mirrorbits /usr/bin/mirrorbits
COPY --from=build /usr/share/mirrorbits/base.html /usr/share/mirrorbits/base.html
COPY --from=build /usr/share/mirrorbits/mirrorlist.html /usr/share/mirrorbits/mirrorlist.html
COPY --from=build /usr/share/mirrorbits/mirrorstats.html /usr/share/mirrorbits/mirrorstats.html
COPY --from=build /go/bin/geoipupdate /usr/bin/geoipupdate   
COPY --from=build /go/mirrorbits/contrib/docker/mirrorbits.conf /etc/mirrorbits.conf

RUN apt-get update -y && \
    DEBIAN_FRONTEND=noninteractive apt-get install -y rsync ca-certificates tzdata nano && \
    apt-get clean

RUN echo "AccountID ${GEOIP_ACCOUNT_ID}\nLicenseKey ${GEOIP_LICENSE_KEY}\nEditionIDs GeoLite2-City GeoLite2-Country GeoLite2-ASN" > /etc/GeoIP.conf && \
    mkdir /usr/share/GeoIP && \
    /usr/bin/geoipupdate

ENV EDITOR=/bin/nano
ENV LANG en_US.UTF-8

ENTRYPOINT /usr/bin/mirrorbits daemon -config /etc/mirrorbits.conf

EXPOSE 8080

The image was trimmed down to 242MB. Note that I have added nano, $EDITOR and $LANG environment so the mirrorbits edit command works well (vim can be used instead, but is bigger).

But an even smaller image can be produced by using Alpine Linux base.

FROM golang:alpine as build

LABEL maintainer="etix@l0cal.com"

ADD . /go/mirrorbits

RUN apk --update add \
    git pkgconf build-base zlib-dev protobuf libprotoc

RUN go get -u github.com/maxmind/geoipupdate2/cmd/geoipupdate && \
    go install -ldflags "-X main.defaultConfigFile=/etc/GeoIP.conf -X main.defaultDatabaseDirectory=/usr/share/GeoIP" github.com/maxmind/geoipupdate2/cmd/geoipupdate

RUN cd /go/mirrorbits && \
    make install PREFIX=/usr

FROM alpine:latest

# Set your own GeoIP settings
ENV GEOIP_ACCOUNT_ID 000000
ENV GEOIP_LICENSE_KEY 0000000000000000

RUN mkdir /usr/share/mirrorbits /var/log/mirrorbits /srv/repo

COPY --from=build /usr/bin/mirrorbits /usr/bin/mirrorbits
COPY --from=build /usr/share/mirrorbits/base.html /usr/share/mirrorbits/base.html
COPY --from=build /usr/share/mirrorbits/mirrorlist.html /usr/share/mirrorbits/mirrorlist.html
COPY --from=build /usr/share/mirrorbits/mirrorstats.html /usr/share/mirrorbits/mirrorstats.html
COPY --from=build /go/bin/geoipupdate /usr/bin/geoipupdate
COPY --from=build /go/mirrorbits/contrib/docker/mirrorbits.conf /etc/mirrorbits.conf

RUN apk --update --no-cache add \
    rsync ca-certificates tzdata

RUN echo -e "AccountID ${GEOIP_ACCOUNT_ID}\nLicenseKey ${GEOIP_LICENSE_KEY}\nEditionIDs GeoLite2-City GeoLite2-Country GeoLite2-ASN" > /etc/GeoIP.conf && \
    mkdir /usr/share/GeoIP && \
    /usr/bin/geoipupdate

ENV EDITOR=/usr/bin/vi

ENTRYPOINT /usr/bin/mirrorbits daemon -config /etc/mirrorbits.conf

EXPOSE 8080

The resulting image size is 108.1MB

I'm also using a very simple docker-compose.yml file

version: '2'

services:
  redis:
    image: 'redis:alpine'
    restart: always
    volumes:
      - mirrorbits_db/:/data/
  app:
    restart: always
    build: 'mirrorbits'
    depends_on:
      - redis
    ports:1
      - 8080:8080
    volumes:
      - /var/lib/openmandriva/abf-downloads/:/srv/repo/
      - /root/mirrors/mirrorbits/mirrorbits.conf:/etc/mirrorbits.conf
volumes:
  mirrorbits_db: {}

Then with
docker exec -it mirrorbits_app_1 ash
(or bash instead of ash if using debian base)
we can use all mirrobits commands (with "mirrorbits edit" command working)

edit: added tzdata needed by ftp to work correctly.

olblak commented

If it can help, we build and maintain a mirrorbits docker image on the Jenkins infrastructure project here, and our image is available on DockerHub
We manage the GeoIP database on the helm chart here

  • 👍1
ashledombos commented

It's great @olblak , thanks for the info. Are you the official maintainer of Mirrorbits now?

olblak commented

If it can help, we build and maintain a mirrorbits docker image on the Jenkins infrastructure project here, and our image is available on DockerHub
We manage the GeoIP database on the helm chart here

Absolutely not, I made the effort several months ago to move to mirrorbits on the Jenkins project and since it's an open infrastructure project, everything is public. So I have examples to share.

I am also behind https://hub.docker.com/orgs/mirrorbits but we DockerHub policy change, it's not really useful anymore

IMHO not installing the GeoIP database in the docker image is better since it needs to be updated on a regular basis

ashledombos commented

Thanks for the informations, I agree with the fact geoip should be updated regularly, and having it in the docker image, unless some trick with a volume and a scheduler, is not a good idea.
What DockerHub policy change was a problem?

olblak commented

DockerHub put in place strong limitations on its free tier. So either you pay for it or you apply to the OSS program as explained here

  • 👍1