confluentinc/confluent-kafka-go

Dockerfile examples

blaketastic2 opened this issue · 10 comments

blaketastic2 commented

Can we put together some examples for Docker? I’d love to see a Alpine and Debian example.

otavioprado commented

golang:latest apparently does not come with librdkafka installed

blaketastic2 commented

golang 1.13.10, alpine 3.10, librdkafka 1.3.0

# first stage
FROM golang:1.13.10-alpine3.10 AS go_kafka_base
RUN apk update && apk add --no-cache  \
        bash              \
        build-base        \
        coreutils         \
        gcc               \
        git               \
        make              \
        musl-dev          \
        openssl-dev       \
        openssl           \
        libsasl           \
        libgss-dev        \
        rpm               \
        lz4-dev           \
        zlib-dev          \
        ca-certificates   \
        wget          &&  \
    cd $(mktemp -d) && \
    wget -nv -O cyrus-sasl-2.1.27.tar.gz https://github.com/cyrusimap/cyrus-sasl/releases/download/cyrus-sasl-2.1.27/cyrus-sasl-2.1.27.tar.gz && \
    tar -xz --strip-components=1 -f cyrus-sasl-2.1.27.tar.gz && \
    rm -f cyrus-sasl-2.1.27.tar.gz && \
    ./configure --prefix=/usr --disable-sample --disable-obsolete_cram_attr --disable-obsolete_digest_attr --enable-static --disable-shared \
        --disable-checkapop --disable-cram --disable-digest --enable-scram --disable-otp --disable-gssapi --with-dblib=none --with-pic && \
    make && \
    make install

RUN cd $(mktemp -d) && \
    wget -nv -O v1.3.0.tar.gz https://github.com/edenhill/librdkafka/archive/v1.3.0.tar.gz && \
    tar -xz --strip-components=1 -f v1.3.0.tar.gz && \
    rm -f v1.3.0.tar.gz && \
    ./configure --prefix=/usr --enable-sasl && \
    make -j && \
    make install

# second stage
FROM go_kafka_base as build_base
WORKDIR /src
COPY go.mod .
COPY go.sum .
RUN go mod download

# third stage
FROM build_base AS builder
WORKDIR /src
COPY . .
RUN echo "Branch: $GIT_BRANCH"
RUN CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags static_all,netgo -o /bin/app -ldflags "-w -s" ./cmd/**/main.go

# final stage
FROM alpine:3.10
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=builder /bin/app /bin/app
EXPOSE 80
ENTRYPOINT ["/bin/app"]
blaketastic2 commented

golang 1.14.2, debian 10.3-slim, librdkafka 1.4.0(bundled)

# first stage
FROM golang:1.14.2-stretch as build_base
WORKDIR /src
COPY go.mod .
COPY go.sum .
RUN go mod download

# second stage
FROM build_base AS builder
WORKDIR /src
COPY . .
RUN CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags netgo -o /bin/app -ldflags "-w -s -X" ./cmd/**/main.go

# final stage
FROM debian:10.3-slim
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates
COPY --from=builder /bin/app /bin/app
EXPOSE 80
ENTRYPOINT ["/bin/app"]
  • 👍3
  • 👎1
sruslanas commented

Or could be just with 2 stages, mine version.

FROM golang:1.14.2-alpine as builder
RUN apk add alpine-sdk
WORKDIR /go/app
COPY rest-api.go /go/app
COPY go.mod /go/app
COPY go.sum /go/app
RUN go mod download
RUN GOOS=linux GOARCH=amd64 go build -o rest-api -tags musl

FROM alpine:latest as runner
WORKDIR /root/
COPY --from=builder /go/app/rest-api .
ENTRYPOINT /root/rest-api
  • 👍15
  • ❤️6
arcosx commented

very small binaries

FROM golang:alpine AS builder

# Set necessary environmet variables needed for our image
ENV GO111MODULE=on \
    CGO_ENABLED=1  \
    GOARCH="amd64" \
    GOOS=linux   

# Move to working directory /build
WORKDIR /build

# Copy and download dependency using go mod
COPY go.mod .
COPY go.sum .
RUN go mod download

# Install Compile plug

RUN apk -U add ca-certificates
RUN apk update && apk upgrade && apk add pkgconf git bash build-base sudo
RUN git clone https://github.com/edenhill/librdkafka.git && cd librdkafka && ./configure --prefix /usr && make && make install

# Copy the code into the container
COPY . .

# Build the application
RUN go build --ldflags "-extldflags -static" -o main .

# Move to /dist directory as the place for resulting binary folder
WORKDIR /dist

# Copy binary from build to main folder
RUN cp /build/main .

# Build a small image
FROM scratch

COPY --from=builder /dist/main /

# Command to run
ENTRYPOINT ["/main"]
  • 👍8
  • 👀2
divo77 commented

I have no problem to run the client on my mac ( golang:1.14.2 , librdkafka-1.4.2) but if I run the code from the docker image using any of those docker files I have a producer returns the status over the channel like:

     eproduce := <-ks.deliveryChan
messageProduce := eproduce.(*kafka.Message)

so up till this spot it works fine reading the messages but then just hang on eproduce := <-ks.deliveryChan and waiting. Any ideas why is it happening ?

honne23 commented

Production grade build adapted from here with unprivileged access. Only constraint is the golang-1.14 image but this can be alleviated with some of the commands from @arcosx:

FROM golang:1.14.2-stretch as builder

RUN apt update && apt install git ca-certificates gcc -y && update-ca-certificates

ENV USER=appuser
ENV UID=10001
# See https://stackoverflow.com/a/55757473/12429735RUN 
RUN adduser \    
    --disabled-password \    
    --gecos "" \    
    --home "/nonexistent" \    
    --shell "/sbin/nologin" \    
    --no-create-home \    
    --uid "${UID}" \    
    "${USER}"

WORKDIR $GOPATH/src/mypackage/myapp/
COPY . .

RUN go mod download
RUN go mod verify

RUN CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags netgo -o /go/bin/main --ldflags "-extldflags -static" ./cmd/main.go              

FROM scratch

COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group


COPY --from=builder /go/bin/main /go/bin/main

USER appuser:appuser
ENTRYPOINT ["/go/bin/main"]
  • 👍1
emretanriverdi commented

Can anyone from here help me about this issue?

#644

patricksferraz commented

For me the example of @honne23 in golang 17 works by adding the musl tag:

FROM golang:1.17-alpine AS builder

ENV PATH="/go/bin:${PATH}"
ENV GO111MODULE=on
ENV CGO_ENABLED=1
ENV GOOS=linux
ENV GOARCH=amd64

WORKDIR /go/src

COPY go.mod .
COPY go.sum .
RUN go mod download

RUN apk -U add ca-certificates
RUN apk update && apk upgrade && apk add pkgconf git bash build-base sudo
RUN git clone https://github.com/edenhill/librdkafka.git && cd librdkafka && ./configure --prefix /usr && make && make install

COPY . .

RUN go build -tags musl --ldflags "-extldflags -static" -o main .

FROM scratch AS runner

COPY --from=builder /go/src/main /

EXPOSE 8080

ENTRYPOINT ["./main"]
  • 👍8