getporter/porter

Feat: Add cli flag for use template dockerfile as is

Closed this issue · 7 comments

alekseybb197 commented

Is your feature request related to a problem? Please describe.
I use porter in my work. And I'm having trouble on a number of occasions:

  • building cnab on an isolated network;
  • building an invocation image using a customized base image;
  • building multi-stage or distroless image;
  • reliable assembly reproducibility.

In all of the above cases, it is required to significantly rewrite the final Dockerfile for the invocation image. However, the current code does not allow this.

The Dockerfile is currently being built in pkg/build/dockerfile-generator.go.
Variables, some of which are hardcoded, are defined in pkg/build/build.go.

Common problems:

  • all lines are hardcoded, which does not allow organizing convenient process control by specifying the necessary lines in the configuration file;
  • the default policy is that even if anchor lines are not specified in the Dockerfile template, the replaceTokens procedure will in any case add the required sections to the end of the generated Dockerfile;
  • and there is also no way to force porter to use a fully custom template.

Describe the solution you'd like
To solve these problems, I think it will be enough to add a command line flag that will override the default behavior. The default mode will use the old way of creating a call image, and if you specify a custom mode, the custom template will be used as is.

schristoff commented

Hey @alekseybb197 - thanks for bringing this up!
From the title of the issue, it seems like you want to use the generated template.Dockerfile as is, is that correct?
When I cat that file, it says the following:

# Add the following line to porter.yaml to instruct Porter to use this template
# dockerfile: template.Dockerfile

When you add this to your porter.yaml is this working properly?
You also mentioned building things on an isolated network, I'm wondering if you have peeked at this documentation on using Porter in an airgapped environment? If that documentation isn't correct or you're coming into a different problem, let me know.

alekseybb197 commented

Hi @schristoff !

I suggest a way to completely eliminate dockerfile generation. And instead use the template without modification.

My suggestion only applies to the build phase. It is then that the image is assembled, which uses references to external artifacts helm, kubectl and distribution packages.

I built a porter with my fix #2803 and we are now successfully using it on our pipelines. The assembly is done using only resources within our network.

Here is an example of a typical build template with the helm mixin:

# syntax=docker.artifacts.examples.com/tools/dockerfile-upstream:1.4.0
FROM docker.artifacts.examples.com/ubuntu:22.04@sha256:67211c14fa74f070d27cc59d69a7fa9aeff8e28ea118ef3babc295a0428a6d21

ARG BUNDLE_DIR
ARG BUNDLE_UID=65532
ARG BUNDLE_USER=nonroot
ARG BUNDLE_GID=0
RUN useradd ${BUNDLE_USER} -m -u ${BUNDLE_UID} -g ${BUNDLE_GID} -o

# code placeholder. insert here!

RUN curl -LO https://artifactory.artifacts.examples.com:443/artifactory/tools/kubernetes/v1.26.0/kubectl &&
mv kubectl /usr/local/bin/kubectl &&
chmod +x /usr/local/bin/kubectl
ENV HELM_EXPERIMENTAL_OCI=1
RUN curl -k https://artifactory.artifacts.examples.com:443/artifactory/tools/kubernetes/helm-v3.8.2-linux-amd64.tar.gz
--output helm3.tar.gz &&
tar -xvf helm3.tar.gz &&
rm helm3.tar.gz &&
mv linux-amd64/helm /usr/local/bin/helm3

RUN rm -rf /var/lib/apt/lists

COPY --link . ${BUNDLE_DIR}

RUN rm ${BUNDLE_DIR}/porter.yaml
RUN rm -fr ${BUNDLE_DIR}/.cnab
COPY --link .cnab /cnab
RUN chgrp -R ${BUNDLE_GID} /cnab && chmod -R g=u /cnab
USER ${BUNDLE_UID}
WORKDIR ${BUNDLE_DIR}
CMD ["/cnab/app/run"]

Maybe I didn't describe the problem accurately. Please excuse me. Ready to fix.

alekseybb197 commented

And this is an example of pipeline code

- task: CmdLine@2
  displayName: Porter install
  inputs:
    workingDirectory: $(workingDirectory)
    script: >
      curl -k -s https://artifactory.artifacts.examples.com:443/artifactory/tools/binaries/porter-1.0.14-custom-dockerfile.tgz -o porter.tgz
      ls -als
      tar zxvf porter.tgz
      export PATH=$PATH:`pwd`/.porter
      echo "autobuild-disabled: true" >>.porter/config.yaml
      porter version
      HOME=`pwd` porter mixin list
      ls -als
- task: Docker@2
  displayName: Docker login
  inputs:
    containerRegistry: $(Registry)
    command: login
- task: CmdLine@2
  displayName: Porter build
  name: build_cnab
  inputs:
    workingDirectory: $(workingDirectory)
    script: >
      set -e
      export PATH=$PATH:`pwd`/.porter
      export HOME=`pwd`
      chmod 755 *.sh
      ls -las
      porter build --verbosity=debug --custom-dockerfile
      docker images -a
      set -e
      docker images -a | grep "$(DockerImageNameCNAB)"
      id=`docker images -a | grep "$(DockerImageNameCNAB)" | head -1 | awk '{print $3}'`
      echo docker tag $id "$(Registry)/$(DockerImageNameCNAB):$(CNABversion)"
      docker tag $id "$(Registry)/$(DockerImageNameCNAB):$(CNABversion)"

Autobuild-disabled is mandatory in this case!

schristoff commented

Would you be willing to discuss this feature at the community meeting next Thursday? I think it would be great for us all to come together and get a shared understanding on this! (Or whatever Thursday you make be available)

  • 👍1
alekseybb197 commented

Thanks @schristoff ! One day ) I am now able to allocate a limited amount of time to work on this project. Regrettably, not regularly.

jarnfast commented

There is a lot of focus on Supply Chain Security and having options for bypassing the (somewhat opinionated) Dockerfile generated by Porter could be useful in situations where more control of the generated image is required.

Having a dedicated CLI flag for using the Dockerfile template as-is may not be the best way and I propose using the experimental flag capability instead (which means we can control it on the command-line or use environment variables).

Somewhere down the road we may get an awesome multi-arch, multi-stage setup that removes the need for the override capability.. 🤞

One use case that I'm currently prototyping is the possibility to reduce the size of the produced invocation images - which is nice when doing air-gapped bundles (others may not care as OCI layers also can be used for minimizing storage consumption in registries).
Some of the Dockerfile generated by Porter causes the .cnab directory to be duplicated in a few layers and mixins may not clean up the apt cache - using multi-stage builds to flatten the invocation image drastically reduces the size.

This can be quickly demonstrated by taking the hello bundle and adding a few mixins:

mixins:
  - exec
  - helm3:
      clientVersion: v3.14.0
  - kubernetes:
      clientVersion: v1.28.6

The resulting invocation image:

$ docker image ls | grep hello
localhost:5000/porter-hello   porter-37da5464f8517662657529ad34851db9   e4de17e20904   5 minutes ago   1.35GB

(disclaimer, some of the mixins have runtimes for Windows and Linux included - thus inflating the size a bit in this example)

Looking at the layers we see:

$ du -sm *
45      02814521c4c2ed717df1fc16f35f3f593e88c4b9b4ff7c62223c24cfffa30b70
279     14fb20875a1619ed6170ea4bfef7e9a9a3092d88e50cb73cefe84d3e91821e47      # .cnab related
10      23628ae557defb669240b7535f606971b8a2c293f14dac230fe7b7cada971044
49      26a8ac49d846f28d738ea9e4376008a355428d6444782eac3d9f62326804f4fa
1       44f1f6529ea5203377476cddb9ac6ba437bab9504041216fb660080ca7ab3961
118     663a63191097190559d382a0632de33e74d02ff952353fa0455423a43f65f295
1       73656d836b9beed0d1278a85f23c47ef64d07f7a3442e1fbcb01b5872802574b
49      984b1de119ea6ee694574e5a837e56466797559cf20325d95ec85c643d8b7c28
75      9eb95fc56d59c63d3d97397fbe6036bb3e377b5ae2f1b182a058ef75432c3379
1       a69c7848c53355f91fb65032a169b69070e9efd1dd6249e3a09dd8a87a63dd3b
1       a9cf22df33fca769cd094f06c411405fefa3a285278065a40d7b96a2405404e3
279     adc4cdde769a620e90cba723d88bf213b5429e2238bee25653276d574c7ee464      # .cnab related
16      b16b7414621829b73725fd6ba8b00c810859b6d85996a6be69ff37bcef8390e6
279     b705608e28be4c929bf09a667242fc3a00368a89936a9f2136e80104b8dffdcc      # .cnab related
1       b81d23fcb94571865369200279b2d4269289686db3b5cf9105eee50089fdc608.json
1       cce8f39b8625e852096d7abfb351393c6df3503be52412a9ed1e5cf7f0dcd2c4
97      f838e6d6d53abc4a6ce1a91f81a4229fc517234aab87fccd63e9e36a832974f7
1       manifest.json
1       repositories

If an override was possible one could produce a multi-stage that flattens the image, snip from custom Dockerfile

# syntax=docker/dockerfile-upstream:1.4.0
FROM --platform=linux/amd64 debian:stable-slim as builder

... <snipped porter args and mixins injections but the worst duplication culprit is the commands below>

# Use the BUNDLE_DIR build argument to copy files into the bundle's working directory
COPY --link . ${BUNDLE_DIR}
RUN rm ${BUNDLE_DIR}/porter.yaml
RUN rm -fr ${BUNDLE_DIR}/.cnab
COPY --link .cnab /cnab
RUN chgrp -R ${BUNDLE_GID} /cnab && chmod -R g=u /cnab

# clean-up on aisle 9
RUN apt-get clean

# flatten
FROM scratch
COPY --from=builder / /

USER ${BUNDLE_UID}
WORKDIR ${BUNDLE_DIR}
CMD ["/cnab/app/run"]

Which results in a way smaller image:

$ docker image ls | grep custom-porter-hello
custom-porter-hello          latest                      361c0e2b8e97   2 minutes ago      500MB

Disclaimer: I'm not privy to the information on why the PR produced by @alekseybb197 was axed - but maybe it's time to reconsider if this functionality could be "MVP" while moving towards "the correct solution"

alekseybb197 commented

Thank you, @jarnfast !

That's roughly how we reasoned when we promoted this solution.
Right now, the code with the specified option is running on the build cnab. https://github.com/alekseybb197/porter/tree/lager

We also solved problems in unnecessary duplication within cnab. This code was handled not by me, but by Sergey Permyakov https://github.com/PowerStateFailure.

Next is a typical dockerfile

# syntax=registry.inside.localnet/docker/dockerfile-upstream:1.4.0
FROM registry.inside.localnet/docker/base-images/ubuntu:22.04@sha256:845a92677c295324bac21a57cbd49bc878e6d8d091a6a3c1efc48f6697130a57

ARG BUNDLE_DIR
ARG BUNDLE_UID=65532
ARG BUNDLE_USER=nonroot
ARG BUNDLE_GID=0
RUN useradd ${BUNDLE_USER} -m -u ${BUNDLE_UID} -g ${BUNDLE_GID} -o

# PORTER_INIT

RUN apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get -y install \
        curl=7.81.0-1ubuntu1.16 less=590-1build1 \
        nano=6.2-1 jq=1.6-2.1ubuntu3 inetutils-ping=2:2.2-2 && \
    rm -rf /var/lib/apt/lists

RUN curl -k -s https://registry.inside.localnet:443/tools/cnab/yq/v4.30.7/yq_linux_amd64 -o yq && \
    mv yq /usr/local/bin/yq && \
    chmod +x /usr/local/bin/yq && \
    curl -k -s https://registry.inside.localnet:443/tools/cnab/helm3/v3.8.2/helm-v3.8.2-linux-amd64.tar.gz -o helm3.tar.gz && \
    tar -xvf helm3.tar.gz && rm helm3.tar.gz && \
    mv linux-amd64/helm /usr/local/bin/helm3 && \
    chmod +x /usr/local/bin/helm3 && \
    rm -rf linux-amd64 && \
    curl -k -s https://registry.inside.localnet:443/tools/kubernetes/v.1.26.0/kubectl -o kubectl && \
    mv kubectl /usr/local/bin/kubectl && \
    chmod +x /usr/local/bin/kubectl

## I like dockle!
RUN rm -rf /var/lib/apt/lists

COPY --chown=${BUNDLE_UID}:${BUNDLE_GID} --link .cnab /cnab

# Use the BUNDLE_DIR build argument to copy files into the bundle's working directory
COPY --chown=${BUNDLE_UID}:${BUNDLE_GID} --link charts ${BUNDLE_DIR}/charts
COPY --chown=${BUNDLE_UID}:${BUNDLE_GID} --link *.sh ${BUNDLE_DIR}

# helm dep update
RUN cd /cnab/app ; \
    for i in $(ls charts) ; \
    do /usr/local/bin/helm3 dep update charts/$i ; \
    done

USER ${BUNDLE_UID}
WORKDIR ${BUNDLE_DIR}
CMD ["/cnab/app/run"]

In addition I also created a PoC distroless invocation image
https://github.com/alekseybb197/diet-cnab