RFC: generalize Dockerfile manipulation for use outside OSBS

[ktdreyer]

When I write Jenkins jobs or other tools to build containers apart from OSBS, I find that I need to re-implement some of the things that OSBS' plugins do.

For example, when I write a Dockerfile for OSBS, I do not hard-code any Yum repository URLs inside the Dockerfile. This means I cannot docker build (podman build) my Dockerfile locally on my laptop. OSBS's pre_add_yum_repo_by_url.py and pre_inject_yum_repo.py modify the Dockerfile on the fly to inject the Yum repositories at build time. I have to do that by hand if I want to get the same results locally.

Similarly, with my Dockerfile for OSBS, I may have a FROM: line that refers to an internal registry, and my local docker (podman) client cannot resolve the FROM: image that I use. OSBS dynamically changes the FROM line at build time with pre_change_from_in_df.py. I have to make that FROM line change by hand if I want to build the image outside OSBS.

I have hacked together something with curl and sed for Jenkins, like this:

# Dynamically inject Yum repositories.
# We are going to use curl and sed to do a poor-man's replacement of what OSBS
# does dynamically with these plugins:
# https://github.com/projectatomic/atomic-reactor/blob/master/atomic_reactor/plugins/pre_add_yum_repo_by_url.py
# https://github.com/projectatomic/atomic-reactor/blob/master/atomic_reactor/plugins/pre_inject_yum_repo.py

curl -O http://file.example.com/~kdreyer/repos/rhel-example-nightly.repo
curl -O http://file.example.com/~kdreyer/repos/ceph-4.0.repo

sed -i '/^FROM /a ADD rhel-example-nightly.repo /etc/yum.repos.d/rhel-example-nightly.repo' Dockerfile
sed -i '/^FROM /a ADD ceph-4.0.repo /etc/yum.repos.d/ceph-4.0.repo' Dockerfile

I would like to improve this experience

When I build RPMs with fedpkg, I can run fedpkg build to send the Koji build RPC, or I can do fedpkg mockbuild to build locally, and the basic user experience is almost entirely the same.

It would be great to have an rpkg local-container-build command that would do the same thing for containers with podman locally. This would make the edit+build+test cycle very fast, because developers would not need to send every experimental change into Koji+OSBS+Pulp registry.

I would like to start by copying the logic from pre_add_yum_repo_by_url.py, pre_inject_yum_repo.py, and pre_change_from_in_df.py into a separate Python library that depends on https://github.com/DBuildService/dockerfile-parse .

I welcome feedback on this approach.

[rcerven]

I don't see any problem if you create separate python library which depends on dockerfile-parse
and modifies Dockerfile for your case

but keep in mind that osbs can use also repos from odcs and content sets,
which are defined in container.yaml and content_sets.yml