[RFC] Record installed slices in `--root` and install only what's missing

[woky]

Currently it's not efficient to use chisel for an image that's based on another image that was also built using chisel. This is related to the current lack of any metadata/database of installed slices in the final images.

There's a desire to have 2 kind of images for some programming language environments (for example .NET or Go):

• runtime-deps image: This image contains required dependencies to run self-contained applications compiled from an language SDK. Examples are self-contained application bundles in .NET or static binaries in Golang. These final products have no dependencies on the language runtime as they bundle it. But they still require libraries on which the language runtime depends.
• runtime image: This image is super-set of runtime-deps containing language runtime in addition to its dependencies.

The way we currently use chisel is something a little bit more complex than this illustrative example (that also pretends chisel is in PATH):

FROM ubuntu:22.04 as builder
RUN chisel cut --release ubuntu-22.04 --root /rootfs slice1_libs slice2_libs

FROM scratch
COPY --from=builder /rootfs /

If we apply this for the 2 images described above, we would end up with something like the following 2 Dokcerfiles.

For runtime-deps image:

FROM ubuntu:22.04 as builder
RUN chisel cut --release ubuntu-22.04 --root /rootfs libc6_libs libgcc-s1_libs libstdc++6_libs

FROM scratch
COPY --from=builder /rootfs /

For runtime image (here .NET is used as an example):

FROM ubuntu:22.04 as builder
RUN chisel cut --release ubuntu-22.04 --root /rootfs-dotnet dotnet-runtime-6.0_libs

FROM runtime-deps
COPY --from=builder /rootfs-dotnet /

Even though we base our final runtime image on runtime-deps, we effectively overwrite everything that was installed in runtime-deps. Chisel cannot know that dependencies of dotnet-runtime-6.0 have already been installed so whole dependency tree is populated in /rootfs-dotnet.

In this simple example one could suggest to not base runtime image on runtime-deps image but instead build it from scratch. But more complex runtime-deps image could have something more in addition to root filesystem created by chisel, e.g.:

FROM ubuntu:22.04 as builder
RUN chisel cut --release ubuntu-22.04 --root /rootfs libc6_libs libgcc-s1_libs libstdc++6_libs \
  && echo app:x:999:999:app:/nonexistent:/nonexistent >/etc/passwd \
  && echo app:x:999: >/etc/group

FROM scratch
COPY --from=builder /rootfs /
ENV DOTNET_VERSION=6.0

Here it makes sense to base runtime image on runtime-deps image (above) because of the additional files created outside of chisel and ENV directives. (There could be also HEALTHCHECK, PORTS and other directives that we would like to inherit in runtime image.)

It'd be nice if chisel had an ability to record what has already been installed, and install only missing dependencies that were not recorded. It's a bit complicated by the fact that we execute chisel in stage container that doesn't have access to the image on top of which we want to copy new slices. But that can be solved by copying the suggested chisel metadata/database into the stage container, e.g. like this:

FROM ubuntu:22.04 as builder
COPY --from=dotnet-runtime-deps /var/lib/chisel.db /rootfs-dotnet/var/lib/chisel.db
RUN chisel cut --release ubuntu-22.04 --root /rootfs-dotnet dotnet-runtime-6.0_libs

FROM runtime-deps
COPY --from=builder /rootfs-dotnet /

Now chisel could read state of /rootfs-dotnet from /rootfs-dotnet/var/lib/chisel.db (or wherever we decide to store the metadata).

One major problem with this is that it'll probably break the assumption that scripts run in a closed world and will lead to non-determinism. Currently each chisel run creates new world from scratch and so scripts runs can be scheduled to produce deterministic output. With such split installation it would no longer hold.

[cjdcordeiro]

Additional piece of information wrt security monitoring:

USN notification
service for ROCKs is only being able to monitor ROCKs based on Ubuntu
(i.e. the rock has a dpkg.query file listing Ubuntu pkgs and versions).

[mthalman]

I have a post which describes a workaround for this issue.

[loewenstein]

#25 got closed with

I think we can close this PR, since the underlying feature will be split into multiple feature requests

Can't seem to find those smaller feature requests linked here though.