rpi-image-gen

rpi-image-gen is a tool used to create custom software images for Raspberry Pi devices. rpi-image-gen runs best on a Raspberry Pi Host running up-to-date 64-bit Raspberry Pi OS.

For the tool used to create Raspberry Pi OS, please go to https://github.com/RPi-Distro/pi-gen

NOTE rpi-image-gen is under active development. Please report issues at https://github.com/raspberrypi/rpi-image-gen. Feature suggestions are welcome.

Terminology

The following terms are used in this document:

Host The machine on which rpi-image-gen is executing. Analogous to GNU Build.

Target The machine on which the image generated by rpi-image-gen will execute. Analogous to GNU Host.

Package A collection of files and software resources, encapsulated in a known format, that constitute part of a larger software system.

Config A file that defines the target rootfs profile, disk image configuration identifier and top-level attributes.

Profile A file that contains the YAML layers for the target rootfs.

Image Identifies the configuration used to create the target disk image file.

Layer A single file of YAML metadata.

Meta A directory containing YAML layers in a defined hierarchy.

Hook A shell script that may be optional and which if found, will be run at a defined point in a particular flow of execution.

SBOM Software Bill Of Materials

CVE Common Vulnerabilities and Exposures

TL;DR

git clone https://github.com/raspberrypi/rpi-image-gen.git
cd rpi-image-gen
sudo ./install_deps.sh
./build.sh

The target image will be in work/deb12-arm64-min/artefacts/deb12-arm64-min.img

Install it onto an SD card using Raspberry Pi Imager (https://www.raspberrypi.com/software/). Select "Use Custom" if using the GUI. If using the command line and /dev/mmcblk0 is the device you want to install to:

sudo rpi-imager --cli work/deb12-arm64-min/artefacts/deb12-arm64-min.img /dev/mmcblk0

Note: The default image intentionally has login passwords disabled. Please look in the examples directory to help understand how to customise the installation of packages and make your own modifications.

Why use this?

Quick to build. You don’t have to build the whole world from source.
By using the same binaries as millions of people are using in production worldwide, you know you’ve not accidentally inserted vulnerabilities with your choice of tool chain, optimisations, build flags, etc.
Use the same version set of libraries and applications as you do on Raspberry Pi OS. You won’t have to install a bleeding edge version of a library to build your application.
Configure your filesystem exactly how you want it and use rpi-sb-provisioner (https://github.com/raspberrypi/rpi-sb-provisioner) to automatically setup and enable signed boot and encrypted filesystems.
Output your software bill of materials (SBOM) to list the exact set of packages that were used to create the image.
Generate a list of CVEs identified from the SBOM and give consumers of your image confidence that your image does not contain any known vulnerabilities.

Getting started

git clone https://github.com/raspberrypi/rpi-image-gen.git

Dependencies

To install the required Host dependencies for rpi-image-gen you should run:

sudo ./install_deps.sh

The file depends contains a list of the dependencies required. The format of this file is <tool>[:<debian-package>].

Overview

rpi-image-gen is a root file system and image creation tool designed with a high degree of customisation, flexibility and control in mind. rpi-image-gen uses bdebstrap (https://github.com/bdrung/bdebstrap) and mmdebstrap (https://gitlab.mister-muffin.de/josch/mmdebstrap) for rootfs construction and genimage (https://github.com/pengutronix/genimage) for image creation. rpi-image-gen is a Bash orientated scripting engine capable of producing software images with different on-disk partition layouts, file systems and profiles using collections of metadata and a defined flow of execution. It provides the means to create a highly customised software image for your Raspberry Pi device. rpi-image-gen is human readable, auditable and easy to use.

A target system is created using the concept of 'collections' where a collection is some YAML metadata in a defined format that describes a list of packages to be installed in, and operations to be performed on, the chroot (i.e. the target rootfs). Multiple YAML layers can be grouped into customisable profiles with all layers being aggregated and merged by bdebstrap. Board and image layout hooks are supported which enable a user to do things like apply a static rootfs overlay directory hierarchy specific to their board, perform custom post-build operations, integrate with third-party build systems, and have complete control over the input genimage configuration file(s) used to produce the target disk image(s).

rpi-image-gen heavily favours a Debian-ish installation process where a temporary apt configuration is assembled and used to seed the chroot. However, unlike debootstrap, mmdebstrap provides many more configuration options which can be defined in the YAML layers read by bdebstrap. This make it possible not only to describe different variants of the base rootfs in a modular way (e.g. ranging from a 'standard' Debian install with Essential packages to a rootfs that consists of nothing more than busybox, libc and a few basic utils), but also to use multiple mirrors in which to retrieve packages, and be able to perform customisable operations at defined stages of chroot creation, e.g. at 'setup', 'extract', 'essential', 'customize' and 'cleanup' points.

rpi-image-gen runs as a regular user and does not require root privileges.

Directory Hierarchy

The rpi-image-gen tree structure contains individual sub-component directories, each of which has a specific purpose in the execution flow. Certain directories may be of particular interest when integrating with rpi-image-gen because it’s possible to set their location to within an externally provided directory. Doing so would, for example, enable a user to:

Use one of the in-tree image layouts, but use their own board directory (e.g to apply a custom product specific roofs overlay, execute hooks to perform bespoke operations).
Augment the YAML layer search with a directory of their choice (e.g. where they own all layers or to integrate with a third-party).
Define their own image partition layout, file system types, mount points, etc.
Use a custom profile which contains a combination of in-tree layers and/or their own (e.g. customise the packages to install, perform specific chroot operations, etc).

EXT_DIR

If rpi-image-gen is provided with a path to an external directory on the command line it will use this directory when resolving sub-components paths for config/, profile/, image/ and board/ and it will include the meta/ sub-directory of this path in the list when searching for YAML meta layers defined by the Profile. The external directory does not need to contain all sub-component directories. If no external directory is provided, variable IGTOP defines the root directory for all sub-components.

Config

The config file is provided as an command line argument and is the first asset to be located, either from the external directory or in-tree. Once the config is loaded and parsed, sub-components for board, profile and image are searched for and their paths resolved, either from the external directory or in-tree. rpi-image-gen will always prioritise the external directory when searching.

Board

The board directory contains board specific assets, e.g. rootfs overlay and hooks.

Image

The image directory contains the necessary assets with which rpi-image-gen will use to create the output disk image(s).

Profile

The profile is a plain text file which supports comments via # and where each line contains a YAML layer. For example, if a Profile contained:

# My device layer
my/fantastic/layer

…my/fantastic/layer.yaml would be searched for.

Other sub-component directories exist for particular purposes and the path to some of those are propagated to all layers via dedicated variables. These are mainly to assist with scripting and template generation, e.g. when creating a systemd config fragment for a network interface, creation of RPi boot firmware config files, etc.

Options

The intention is for rpi-image-gen to have parity with all of the options supported by pi-gen, either as a 1-1 mapping or by functional equivalence. At the current time, this is not the case. The following options are supported and can be specified in the Options file or in the appropriate section of the Config file. Please note there is currently no support for reading them from the environment.

TARGET_BOARD (Default: pi5)
```
The name of the board to build for.
```
IMAGE_VERSION (Default: Today’s date represented as YYYY-MM-DD)
```
Version string of the image to build.
```
IMAGE_NAME (Default: <TARGET_BOARD>-<config name>-<IMAGE_VERSION>)
```
The basename of the image to build.
```
IMAGE_SUFFIX (Default: img)
```
The suffix of the generated image(s).
```

IMAGE_COMPRESSION (Default: none)

Identifier for the compression scheme used when deploying images and assets.

WORK_DIR (Default: <IGTOP>/work/<IMAGE_NAME>)

Root of the directory hierarchy containing rpi-image-gen output artefacts. Note, depending on the system being built, this directory can amount to a substantial amount of consumed disk space.

IMAGE_OUTPUTDIR (Default: <WORK_DIR>/artefacts)
```
Location of all build product artefacts.
```

IMAGE_DEPLOYDIR (Default: <WORK_DIR>/deploy)

Location where final images and assets will be installed to.

EXT_DIR (Default: unset)

Path to an external directory of where to search for sub-components and meta layers. This is set automatically via the the command line.

EXT_NSDIR (Default: unset)

Path to an external directory namespace of where to search for addition meta layers. This is set automatically via the the command line.

APT_KEYDIR (Default: <WORK_DIR>/keys)

If a particular collection of keys are required for bdebstrap to download packages from the mirror(s) provided, set the directory containing them here. This will be passed to bdebstrap via aptopt Dir::Etc::TrustedParts. If not specified, rpi-image-gen assembles the keys it requires into this directory. This particular setting of Dir::Etc::TrustedParts will not be included in the image. If using this option, please make sure to install your key(s) into the chroot explicitly if a key contained in this directory points to a location that is not otherwise populated during chroot creation (for example by installing a keyring package).

APT_PROXY (Default: unset)

If you require the use of an apt proxy, set it here. This proxy setting will not be included in the image, making it safe to use an `apt-cacher` or similar package for development.
Also see `APT_PROXY_HTTP`.

APT_PROXY_HTTP (Default: unset)

If you require the use of an apt http proxy, set it here. This proxy setting will not be included in the image, making it safe to use an `apt-cacher` or similar package for development.
To maintain compatibility with pi-gen, this variable will inherit the value of `APT_PROXY` if that variable is set.

LOCALE_DEFAULT (Default: 'en_GB.UTF-8' )
```
Default system locale.
```

TARGET_HOSTNAME (Default: 'raspberrypi' )

Sets the hostname to the specified value.

KEYBOARD_KEYMAP (Default: 'gb' )

Default keyboard keymap.

To get the current value from a running system, run `debconf-show keyboard-configuration` and look at the `keyboard-configuration/xkb-keymap` value.

KEYBOARD_LAYOUT (Default: 'English (UK)' )

Default keyboard layout.

To get the current value from a running system, run `debconf-show keyboard-configuration` and look at the `keyboard-configuration/variant` value.

TIMEZONE_DEFAULT (Default: 'Europe/London' )

Default timezone. Also see `TIMEZONE_AREA` and `TIMEZONE_CITY`.

To get the current value from a running system, look in `/etc/timezone`.

TIMEZONE_AREA (Default: 'Europe' )

Default timezone area. To maintain compatibility with pi-gen, this variable will inherit the first part of `TIMEZONE_DEFAULT`.

TIMEZONE_CITY (Default: 'London' )

Default timezone city. To maintain compatibility with pi-gen, this variable will inherit the second part of `TIMEZONE_DEFAULT`.

FIRST_USER_NAME (Default: pi)

Create a user account with this username. Please note that this user will *NOT* currently be renamed on the first boot.

FIRST_USER_PASS (Default: unset)

Password for the first user. If unset, the account is locked.

SBOM_OUTPUT_FORMAT (Default: spdx-json)
```
The output format of the SBOM.
```

Execution Flow

The three main stages of execution in rpi-image-gen are described below.

Input Parameter Assembly

Before commencing creation of the rootfs, the configuration of the system is assembled and translated into a set of environment variables established from different sources in the following order:

Default settings
Config file settings
User option file settings

The config file uses .ini file syntax (https://en.wikipedia.org/wiki/INI_file) to set key=value pairs.
The options file is a shell fragment containing key=value pairs.

rpi-image-gen adopts a system-wide prefix for all variables it exposes to YAML layers and hooks. The prefix is IGconf. In addition, when reading the config file, a variable declared in a section has the section name included in its prefix when it’s translated. Sections not specifically read will be ignored. Variables set from the options file will be prefixed and translated as-is. Translation includes the variable name being converted to lower case.

Example

These are equivalent:

Default

IGconf_target_board=pi5

Config

[target]
board=pi5

Options

TARGET_BOARD=pi5

Processing the input sources and aggregating the variables this way allows the setting of a variable to override a previous setting. This may be particularly useful where customisation may only require the modification of a small number of variables compared with what was set previously. Furthermore, rpi-image-gen evaluates each variable after translation which means that variables set previously can be used to set other variables.

Example

Default

IGconf_target_board=pi5
IGconf_image_suffix=img

Options

image_suffix=${IGconf_target_board}.bin

Yields:

IGconf_image_suffix=pi5.bin

The rpi-image-gen options file serves the same purpose as the pi-gen config file. It is envisaged that pi-gen users can use the same file to reflect their customisations when creating an image with rpi-image-gen, although at the current time support for an identical set of options is incomplete.

After processing the sources, all IGconf variables are included in the environment of subsequent stages. This means that YAML layers and hooks have access to all IGconf variables.

Tip

Via the Options file or a supported Config file section, it is possible to define new custom variables. rpi-image-gen does not 'filter' variables or perform any sort of manipulation of their values/contents. The propagation of all variables, including user defined custom variables, may be beneficial to YAML layers and hooks.

Example

Default

IGconf_first_user_name=pi

Config

[system]
flavour=debug
debugger=autoattach

Options

system_debug_port=8080
system_debug_user=$IGconf_first_user_name

Yields

IGconf_system_flavour=debug
IGconf_system_debugger=autoattach
IGconf_system_debug_port=8080
IGconf_system_debug_user=pi

Understanding how to set and create variables is an important part of using rpi-image-gen because it forms the foundation of user customisation.

Root File System Construction

After assembling the environment variables from the input sources, rpi-image-gen reads the Profile and validates the path of each YAML layer before assembling each layer as an argument to bdebstrap. A layer must adhere to bdebstrap YAML syntax. Please refer to the bdebstrap man page for further details. It’s also worth pointing out that if authoring shell expressions in YAML, it may be necessary to adopt usage of particular block scalar styles to achieve newlines inside a block. For example:

  - |-
    chroot $1 bash -- <<- EOCHROOT
    source /opt/device/setup.sh
    run_provisioning
    EOCHROOT

In addition to each YAML layer, all IGconf variables are also passed to bdebstrap as arguments. This enables access to these variables from all layers.

A number of 'core hooks' are installed via bdebstrap command line arguments. These are run in addition to shell operations specified via by the YAML and provide the means for rpi-image-gen to execute common hooks at particular points in the creation of the chroot. They also serve the purpose of providing functionality at defined points that a custom layer(s) may need. For example, it may be desirable for all initramfs kernel images installed in the chroot to be rebuilt before the target file system images are generated. It’s unlikely that this operation needs to be done more than once during chroot creation, so there is a hook that runs update-initramfs. This removes the need for a layer to invoke this command specifically and it helps to reduce potential complications that could arise depending on the aggregation order of layers. Because bdebstrap prioritises hooks which are provided as command line arguments, their execution point in the flow is deterministic.

rpi-image-gen runs bdebstrap in a new Linux namespace via podman unshare. This is required so that bdebstrap creates files with the correct ownership information, which is particularly important when creating a chroot that contains a user account. See podman-unshare(1) and user_namespaces(7) for further details.

Before bdebstrap invokes mmdebstrap to begin creation of the chroot, it writes the fully aggregated and merged YAML to $IGconf_image_outputdir as config.yaml. This is an incredibly useful file because it’s essentially the 'recipe' for generating the chroot. It also creates a file called manifest in the same directory which lists all the packages that were installed in the chroot.

The mmdebstrap execution is not regarded as in the scope of this document. mmdebstrap follows the rules governed by its design and by the configuration provided to it by bdebstrap. In turn, the vast majority of bdebstrap configuration is derived from the YAML layers provided to it by rpi-image-gen. From this point of view, rpi-image-gen could be regarded as a thin toolkit wrapper with which to design a system purely from YAML constructs and a set of environment variables. The application of both these things, once understood, is very powerful and provides the means to create a completely customised rootfs.

For example, it is possible to create a usable, minimal chroot with only the following YAML layer:

---
name: bookworm-arm64-svelte
mmdebstrap:
  architectures:
    - arm64
  mode: auto
  variant: custom
  suite: bookworm
  mirrors:
    - deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
    - deb http://deb.debian.org/debian-security bookworm-security main contrib non-free non-free-firmware
    - deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware
  packages:
    - dpkg
    - busybox
    - libc-bin
    - base-files
    - base-passwd
    - debianutils

SBOM

A SBOM should be a reproducible and immutable artefact of the build. Generation of it takes place at the end of the post-build stage after all other hooks have run, and immediately before genimage invocation. It is not advised to perform any operation on the rootfs that could affect the data encapsulated by the SBOM after it has been generated. The SBOM is generated using syft (https://github.com/anchore/syft) which will be downloaded if not installed on the host. The SBOM is written to the image output directory (IGconf_image_outputdir) and its output format is configurable via the SBOM_OUTPUT_FORMAT variable which can be set via the Config or Options file. The output format is currently the only option available to the user.

For example, these are equivalent:

Config

[sbom]
output_format=cyclonedx-json

Options

SBOM_OUTPUT_FORMAT=cyclonedx-json

The value of IGconf_sbom_output_format is passed directly to syft. Please refer to the syft documentation for the supported output formats. Nothing in the rootfs is excluded from the SBOM scan. syft is invoked via podman unshare so it can execute with the same privileges as bdebstrap did when the rootfs was created.

Utilisation of the SBOM is beyond the scope of this document. However, the following is offered as an example of how the SBOM could be used to generate a report of known CVEs using grype (https://github.com/anchore/grype):

$ grype --by-cve sbom:./work/deb12-arm64-min/artefacts/deb12-arm64-min.sbom -o json > vulnerabilities.json

Image Generation

After the chroot is successfully created (i.e. 'post-build'), file system overlays are applied (if present) before a number of hooks are run. These hooks provide additional integration points and provide a powerful way to perform custom operations on the chroot before image generation commences. One of hooks that runs at this point is the pre-image hook. Similar to bdebstrap invocation, every hook is invoked in an environment where all IGconf variables are available. Hooks are also invoked in the directory in which they exist, which means they can use relative paths to assets and sub-directories specific to their function.

The pre-image hook is called by the core engine with defined arguments such as the path to the chroot, and the path to the genimage input directory. It is the responsibility of this hook to perform the tasks required to create the file(s) (i.e. the templates) in the genimage input directory so that genimage can process them to create the target image(s). Failure to generate a file(s) of the expected naming convention (genimage*.cfg), or to use syntax that renders the file(s) unable to be read by genimage, will result in an error and no target image will be generated. Please refer to the genimage documentation for usage information and examples of creation templates, parameters, etc.

When invoking genimage, rpi-image-gen sets the value of --inputpath and --outputpath to the same location so that it’s possible to reference one image from another. For example, image1 may be a squashfs image containing various assets that needs to be present on a file system in image2 so it can be mounted at boot.

rpi-image-gen is again somewhat of a thin toolkit wrapper, this time leveraging genimage functionality with its documented arguments and parameters. Like bdebstrap, genimage is run via podman unshare which is necessary to enable file system creation utilities to incorporate correct and valid file system metadata from the chroot when creating the partition images, before assembling them into the final target image(s).

Overlays

An 'overlay' is a statically defined directory tree hierarchy that is copied into the chroot after bdebstrap completes execution. An overlay is identified by a directory named 'rootfs-overlay' and can reside in two independent locations:

<image dir>/rootfs-overlay (Usage: optional) (Provided-by: image)
```
Image layout specific root file system contents.
```
<board dir>/rootfs-overlay (Usage: optional) (Provided-by: board)
```
Board specific root file system contents.
```

Overlays are applied in the order they are listed above.

Hooks

The hooks available for user customisation are documented below. If a hook is to be executed, it must have executable permissions for the user performing the build.

post-build

<image dir>/post-build.sh (Usage: optional) (Provided-by: image)
```
Image layout specific post-build operations.
```
<board dir>/post-build.sh (Usage: optional) (Provided-by: board)
```
Board specific post-build operations.
```

Both hooks are executed in the order they are listed above.

pre-image

<board dir>/pre-image.sh (Usage: Mandatory - see notes below) (Provided-by: board)
```
Board owned pre-image generation operations.
```
<image dir>/pre-image.sh (Usage: Mandatory - see notes below) (Provided-by: image)
```
Image layout owned pre-image generation operations.
```

Only one of these hooks is executed. The board pre-image hook has priority. If it exists, it will be executed, else the image layout pre-image hook will be executed.

post-image

<board dir>/post-image.sh (Usage: Optional - see notes below) (Provided-by: board)
```
Board owned post-image operations.
```
<image dir>/post-image.sh (Usage: Optional - see notes below) (Provided-by: image)
```
Image layout owned post-image operations.
```

Only one of these hooks is executed. The board post-image hook has priority. If it exists, it will be executed. If it doesn’t and the image layout post-image hook exists, that will be executed. If neither exist, a default post-image hook will be executed. It is the responsibility of the post-image hook to perform all operations required to deploy artefacts to IMAGE_DEPLOYDIR (IGconf_image_deploy_dir), for example compressing image and SBOM.