About

The main idea is to have a tool for generating disk images from a textual description.

On the one hand, the descritpion provides the layout of the image (partition layout, UBI or LVM volumes, etc...) and what filesystems to install on the partitions in a way that can be nested arbitrarily deep.

On the other hand, the description provides a way to pack files into the filesystems (e.g. data from a tarball, a directory or a manual listing).

Right now, this is a project is in a not really useful, experimental stage, but advanced far enough that I dare to publish it as "work in progress".

Even the name itself is still work in progress.

Motivation

Let's say, you are playing around with an embedded board that boots of an SD card. You want to automatically create a customized image where most of the filesystem is inside a SquashFS, which is itself stored inside a FAT32 boot partition, together with the kernel and bootloader configuration files. The home directories (and possibly some other muteable state) are supposed to be stored on an extra partition with a mutable filesystem. The entire mount setup itself is done by an initrd with a custom script inside it.

You already have the entire rootfs in a tarball, generated by something like Yocto, mmdebstrap or another bootstrapping tool.

So now you bust out a plethorea of different programs (parted, mkfs*, etc...) to create the image, loop mount it so you can populate the partitions and so on and wrap all of that into a shell script.

Because the shell script loop mounts stuff, it needs to be run as root. Or you discover that various mkfs* tools have ways of pre-populating the filesystem that are of course different between tools and you still need temporary files and some dd magic to make it work with partitions. If the tools you use can only pack files from a directory, you may need to unpack the tarball and do some fakeroot magic.

Imagine, instead, you could simply describe everything you want in a simple configuration file as follows:

mbrdisk {
	type msdos
	align 1M

	partition {
		type primary
		boot true

		fat32 "bootvolume" {
			volumefile "/initrd.xz" {
				cpio "kernelinitrd" {
					compression "xz"
				}
			}

			volumefile "/rootfs.sqfs" {
				squashfs "rootfs"
			}
		}
	}

	partition {
		type primary

		ext4 "homedirs"
	}
}

mountgroup {
	bind "/:rootfs"
	bind "/home:homedirs"
	bind "/boot:bootvolume"

	tarunpack "rootfs.tar.gz"
}

mountgroup {
	bind "/:kernelinitrd"

	filelist "./" {
		dir dev 0755 0 0
		dir lib 0755 0 0
		dir bin 0755 0 0
		dir sys 0755 0 0
		dir proc 0755 0 0
		dir newroot 0755 0 0
		dir images 0755 0 0
		slink sbin 0777 0 0 bin
		nod dev/console 600 0 0 c 5 1
		file init 0755 0 0 initrd.sh
	}

	filter {
		allow "bin/busybox"
		allow "lib/ld*.so*"
		allow "lib/libc*.so*"

		tarunpack "rootfs.tar.gz"
	}
}

We have an DOS style partition table with 2 partitions, one is FAT32 formated, the other one ext4 formated. The FAT32 partition has a list of volumefiles, which is basically a way to create special files that can then function in the same way as a partition, with a nested filesystem on their own.

Each filesystem has a name attached that we then use to populate the filesystems. Each "mountgroup" specifies a list of named filesystems that data is directed to. A tarball is unpacked "into" the mountgroup, splitting up its contents across the different filesystems on different partitions. For the initrd, a listing similar to geninitramfs is used, in combination with a filter that splits out busybox and friends from the rootfs tarball.

Finally, a single program is run to generate the image from the description:

$ mkimage -c layout.cfg -O sdcard.img

Plans

Near term future

Input data source:

possibly compressed tarballs
manual, geninitramfs style listing
shell glob filter with other, nested sources

Output formats:

Raw (i.e. no headers)
MBR & GPT partition tables

Filesystems:

SquashFS (through libsquashfs)
FAT32
possibly compressed cpio
possibly compressed tar
zip

Medium term future