Embedded Linux Build Root for Raspberry Pi 4 Step-By-Step.
In this tutorial we will build and configure a minimal image for raspberry pi 4 with customized toolchain, uboot, kernel and root filesystem.
- Linux Machine OS (ex: Ubuntu 20.04 LTS)
- Raspberry Pi 4
- SD Card >= 2gb
- SD Card Reader
- USB to TTL (PL2303)
- Configure the SD Card
- Configure and build the Toolchain
- Building the Bootloader
- Config and Build the Kernel
- Create the root filesystem
- Boot the RPI4
- Plug in the SD Card
- Open Disks app (Ubuntu)
- Format the SD Card
- Creat two partitions
- (a) boot : [100 - 500] MB = FAT format
- (b) root : Remaining size = Ext4 format
- Mount the Partitions from the "Play" Symbol
sudo apt -y install gcc-arm-linux-gnueabihf binutils-arm-linux-gnueabihf
sudo apt-get -y install bison flex bc libssl-dev make gcc automake chrpath g++ git gperf gawk help2man libexpat1-dev libncurses5-dev libsdl1.2-dev libtool libtool-bin libtool-doc python2.7-dev texinfo minicom
The toolchain is an essential tool to compile the source code to executable one runnning on the RPI4. We will use Crosstool-NG. Crosstool-NG build Explanation
cd ~
git clone https://github.com/crosstool-ng/crosstool-ng
cd crosstool-ng/
# Switch to the latest release
git checkout crosstool-ng-1.24.0 -b 1.24.0
./bootstrap
./configure --prefix=${PWD}
make
make install
export PATH="${PWD}/bin:${PATH}"
- Frst, we must configure the crosstool-NG to build the toolchain for the RPI4. You can find a sample toolchain for RPI3 ...
ct-ng list-samples
- At the time of writing, there are no configurations for RPi_4. To avoid creating one from scratch, we can use an existing configuration of RPi_3 and modify it to match RPi_4
ct-ng show-aarch64-rpi3-linux-gnu
- Select aarch64-rpi3-linux-gnu as a base-line configuration
ct-ng aarch64-rpi3-linux-gnu
- To configure the base line of the toolchain, we first need to know the CPU model of the RPI_4 which is
Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
ct-ng menuconfig
-
We will make 3 changes :
- Allow Extending the toolchain after it is created ...
Paths and misc options -> Render the toolchain read-only = false
- Change the ARM Cortex core ...
Target options -> Emit assembly for CPU: Change cortex-a53 to cortex-a72
- Chane the tuple’s vendor string ...
Toolchain options -> Tuple’s vendor string: Change rpi3 to rpi4
- Allow Extending the toolchain after it is created ...
-
Build the Toolchain :
ct-ng build
- The toolchain will be named aarch64-rpi4-linux-gnu and created in ...
${HOME}/x-tools/aarch64-rpi4-linux-gnu
The bootloader’s job is to set up the system to a basic level and load the kernel.
cd ~
git clone git://git.denx.de/u-boot.git
cd u-boot
git checkout v2021.10 -b v2021.10
export PATH=${HOME}/x-tools/aarch64-rpi4-linux-gnu/bin/:$PATH
export CROSS_COMPILE=aarch64-rpi4-linux-gnu-
make rpi_4_defconfig
make
- We will copy the u-boot binary file to the "boot" partition of the SD Card we created earlier. NOTE The mount destination may be different, ex: /media/${USER}/boot/.
sudo cp u-boot.bin /mnt/boot
NOTE: Raspberry Pi has its own proprietary bootloader, which is loaded by the ROM code and is capable of loading the kernel. However, since I’d like to use the open source u-boot, I’ll need to configure the Raspberry Pi boot loader to load u-boot and then let u-boot load the kernel.
- Download Raspberry Pi firmware/boot directory
cd ~
svn checkout https://github.com/raspberrypi/firmware/trunk/boot
- Copy RPI4 bootloader into boot partition
sudo cp boot/{bootcode.bin,start4.elf} /mnt/boot/
- Let RPI4 bootloader load U-Boot
cat << EOF > config.txt
enable_uart=1
arm_64bit=1
kernel=u-boot.bin
gpu_mem=512
EOF
sudo mv config.txt /mnt/boot/
Now we will compile the kernel Image.
Despite that the original Linux kernel from Linus Torvalds should work. But, using the RPI's fork of it would be more stable.
cd ~
git clone --depth=1 -b rpi-5.10.y https://github.com/raspberrypi/linux.git
cd linux
We just use the default config for the RPI4 model B, by the knowledge of its Specs.
PATH=$PATH:~/x-tools/aarch64-rpi4-linux-gnu/bin
make ARCH=arm64 CROSS_COMPILE=aarch64-rpi4-linux-gnu- bcm2711_defconfig
make -j$(nproc) ARCH=arm64 CROSS_COMPILE=aarch64-rpi4-linux-gnu-
Now we copy the kernel image and device tree binary (*.dtb) into the boot partition on the SD card.
sudo cp arch/arm64/boot/Image /mnt/boot
sudo cp arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b.dtb /mnt/boot/
We will Customize the Root filesystem folders and files.
mkdir rootfs
cd rootfs
mkdir {bin,dev,etc,home,lib64,proc,sbin,sys,tmp,usr,var}
mkdir usr/{bin,lib,sbin}
mkdir var/log
# Create a symbolink lib pointing to lib64
ln -s lib64 lib
tree -d
# Output ...
# .
# ├── bin
# ├── dev
# ├── etc
# ├── home
# ├── lib -> lib64
# ├── lib64
# ├── proc
# ├── sbin
# ├── sys
# ├── tmp
# ├── usr
# │ ├── bin
# │ ├── lib
# │ └── sbin
# └── var
# └── log
# 16 directories
# Change the owner of the directories to be root
# Because current user doesn't exist on target device
sudo chown -R root:root *
We’ll use Busybox for essential Linux utilities such as shell. So, we need to install it to the rootfs directory just created.
- Download Source code
wget https://busybox.net/downloads/busybox-1.33.2.tar.bz2
tar xf busybox-1.33.2.tar.bz2
cd busybox-1.33.2/
- Configuring
CROSS_COMPILE=${HOME}/x-tools/aarch64-rpi4-linux-gnu/bin/aarch64-rpi4-linux-gnu-
make CROSS_COMPILE="$CROSS_COMPILE" defconfig
# Change the install directory to be the one just created
sed -i 's%^CONFIG_PREFIX=.*$%CONFIG_PREFIX="/home/yusuf/rootfs"%' .config
- Building
make CROSS_COMPILE="$CROSS_COMPILE"
- Installing
# Use sudo because the directory is now owned by root
sudo make CROSS_COMPILE="$CROSS_COMPILE" install
- Next, we install some shared libraries required by Busybox. We can find those libraries by the following command ...
readelf -a ~/rootfs/bin/busybox | grep -E "(program interpreter)|(Shared library)"
# Output ...
# [Requesting program interpreter: /lib/ld-linux-aarch64.so.1]
# 0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
# 0x0000000000000001 (NEEDED) Shared library: [libresolv.so.2]
# 0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
- Coping these files from the toolchain’s sysroot directory to the rootfs/lib directory ...
export SYSROOT=$(aarch64-rpi4-linux-gnu-gcc -print-sysroot)
sudo cp -L ${SYSROOT}/lib64/{ld-linux-aarch64.so.1,libm.so.6,libresolv.so.2,libc.so.6} ~/rootfs/lib64/
Two device nodes are needed by Busybox.
cd ~/rootfs
sudo mknod -m 666 dev/null c 1 3
sudo mknod -m 600 dev/console c 5 1
sudo cp -r ~/rootfs/* /mnt/root/
Finally, with the Puzzle pieces are ready, we can boot the board using the U-Bootwe built earlier.
We need to configure the u-boot so that it can pass the correct kernel commandline and device tree binary to kernel. For simplicity, I’ll use the Busybox shell as the init program.
- Creating txt file for the boot command ...
- Load the kernel image from partition 1 (boot partition) into memory.
- Load the initramfs from partition 1 (boot partition) into memory.
- Set kernel commandline.
- Boot using the given kernel, device tree binary and initramfs.
cd ~/u-boot/
cat << EOF > boot_cmd.txt
fatload mmc 0:1 \${kernel_addr_r} Image
setenv bootargs "console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rw rootwait init=/bin/sh"
booti \${kernel_addr_r} - \${fdt_addr}
EOF
- Generating the boot.scr file
~/u-boot/tools/mkimage -A arm64 -O linux -T script -C none -d boot_cmd.txt boot.scr
- Copy the compiled boot script to boot partition
sudo cp boot.scr /mnt/boot/
At Last we can boot the Image on RPI4.
- Check on the boot partition files ...
tree /mnt/boot/
# OUTPUT ...
# /mnt/boot/
# ├── bcm2711-rpi-4-b.dtb
# ├── bootcode.bin
# ├── boot.scr
# ├── config.txt
# ├── Image
# ├── start4.elf
# ├── uRamdisk
# └── u-boot.bin
# 0 directories, 7 files
- Unmount the SD Card
- Plug the SD Card in the RPI4
- Connect RPI4 witg the usb TTL
- RX -> TX
- TX -> RX
- GND -> GND
- Connect the usb TTL to the Laptop
- In Terminal, Open minicom to read Serial from the RPI4 ...
cd /dev/
# Search for the TTL Port
dmesg | grep pl2303
# As default you will see it at ttyUSB0
sudo minicom -D ttyUSB0
- Power the RPI4 up
- Now you should see the u-boot and the kernel loading till the console, if the image have been created successfully.
- Creating the RPI4 Toolchain
- Hechao's Blog
- link
- Mastering Embedded Linux Programmin - Third Edition
- linux From Scratch
- How Toolchan is Constructed
Best Wishes :)