Continual-Learning-on-Max78000: A C repository from Gio200023

Continual Learning on Max78000 Microcontroller

Application of Continual Learning on Embedded Device
University of Trento

August 31, 2022

Repository Structure
Project Description
Tools Installation
Model Training and Synthesis
Build and Flash

Repository Structure

The repo is structured in the following components:

Latex: contains the source latex, the images used for the thesis and a PDF version of the thesis.
emnist_scripts: contains the EMNIST network model, the EMNIST dataset and scripts to generate samples input from the EMNSIT dataset.
python_scripts: contains python scripts used during the implementation of the project. Each script is commentend with the explanations of it.
sample_data_emnist: contains 1000 smaples from the EMNIST dataset. Those samples are all 'A'.
main.c: it is the head file of the project. This file contains the main pipeline of the CL system, from the CNN inferences to the weights update.
backpropagation.c: this file contain the algorithm applied to find/update the weights and the biases.
weights.c: contains the weights and biases array.
cnn.c: is the driver file to interact with the CNN accelerator.

Project Description

This project develops a Continual Learning system on an embedded device. Specifically, the device is the Max78000. The board supports a CNN accelerator that enables battery-powered applications to execute AI inferences while spending only microjoules of energy.

The nexts sections explain how to install the tools required to develop and deploy a project in Max78000. ---

Tools Installation

The SDK (Software Development Kit) is the most important tool to develop easily a project. Includint the SDK, the expected file system should be like this:

../ai8x-training/
../ai8x-synthesis/
../MaximSDK/

".." is the root project

Clone from Github the repositories required inside the root project (I cloned them into my $HOME directory):

$ git clone --recursive https://github.com/MaximIntegratedAI/ai8x-training.git
$ git clone --recursive https://github.com/MaximIntegratedAI/ai8x-synthesis.git
$ git clone --recursive https://github.com/MaximIntegratedAI/MAX78000_SDK

Note! Inside the synthesis folder there is already a SDK folder, I also cloned the third repository to have more clarity and division. They are the same repositories

Prerequisites

This software requires PyTorch. Since PyTorch training is very long and require a lot of computation, to train the network model a Cuda 11 device is higly recommended. Ubuntu 20.04 LTS is the official operating system supported by the Maxim tools. I worked on MacOS system to develop the project and used an external computer to train the network. Windows operating system is also supported using WSL2 (windows subsytem for Linux), so every step needed for the linux system are also needed for the windows WSL2 one.

Project Installation

Some base packages are required.

macOS

$ brew install libomp libsndfile tcl-tk

Linux

$ sudo apt-get install -y make build-essential libssl-dev zlib1g-dev \
  libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
  libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev \
  libsndfile-dev portaudio19-dev

Python 3.8 Pyenv

Python interpreter is almost important as the other tools. My reccomandation is to install pyenv to isolate the different packages needed by the synthetizer and the training folders.

MacOS

$ brew install pyenv pyenv-virtualenv

Linux

$ curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash  # NOTE: Verify contents of the script before running it!!

To enable pyenv each time it called, attached to the end of the .zshrc or .bashrc (based on your shell) the terminal output after installing Pyenv.

Then open a new terminal and type:

$ pyenv install 3.8.11

Once installed, create a new environment:

cd ai8x-training
pyenv local 3.8.11
# check with:
python --version

The output should be 3.8.11 Then continue with the following and activate the environment:

$ python -m venv venv --prompt ai8x-training
$ source venv/bin/activate

Then continue with:

(ai8x-training) $ pip3 install -U pip wheel setuptools

Next steps differs depending on the system uses CUDA 11.x, or not. CUDA 11.x on Linux:

(ai8x-training) $ pip3 install -r requirements-cu11.txt

For all other systems, including MacOS and CUDA 10.2 on Linux:

(ai8x-training) $ pip3 install -r requirements.txt

Now for the synthetis repository do the same exact things, just change directory and remember to

(ai8x-training) $ deactivate

before proceeding to generate another environment for the synthetis folder.

The ai8x-synthesis does not require CUDA

Installation Completed

Now the installation of the network tools is done. Remember to deactivate/activate each time you change a folder. Now proceed wit the installation of the SDK.

Embedded Software Development Kit

The Embedded SDK for MAX78000 and MAX78002 is used to compile, flash, and debug the output of the ai8x-synthesis (“izer”) tool. It also enables general software development for the microcontroller cores of the MAX78000 and MAX78002. It consists of the following components:

Peripheral Drivers
Board Support Packages (BSPs)
Libraries
Examples
Toolchain
- Arm GCC
- RISC-V GCC
- Make
- OpenOCD

There are two ways to install the SDK.

Method 1: SDK installer

The Microcontroller SDK for MAX78000/MAX7802 (“MaximSDK”) is available via the installer links below. These installers require a GUI on your system.

Download the MaximSDK installer for your operating system from one of the links below.
Run the installer executable. Note: On Linux, this may require making the file executable with the following command:
```
$ chmod +x MaximMicrosSDK_linux.run
```
Follow the instructions in the installer to the component selection.
Select the components to install. At minimum, the following components must be selected. This will enable command-line development.
- GNU RISC-V Embedded GCC
- GNU Tools for ARM Embedded Processors
- Open On-Chip Debugger
- MSYS2 (only on Windows)
- Microcontrollers
  - MAX78000 Resources
  - MAX78002 Resources
- Product Libs
  - CMSIS Core Libraries
  - Miscellaneous Drivers
  - Peripheral Drivers
(Optional) Select the “Eclipse” and/or “Visual Studio Code Support” components to add support for those IDEs.
Continue through the instructions to complete the installation of the MaximSDK.
(macOS only) Install OpenOCD dependencies using Homebrew
```
$ brew install libusb-compat libftdi hidapi libusb
```

(Linux and macOS only) Add the location of the toolchain binaries to the system PATH.

On Linux and macOS, copy the following contents into ~/.profile... On macOS, also copy the following contents into ~/.zprofile... ...and change MAXIM_PATH to the installation location of the MaximSDK.

# MaximSDK location
MAXIM_PATH=$HOME/MaximSDK  # Change me!
export MAXIM_PATH

# Arm GCC
ARMGCC_DIR=$MAXIM_PATH/Tools/GNUTools/10.3
echo $PATH | grep -q -s "$ARMGCC_DIR/bin"
if [ $? -eq 1 ] ; then
    PATH=$PATH:"$ARMGCC_DIR/bin"
    export PATH
    export ARMGCC_DIR
fi

# RISC-V GCC
RISCVGCC_DIR=$MAXIM_PATH/Tools/xPack/riscv-none-embed-gcc/10.2.0-1.2
echo $PATH | grep -q -s "$RISCVGCC_DIR/bin"
if [ $? -eq 1 ] ; then
    PATH=$PATH:"$RISCVGCC_DIR/bin"
    export PATH
    export RISCVGCC_DIR
fi

# OpenOCD
OPENOCD_DIR=$MAXIM_PATH/Tools/OpenOCD
echo $PATH | grep -q -s "$OPENOCD_DIR"
if [ $? -eq 1 ] ; then
    PATH=$PATH:$OPENOCD_DIR
    export PATH
    export OPENOCD_DIR
fi

On Windows, this step is not necessary. However, “MaximSDK/Tools/MSYS2/msys.bat” file must be used to launch the MSYS2 terminal for command-line development.

Once the tools above have been installed, continue with Final Check.

Method 2: Manual Installation

The MAX78000/MAX78002 SDK is available as a git submodule of ai8x-synthesis. It is checked out automatically to a version compatible with the project into the folder ai8x-synthesis/sdk. This submodule contains all of the SDK's components except the Arm GCC, RISC-V GCC, and Make. These must be downloaded and installed manually.

Download and install the Arm Embedded GNU Toolchain from https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads.
- Recommended version: 10.3-2021.10 (newer versions may or may not work correctly)
- Recommended installation location: /usr/local/gcc-arm-none-eabi-10.3-2021.10/
Download and install the RISC-V Embedded GNU Toolchain from https://github.com/xpack-dev-tools/riscv-none-embed-gcc-xpack/releases/
- Recommended version: 10.2.0-1.2 (newer versions may or may not work correctly)
- Recommended installation location: /usr/local/riscv-none-embed-gcc/10.2.0-1.2/
Install GNU Make
- (Linux/macOS) “make” is available on most systems by default. If not, it can be installed via the system package manager.
- (Windows) Install MSYS2 first, then install “make” using the MSYS2 package manager:
```
$ pacman -S --needed base filesystem msys2-runtime make
```
Install packages for OpenOCD. OpenOCD binaries are available in the “openocd” sub-folder of the ai8x-synthesis repository. However, some additional dependencies are required on most systems. See openocd/Readme.md for a list of packages to install, then return here to continue.

Add the location of the toolchain binaries to the system path.

On Linux and macOS, copy the following contents into ~/.profile... On macOS, also copy the following contents into ~/.zprofile... ...adjusting for the actual PATH to the compilers and the system’s architecture (TARGET_ARCH):

# Arm GCC
ARMGCC_DIR=/usr/local/gcc-arm-none-eabi-10.3-2021.10  # Change me!
echo $PATH | grep -q -s "$ARMGCC_DIR/bin"
if [ $? -eq 1 ] ; then
    PATH=$PATH:"$ARMGCC_DIR/bin"
    export PATH
    export ARMGCC_DIR
fi

# RISC-V GCC
RISCVGCC_DIR=/usr/local/xpack-riscv-none-embed-gcc-10.2.0-1.2  # Change me!
echo $PATH | grep -q -s "$RISCVGCC_DIR/bin"
if [ $? -eq 1 ] ; then
    PATH=$PATH:"$RISCVGCC_DIR/bin"
    export PATH
    export RISCVGCC_DIR
fi

# OpenOCD
OPENOCD_DIR=$HOME/Documents/Source/ai8x-synthesis/openocd/bin/TARGET_ARCH  # Change me!
echo $PATH | grep -q -s "$OPENOCD_DIR"
if [ $? -eq 1 ] ; then
    PATH=$PATH:$OPENOCD_DIR
    export PATH
    export OPENOCD_DIR
fi

On Windows, add the toolchain paths to the system PATH variable (search for “edit the system environment variables” in the Windows search bar).

Once the tools above have been installed, continue to the Final Check step below.

Final Check

After a successful manual or SDK installation, the following commands will run from on the terminal and display their version numbers:

arm-none-eabi-gcc -v
arm-none-eabi-gdb -v
make -v
openocd -v

gen-demos-max78000.sh and gen-demos-max78002.sh will create code that is compatible with the SDK and copy it into the SDK’s Example directories.

Model Training and Synthesis

Every step required to create a Network Model, train it and synthesize it (transform it in C code) are explained in ai8x-synthesis. There you can find each steps to developed,train and load a custom model or a pretrained model on Max78000.

Build and Flash

To build and flash the code on the Max78000 the programs explained before are needed. Particularly, Arm-Gdb and Openocd. There are many methods to build and flashing programs on Max78000.

Method 1: Terminal

Build

Inside each template project there is a Makefile. The command:

$ make

builds the project following the rules inside the makefile. First thing to do is select the right board type. For example at the 63 line inside the makefile of this project there is

#BOARD=EvKit_V1
BOARD=FTHR_RevA

Just comment or uncomment the right board.

Other possible commands are:

$ make clean
$ make distclean

the first one removes the build folder, the second one removes the build folder and clean the peripheral too.

Flash

To flash the code inside the Max78000 thourgh terminal, there are few steps to do.

Open a remote connection between the host and the board. To do this open a window terminal, go inside the MaximSDK/Tools/OpenOCD, and type:

$ ./openocd -f scripts/interface/cmsis-dap.cfg -f scripts/target/max78000.cfg -s/c/MaximSDK/Tools/OpenOCD/scripts

The output should be something like this:

Open On-Chip Debugger 0.11.0+dev-g4cdaa275b (2022-03-02-20:48)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : CMSIS-DAP: SWD  supported
Info : CMSIS-DAP: Atomic commands supported
Info : CMSIS-DAP: Test domain timer supported
Info : CMSIS-DAP: FW Version = 0256
Info : CMSIS-DAP: Serial# = 0444170137ff8ae200000000000000000000000097969906
Info : CMSIS-DAP: Interface Initialised (SWD)
Info : SWCLK/TCK = 1 SWDIO/TMS = 1 TDI = 0 TDO = 0 nTRST = 0 nRESET = 1
Info : CMSIS-DAP: Interface ready
Info : clock speed 2000 kHz
Info : SWD DPIDR 0x2ba01477
Info : max32xxx.cpu: Cortex-M4 r0p1 processor detected
Info : max32xxx.cpu: target has 6 breakpoints, 4 watchpoints
Info : starting gdb server for max32xxx.cpu on 3333
Info : Listening on port 3333 for gdb connections

Flash the code inside the Max78000 with arm-gdb. To do this go inside the project folder (build the project with make) and type:

$ arm-none-eabi-gdb build/Continual-Learning-on-Max78000.elf

The .elf file change name based on your settings. Once inside the arm-gdb enviornment, is time to flash the code. First establish a connection between the host and the board, and reset the board.

$ target extended-remote :3333
$ monitor reset halt

Then load and verify the code:

$ load
$ compare-sections

Finally, reset again the board timer and let the program start:

$ monitor reset halt
$ c

Method 2: Visual Studio Code Tasks

This method is preferred because it's simplier to use. There are a few step to configure Vscode but once done that, flashing and building become very fast. First thing to do is to set the MAXIM_PATH into the vscode settings. Press cmd+shift+p for MacOS, ctl+shift+p for Linux/Windows and type setting and select Preferences: Open User Settings (JSON) and add:

"MAXIM_PATH":"/Users/gio/MaximSDK", //Change this to the installation location of the MaximSDK from step 1.
"update.mode": "manual" // Disable auto updates of VS Code (Optional but strongly recommended)

Inside .vscode folder there are the config files for the building and flashing of the program on the Max78000 using Visual Studio Code.

Gio200023/Continual-Learning-on-Max78000

Continual Learning on Max78000 Microcontroller

Table of Contents

Repository Structure

Project Description

Tools Installation

Prerequisites

Project Installation

macOS

Linux

Python 3.8 Pyenv

MacOS

Linux

Installation Completed

Embedded Software Development Kit

Method 1: SDK installer

Method 2: Manual Installation

Final Check

Model Training and Synthesis

Build and Flash

Method 1: Terminal

Build

Flash

Method 2: Visual Studio Code Tasks