Pico-CNN

Pico-CNN is a (almost) library free and lightweight neural network inference framework for embedded systems (Linux and bare-metal) implemented in C++. Neural networks can be trained with any training framework which supports export of ONNX (open neural network exchange, onnx.ai) and afterwards deployed using Pico-CNN's ONNX import.

Setup and Use

Please read the whole document carefully!

Ubuntu

sudo apt install libjpeg-dev

If you want to use cppunit:

sudo apt install libcppunit-dev

Scientific Linux

sudo yum install libjpeg-devel

If you want to use cppunit:

git clone --branch cppunit-1.14.0 git://anongit.freedesktop.org/git/libreoffice/cppunit/
cd cppunit
./autogen.sh
./configure
make
sudo make install
sudo ln -s /usr/local/include/cppunit /usr/local/include/CppUnit
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

macOS

brew install jpeg

If you want to use cppunit:

brew install cppunit

C++-Standard

Depending on the (system-) compiler you are using you might have to add a specific C++-Standard to the CFLAGS variable in the generated Makefile. If you are using an older version of g++ it should suffice to chose -std=c++11 or -std=gnu++11 as the C++-Standard.

All Operating Systems

Install Python in version 3.6.5 for example with pyenv (probably also works with other versions of Python 3.6). Then install the required Python packages with the requirements.txt file located in pico-cnn/onnx_import:

cd onnx_import
pip install -r requirements.txt

Of course you can always install the requirements into a virtual environment like this:

cd onnx_import
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

In the future you always have to activate the environment:

cd onnx_import
source venv/bin/activate

Testing with `cppunit`

cd test
make tests
./tests

Utilities

If you want to use scripts from the pico-cnn/util folder you should create a virtual environment for it and install the respective python packages:

cd utils
python -m venv venv
source venv/bin/activate
pip install -r util_requirements.txt

In the future you always have to activate the environment:

cd utils
source venv/bin/activate

Tested Neural Networks

Pico-CNN was tested with the following neural networks:

LeNet
MNIST Multi-Layer-Perceptron (MLP)
MNIST Perceptron
AlexNet
VGG-16
VGG-19
MobileNet-V2
Inception-V3
Inception-Resnet-V2
TC-ResNet-8

All networks

Dummy Input

For every imported onnx model a dummy_input.cpp will be generated, which uses random numbers as input and calls the network, so that no network specific input data has to be downloaded to run inferences.

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input PATH_TO_ONNX/model.onnx
cd generated_code/model
make dummy_input
./dummy_input network.weights.bin NUM_RUNS GENERATE_ONCE

GENERATE_ONCE = 0 will lead to new random input for each inference run.
GENERATE_ONCE = 1 will lead to the same random input for each inference run.

You can monitor overall progress of the current RUN by adding -DDEBUG in the generated Makefile.

Reference Input

There will also be generated a reference_input.cpp which can be used to validate the imported network against the reference input/output that is provided for onnx models from the official onnx model-zoo. The data has to be preprocessed with the following script (it is assumed that the pico-cnn/utils specific virtual environment is activated):

python util/parse_onnx_reference_files.py --input PATH_TO_REFERENCE_DATA

The script will generate an input_X.data and output_X.data file which can then be used like this:

cd onnx_import/generated_code/model
make reference_input
./reference_input network.weights.bin PATH_TO_SAMPLE_DATA/input_X.data PATH_TO_SAMPLE_DATA/output_X.data

If the model was acquired in some other way (self-trained or converted) you can create sample data with the following script (it is assumed that the pico-cnn/utils specific virtual environment is activated):

cd util
python3.6 generate_reference_data.py --model model.onnx --file PATH_TO_INPUT_DATA --shape 1 NUM_CHANNELS HEIGHT WIDTH

If --file is not given the script will use random values instead. Supported file types are audio/x-wav, image/jpeg and image/x-portable-greymap.

MNIST Dataset

LeNet-5

LeNet-5 implementation as proposed by Yann LeCun et. al [LeCun1998] ONNX model at: ./data/lenet/lenet.onnx

Note: MNIST dataset required to run the LeNet specific code.

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input ../data/lenet/lenet.onnx

Copy examples/lenet.cpp from examples folder to onnx_import/generated_code/lenet.

cd generated_code/lenet
make lenet
./lenet network.weights.bin PATH_TO_MNIST

MNIST Multi-Layer-Perceptron (MLP)

ONNX model at: ./data/mnist_mlp/mnist_mlp.onnx

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input ../data/mnist_mlp/mnist_mlp.onnx

Copy examples/mnist_mlp.cpp to onnx_import/generated_code/mnist_mlp.

cd generated_code/mnist_mlp
make mnist_mlp
./mnist_mlp network.weights.bin PATH_TO_MNIST

MNIST Perceptron

Single fully-connected layer. ONNX model at: ./data/mnist_simple_perceptron/mnist_simple_perceptron.onnx

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input ../data/mnist_simple_perceptron/mnist_simple_perceptron.onnx

Copy examples/mnist_simple_perceptron.cpp to onnx_import/generated_code/mnist_simple_perceptron.

cd generated_code/mnist_simple_perceptron
make mnist_simple_perceptron
./mnist_simple_perceptron network.weights.bin PATH_TO_MNIST

ImageNet Dataset

AlexNet

AlexNet [Krizhevsky2017] ONNX model retrieved from: https://github.com/onnx/models/tree/master/vision/classification/alexnet

Note: ImageNet dataset required to run the AlexNet specific code.

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input PATH_TO_ONNX/alexnet.onnx

Copy examples/alexnet.cpp to onnx_import/generated_code/alexnet.

cd generated_code/alexnet

Add -ljpeg to LDFLAGS in Makefile

make alexnet
./alexnet network.weights.bin PATH_TO_IMAGE_MEANS PATH_TO_LABELS PATH_TO_IMAGE

VGG-16

VGG-16 [Simonyan2014] ONNX model retrieved from: https://github.com/onnx/models/tree/master/vision/classification/vgg

Note: ImageNet dataset required to run the VGG-16 specific code.

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input PATH_TO_ONNX/vgg16.onnx

Copy examples/vgg.cpp to onnx_import/generated_code/vgg16.cpp.

cd generated_code/vgg16

Add -ljpeg to LDFLAGS in Makefile

make vgg16
./vgg16 network.weights.bin PATH_TO_IMAGE_MEANS PATH_TO_LABELS PATH_TO_IMAGE

VGG-19

VGG-19 [Simonyan2014] ONNX model retrieved from: https://github.com/onnx/models/tree/master/vision/classification/vgg

Note: ImageNet dataset required to run the VGG-19 specific code.

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input PATH_TO_ONNX/vgg19.onnx

Copy examples/vgg.cpp to onnx_import/generated_code/vgg19.cpp.

cd generated_code/vgg19

Add -ljpeg to LDFLAGS in Makefile

make vgg19
./vgg19 network.weights.bin PATH_TO_IMAGE_MEANS PATH_TO_LABELS PATH_TO_IMAGE

MobileNet-V2

MobileNet-V2 [Sandler2019] ONNX model retrieved from: https://github.com/onnx/models/tree/master/vision/classification/mobilenet

See Reference Input section for details on input and output data generation.

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input PATH_TO_ONNX/mobilenetv2-1.0.onnx
cd generated_code/mobilenetv2-1
make reference_input
./reference_input network.weights.bin input.data output.data

Inception-V3

Inception-V3 [Szegedy2014] ONNX model retrieved from: http://download.tensorflow.org/models/inception_v3_2016_08_28.tar.gz and converted using TensorFlow slim.

See Reference Input section for details on input and output data generation.

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input PATH_TO_ONNX/inceptionv3.onnx
cd generated_code/inceptionv3
make reference_input
./reference_input network.weights.bin input.data output.data

Inception-ResNet-V2

Inception-ResNet-V2 [Szegedy2016] ONNX model retrieved from: http://download.tensorflow.org/models/inception_resnet_v2_2016_08_30.tar.gz and converted using TensorFlow slim.

See Reference Input section for details on input and output data generation.

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input PATH_TO_ONNX/inception_resnet_v2.onnx
cd generated_code/inception_resnet_v2
make reference_input
./reference_input network.weights.bin input.data output.data

Speech Recognition

TC-ResNet-8

TC-ResNet-8 [Choi2019] trained on the Google Speech Commands Dataset

cd onnx_import
python3.6 onnx_to_pico_cnn.py --input PATH_TO_ONNX/tc-res8-update.onnx
cd generated_code/tc-res8-update
make reference_input
./reference_input network.weights.bin input.data output.data

Contributors

Alexander Jung (Maintainer) @alexjung
Konstantin Lübeck (Maintainer) @k0nze
Nils Weinhardt (Developer) @sea-buckthorn

Citing Pico-CNN

Please cite Pico-CNN in your publications if it helps your research:

@inproceedings{luebeck2019picocnn,
    author = {Lübeck, Konstantin and Bringmann, Oliver},
    title = {A Heterogeneous and Reconfigurable Embedded Architecture for Energy-Efficient Execution of Convolutional Neural Networks},
    booktitle = {Architecture of Computing Systems (ARCS 2019)},
    year = {2019},
    month = {May},
    pages = {267--280},
    address = {Copenhagen, Denmark},
    isbn = {978-3-030-18656-2}
}

References

[LeCun1998] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998. ↩

[Krizhevsky2017] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, May 2017. ↩

[Simonyan2014] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv:1409.1556 [cs], Sep. 2014. ↩

[Sandler2019] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” arXiv:1801.04381 [cs], Mar. 2019. ↩

[Szegedy2014] C. Szegedy et al., “Going Deeper with Convolutions,” arXiv:1409.4842 [cs], Sep. 2014. ↩

[Szegedy2016] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi, “Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning,” arXiv:1602.07261 [cs], Aug. 2016. ↩

[Choi2019] S. Choi et al., “Temporal Convolution for Real-time Keyword Spotting on Mobile Devices,” arXiv:1904.03814 [cs, eess], Nov. 2019. ↩

ekut-es/pico-cnn

Pico-CNN

Setup and Use

Ubuntu

Scientific Linux

macOS

C++-Standard

All Operating Systems

Testing with cppunit

Utilities

Tested Neural Networks

All networks

Dummy Input

Reference Input

MNIST Dataset

LeNet-5

MNIST Multi-Layer-Perceptron (MLP)

MNIST Perceptron

ImageNet Dataset

AlexNet

VGG-16

VGG-19

MobileNet-V2

Inception-V3

Inception-ResNet-V2

Speech Recognition

TC-ResNet-8

Contributors

Citing Pico-CNN

References

Testing with `cppunit`