It is a 2 step process:
- Flash and update your Coral Dev Mini to the most up to date version
- Transform the YoloV5s model to Edge Quantized version
- Run inference on Coral
Coral board behaves strange when unplugged without shutdown, corrupting boot filesystem. Since you will be doing a lot of operations I recommend a USB to Serial and monitoring Coral output all the time, this way you will connect easily to the board after flash/restarts.
Coral board has only 2G of RAM and 8G of storage. I strongly recommend to download the packages on a USB stick and add a swap file to the board on a separate SD card.
To add swap file to Coral Dev Mini please follow the instructions bellow. f0cal/google-coral#61
I used a 8GB USB stick and another 8GB SD card for Swap. If you get disk full messages I recommend to restart that operation from scratch after adding more space to the device.
To make sure you have the right environment please flash the latest version 2020 of the Google Coral Dev Mini board by following the steps here.
ATTENTION! This will erase completely the board and loose any work you might have. Also this will erase network configuration so you need to follow the initiation procedure in order to connect to Wifi.
Once the board is ready with the new version, upgrade all the package on the board as described here, and beware for system date to be correct.
sudo apt-get update
sudo apt-get dist-upgrade
sudo reboot now
The library opencv-python does not install by default on Coral so you must follow the steps bellow in order to succeed.
install OpenCV (opencv-python package) for Coral as described in this link: https://krakensystems.co/blog/2020/doing-machine-vision-on-google-coral I did the steps in the Quick Solution chapter.
ALTERNATIVE: for OpenCV is to take 12 hours to compile it yourself as described in this issue: google-coral/examples-camera#76
To build pytorch as tutorial (24h in my case) my advice is to use "screen" to keep a long running session open.
so:
sudo apt-get install screen
Once you start a screen use Ctrl-A + D to disconnect, then
screen -r
to reconnect a running session.
Now prepare for a 24 hours journey, start your screen and a separate shell to monitor the board.
Setting MAX_JOBS will enable only 3 parallel CCL to be executed in the installation of python packages. In my tests this is confortable for the 2G memory available.
Credits to https://stackoverflow.com/questions/63047424/how-to-install-pytorch-in-coral-dev-board, I just added the EXPORT command.
export MAX_JOBS=3
# clone pyTorch repo
git clone http://github.com/pytorch/pytorch
cd pytorch
git submodule update --init --recursive
# install prereqs
sudo pip3 install -U setuptools
sudo apt-get install python3-pycoral
Now start building TORCH with
python3 setup.py build
ATTENTION: Monitor the build in a separate shell with TOP to see if your progress stops. In my case around 62% a file named RegisterMeta.cpp will freeze as not enough RAM is abailable. Ctrl+C to interrupt and set this env variable to 1.
export MAX_JOBS=1
Restart the process, will continue where it left
python3 setup.py build
python3 setup.py install
After some time it compiles and reaches batch_norm_kernel.cpp file so please set back MAX_JOBS = 3 to speedup process, or leave it like this as it will take very long time.
Install other requirements
pip3 install numpy
pip3 install pandas # prepare to wait 3 hours
pip3 install tqdm # aprox 10 minutes
Install Torch Vision
sudo pip3 install matplotlib # aproxx 20 min
sudo pip3 install seaborn
sudo pip3 install scipy==1.4.1
sudo pip3 install ninja # this should take 3-45 minutes
Again, please set MAX_JOBS=2 because Ninja will propose a too large value in my opinion for the available RAM and monitor with TOP
Git clone https://github.com/pytorch/vision
cd vision
python3 setup.py install # this should take about 1 hour
After tests it seems only the 96x96 image size fits the EdgeTPU memory on all operations. This however is giving low quality in my tests.
git clone https://github.com/bogdannedelcu/yolov5-export-to-coraldevmini
git clone https://github.com/bogdannedelcu/yolov5/
cd yolov5
git checkout tf-edgetpu
Now copy the model and the images in the yolov5 folder and start detecting:
detect.py --weight yolov5s-int8_edgetpu_416.tflite --img 416 --tfl-int8 --tfl-detect --source bus.jpg --edgetpu
DISCLAIMER: I updated the tf-edgetpu branch of the YoloV5 repository to use only tflite and pycoral packages, as tensorflow is not available on the board.
Use the foollowing Google Collab to transform the PyTorch model to a TFLite quantized.
| Image file | Time on CPU F16 | Time on EdgeTPU F16 | Time on EdgeTPU Quantized 416px | Time on EdgeTPU Quantized 224px |
|---|---|---|---|---|
| bus.jpg - run once | 3.2 | 6.2 | 0.6 | 0.3 |
| bus.jpg - run 100 times | 3.2 | 6.2 | 0.38 | 0.14 |
- time averaged after 1 and respective 100 interpreter.invoke() calls
My conclusion, on a Coral Dev Board Mini EdgeTPU Quantized model compiled and run we get :
- at 224x224 image size a 7FPS
- at 416x416 image size a 2.6FPS (less operations run on EdgeTPU than at 224x224)
I included in the repository some images of the output on 224 and 416 pixels and also the conversion report from edgetpu_compiller for both versions. There are some layers in the NN which are not converted due to compatibility and size (416 is worse than 224)