/rknn_model_zoo

Primary LanguageCApache License 2.0Apache-2.0

简体中文 | English

RKNN Model Zoo

Description

RKNN Model Zoo is developed based on the RKNPU SDK toolchain and provides deployment examples for current mainstream algorithms. Include the process of exporting the RKNN model and using Python API and CAPI to infer the RKNN model.

  • Support RK3562, RK3566, RK3568, RK3588 platforms. (RV1103, RV1106 platforms support mobilenet, yolov5)
  • RK1808, RK3399PRO, RV1109, RV1126 will be supported in next version. (For yolov5/6/7/8, yolox, ppyoloe demos, they are available in v1.5.0, please switch to v1.5.0 to get them)

Dependency library installation

RKNN Model Zoo relies on RKNN-Toolkit2 for model conversion. The Android compilation tool chain is required when compiling the Android demo, and the Linux compilation tool chain is required when compiling the Linux demo. For the installation of these dependencies, please refer to the Quick Start documentation at https://github.com/airockchip/rknn-toolkit2/tree/master/doc.

  • Please note that the Android compilation tool chain recommends using version r18 or r19. Using other versions may encounter the problem of Cdemo compilation failure.

Model support

In addition to exporting the model from the corresponding respository, the models file are available on https://console.zbox.filez.com/l/8ufwtG (key: rknn).

Demo
(Clip to Description)		 Algorithm Category Dtype support Pretrain model
(Clip to download)
mobilenet Classification FP16/INT8 mobilenetv2-12.onnx
resnet Classification FP16/INT8 resnet50-v2-7.onnx
yolov5 Object detection FP16/INT8 yolov5n.onnx
yolov5s_relu.onnx
yolov5s.onnx
yolov5m.onnx
yolov6 Object detection FP16/INT8 yolov6n.onnx
yolov6s.onnx
yolov6m.onnx
yolov7 Object detection FP16/INT8 yolov7-tiny.onnx
yolov7.onnx
yolov8 Object detection FP16/INT8 yolov8n.onnx
yolov8s.onnx
yolov8m.onnx
yolox Object detection FP16/INT8 yolox_s.onnx
yolox_m.onnx
ppyoloe Object detection FP16/INT8 ppyoloe_s.onnx
ppyoloe_m.onnx
deeplabv3 Image segmentation FP16/INT8 deeplab-v3-plus-mobilenet-v2.pb
yolov5-seg Image segmentation FP16/INT8 yolov5n-seg.onnx
yolov5s-seg.onnx
yolov5m-seg.onnx
yolov8-seg Image segmentation FP16/INT8 yolov8n-seg.onnx
yolov8s-seg.onnx
yolov8m-seg.onnx
ppseg Image segmentation FP16 pp_liteseg_cityscapes.onnx
RetinaFace Face key points INT8 RetinaFace_mobile320.onnx
RetinaFace_resnet50_320.onnx
LPRNet Car Plate Recognition FP16/INT8 lprnet.onnx
PPOCR-Det Text detection FP16/INT8 ppocrv4_det.onnx
PPOCR-Rec Text recognition FP16 ppocrv4_rec.onnx
lite_transformer Neural Machine Translation FP16 lite-transformer-encoder-16.onnx
lite-transformer-decoder-16.onnx

Model performance benchmark(FPS)

demo model_name inputs_shape dtype RK3566
RK3568		 RK3562 RK3588
@single_core
mobilenet mobilenetv2-12 [1, 3, 224, 224] INT8 197.4 266.8 433.0
resnet resnet50-v2-7 [1, 3, 224, 224] INT8 40.6 54.5 108.6
yolov5 yolov5s_relu [1, 3, 640, 640] INT8 26.7 31.6 63.3
yolov5n [1, 3, 640, 640] INT8 41.6 43.8 68.1
yolov5s [1, 3, 640, 640] INT8 19.9 22.7 42.5
yolov5m [1, 3, 640, 640] INT8 8.7 10.6 19.3
yolov6 yolov6n [1, 3, 640, 640] INT8 50.2 51.5 93.8
yolov6s [1, 3, 640, 640] INT8 15.2 16.8 34.1
yolov6m [1, 3, 640, 640] INT8 7.5 8.0 17.6
yolov7 yolov7-tiny [1, 3, 640, 640] INT8 29.9 34.9 69.7
yolov7 [1, 3, 640, 640] INT8 4.7 5.5 10.9
yolov8 yolov8n [1, 3, 640, 640] INT8 35.7 38.5 59.6
yolov8s [1, 3, 640, 640] INT8 15.4 17.1 32.8
yolov8m [1, 3, 640, 640] INT8 6.6 7.5 14.8
yolox yolox_s [1, 3, 640, 640] INT8 15.5 17.7 32.9
yolox_m [1, 3, 640, 640] INT8 6.7 8.1 14.8
ppyoloe ppyoloe_s [1, 3, 640, 640] INT8 17.5 19.7 32.9
ppyoloe_m [1, 3, 640, 640] INT8 7.9 8.3 16.2
deeplabv3 deeplab-v3-plus-mobilenet-v2 [1, 513, 513, 1] INT8 10.7 20.7 34.4
yolov5_seg yolov5n-seg [1, 3, 640, 640] INT8 33.9 36.3 58.0
yolov5s-seg [1, 3, 640, 640] INT8 15.3 17.2 32.6
yolov5m-seg [1, 3, 640, 640] INT8 6.8 8.1 15.2
yolov8_seg yolov8n-seg [1, 3, 640, 640] INT8 29.1 30.7 49.1
yolov8s-seg [1, 3, 640, 640] INT8 11.8 11.3 25.4
yolov8m-seg [1, 3, 640, 640] INT8 5.2 6.1 11.6
ppseg pp_liteseg_cityscapes [1, 3, 512, 512] FP16 2.6 4.6 13.0
RetinaFace RetinaFace_mobile320 [1, 3, 320, 320] INT8 142.5 279.5 234.7
RetinaFace_resnet50_320 [1, 3, 320, 320] INT8 18.5 26.0 48.8
LPRNet lprnet [1, 3, 24, 94] INT8 58.2 119.7 204.4
PPOCR-Det ppocrv4_det [1, 3, 480, 480] INT8 24.4 27.5 43.0
PPOCR-Rec ppocrv4_rec [1, 3, 48, 320] FP16 20.0 45.1 35.7
lite_transformer lite-transformer-encoder-16 embedding-256, token-16 FP16 130.8 656.7 261.5
lite-transformer-decoder-16 embedding-256, token-16 FP16 114.3 151.3 164.0
  • This performance data are collected based on the maximum NPU frequency of each platform.
  • This performance data calculate the time-consuming of model inference. Does not include the time-consuming of pre-processing and post-processing.

Compile Demo

For Linux develop board:

./build-linux.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]
    -t : target (rk356x/rk3588/rv1106)
    -a : arch (aarch64/armhf)
    -d : demo name
    -b : build_type(Debug/Release)
    -m : enable address sanitizer, build_type need set to Debug

# Here is an example for compiling yolov5 demo for 64-bit Linux RK3566.
./build-linux.sh -t rk356x -a aarch64 -d yolov5

For Android develop board:

# For Android develop boards, it's require to set path for Android NDK compilation tool path according to the user environment
export ANDROID_NDK_PATH=~/opts/ndk/android-ndk-r18b
./build-android.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]
    -t : target (rk356x/rk3588)
    -a : arch (arm64-v8a/armeabi-v7a)
    -d : demo name
    -b : build_type (Debug/Release)
    -m : enable address sanitizer, build_type need set to Debug

# Here is an example for compiling yolov5 demo for 64-bit Android RK3566.
./build-android.sh -t rk356x -a arm64-v8a -d yolov5

Release Notes

Version Description
1.6.0 New demo release, including object detection, image segmentation, OCR, car plate detection&recognition etc.
Full support for RK3566, RK3568, RK3588, RK3562 platforms.
Limited support for RV1103, RV1106 platforms.
1.5.0 Yolo detection demo release.

Environment dependencies

All demos in RKNN Model Zoo are verified based on the latest RKNPU SDK. If using a lower version for verification, the inference performance and inference results may be wrong.

Version RKNPU2 SDK RKNPU1 SDK
1.6.0 >=1.6.0 - Coming soon
1.5.0 >=1.5.0 >=1.7.3

RKNPU Resource

License

Apache License 2.0