tzm1003306213/kneron-mmdetection

Kneron AI Training/Deployment Platform (mmDetection-based)

Introduction

kneron-mmdetection is a platform built upon the well-known mmdetection for detection and instance segmentation. We encourage you to start with YOLOX: Step-By-Step to build basic knowledge of Kneron-Edition mmdetection, and read mmdetection docs for detailed mmdetection usage.

In this repository, we provide an end-to-end training/deployment flow to realize on Kneron's AI accelerators:

1. Training/Evalulation:
   • Modified model configuration file and verified for Kneron hardware platform
   • Please see Overview of Benchmark and Model Zoo for Kneron-Verified model list
2. Converting to ONNX:
   • pytorch2onnx_kneron.py (beta)
   • Export optimized and Kneron-toolchain supported onnx
     • Automatically modify model for arbitrary data normalization preprocess
3. Evaluation
   • test_kneron.py (beta)
   • Evaluate the model with pytorch checkpoint, onnx, and kneron-nef
4. Testing
   • inference_kn (beta)
   • Verify the converted NEF model on Kneron USB accelerator with this API
5. Converting Kneron-NEF: (toolchain feature)
   • Convert the trained pytorch model to Kneron-NEF model, which could be used on Kneron hardware platform.

Quantization-Aware-Training

1. Training

CUDA_VISIBLE_DEVICES=0,1,2,3,4 tools/dist_train.sh configs/qat/yolox_s_8x8_300e_coco_img_norm.py 5 --weight-decay 3e-4 --lr 0.00001 --lr-qat 0.001 --min-lr 1e-8 --bitwidth-range 4 8 16 --act-bw-range 8 16 --opt fusedlamb --qat --state-dict work_dirs/latest.pth

2. Converting to KQAT model:

python tools/train.py configs/qat/yolox_s_8x8_300e_coco_img_norm_onnx.py --bitwidth-range 4 8 16 --act-bw-range 8 16 --qat --onnx output_with_qat_nodes.onnx

3. Converting to ONNX+JSON:

python ../kqat/kqat/onnx_extract.py --input output_with_qat_nodes.onnx --output output.onnx

License

This project is released under the Apache 2.0 license.

Changelog

N/A

Overview of Benchmark and Kneron Model Zoo

Backbone	size	Mem (GB)	box AP	Config	Download
YOLOX-s	640	7.6	40.5	config	model

Installation

• Please refer to YOLOX: Step-By-Step, Step 0. Environment for installation.
• Please refer to Kneron PLUS - Python: Installation for the environment setup for Kneron USB accelerator.

Getting Started

Tutorial - Kneron Edition

• YOLOX: Step-By-Step: A tutorial for users to get started easily. To see detailed documents, please see below.

Documents - Kneron Edition

• [Kneron ONNX Export] (under development)
• [Kneron Inference] (under development)
• Kneron Toolchain Step-By-Step (YOLOv3)
• Kneron Toolchain Manual

Original mmdetection Documents

• Original mmdetection getting started: It is recommended to read the original mmdetection getting started documents for other mmdetection operations.
• Original mmdetection readthedoc: Original mmdetection documents.

Contributing

kneron-mmdetection a platform built upon OpenMMLab-mmdetection

• For issues regarding to the original mmdetection: We appreciate all contributions to improve OpenMMLab-mmdetection. Ongoing projects can be found in out GitHub Projects. Welcome community users to participate in these projects. Please refer to CONTRIBUTING.md for the contributing guideline.

• For issues regarding to this repository kneron-mmdetection: Welcome to leave the comment or submit pull requests here to improve kneron-mmdetection

Related Projects

• kneron-mmsegmentation: Kneron training/deployment platform on OpenMMLab - mmSegmentation semantic segmentation toolbox