/modified_refinedet

Modified RefineDet

Primary LanguagePython

Development of Fast Refinement Detectors on AI Edge Platforms

Under review

Abstract

Refinement detector (RefineDet) is a state-of-the-art model in object detection that has been developed and refined based on high-end GPU systems. In this study, we discovered that the speed of models developed in high-end GPU systems is inconsistent with that in embedded systems. In other words, the fastest model that operates on high-end GPU systems may not be the fastest model on embedded boards. To determine the reason for this phenomenon, we performed several experiments on RefineDet using various backbone architectures on three different platforms: NVIDIA Titan XP GPU system, Drive PX2 board, and Jetson Xavier board. Finally, we achieved real-time performances (approximately 20 fps) based on the experiments on AI edge platforms such as NVIDIA Drive PX2 and Jetson Xavier boards. We believe that our current study would serve as a good reference for developers who wish to apply object detection algorithms to AI edge computing hardwares. The complete code and models are publicly available on the web (link).

1. Installation

1.1 Requirement

  • Ubuntu 16.04
  • Caffe
  • Python2
  • CUDA 9.0

1.2 ReifineDet

1.3 Rebuild for Modified RefineDet

  • Download requirements from this repository
    git clone https://github.com/mkchoi-0323/modified_refinedet.git

  • Rebuild caffe for original RefineDet with new caffe layers (depthwise convolution and axpy laer)
    cd RefineDet
    make clean

    1. copy every .cpp and .cu files in addtional_layers
    2. paste .cpp and .cu files to original RefineDet/src/caffe/layers/
    3. copy every .hpp files in additional_layers
    4. paste .hpp files to original RefineDet/include/caffe/layers/

  • Rebuild by caffe compile command
    make all -j4

2. Training

  • Download desired python code to generate head and backbone

    • For example, you should prepare MobileNetV1_COCO_320.py and model_libs_MobileNetV1.py to train new model of RefineDet with MobileNetV1 blocks

  • Make model path and copy python trainin interface
    mkdir RefineDet/models/MobileNet/coco/refinedet_mobilenetv1_320x320
    cp modified_refinedet/heads/MobileNetV1_COCO_320.py RefineDet/models/MobileNet/coco/refinedet_mobilenetv1_320x320/

  • Download pre-trained backbone (link)

  • Copy pre-trained backbone to model path
    cp modified_refinedet/pretrained_backbones/mobielnet.caffemodel RefineDet/models/MobileNet/

  • Copy model interface to original RefineDet
    mv RefineDet/python/caffe/model_libs.py RefineDet/python/caffe/model_libs.py.backup
    cp modified_refinedet/heads/model_libs_MobileNetV1.py RefineDet/python/caffe/model_libs.py

  • Run training interface
    cd RefineDet
    python models/MobileNet/coco/refinedet_mobilenetv1_320x320/MobileNetV1_COCO_320.py

3. Testing

  • Copy dataset loader to original RefineDet interface
    mv RefineDet/test/lib/dataset/coco.py RefineDet/test/lib/dataset/coco.py.backup
    mv RefineDet/test/lib/dataset/factory.py RefineDet/test/lib/dataset/factory.py.backup
    mv modified_refinedet/test/lib/dataset/coco.py RefineDet/test/lib/dataset/
    mv modified_refinedet/test/lib/dataset/factory.py RefineDet/test/lib/dataset/

  • Copy testing interface
    mv modified_refinedet/test/lib/dataset/coco.py RefineDet/test/lib/dataset/
    mv modified_refinedet/refinedet_test_MobileNet.py

  • Download pre-trained models (link)

  • Set model and deploy path to proper location

3.1 val2017

  • Set model and deploy path to proper location
    vim modified_refinedet/test/refinedet_test_MobileNet.py

  • Run test
    cd RefineDet
    python modified_refinedet/test/refinedet_test_MobileNet.py

3.2 test-dev2017

  • Set model and deploy path to proper location
    vim modified_refinedet/test/refinedet_test-dev.py

  • Run test
    cd RefineDet
    python modified_refinedet/test/refinedet_test-dev.py