/CHaiDNN

HLS based Deep Neural Network Accelerator Library for Xilinx Ultrascale+ MPSoCs

Primary LanguageC++OtherNOASSERTION

CHaiDNN

The CHaiDNN is Xilinx Deep Neural Network library for acceleration of deep neural networks on Xilinx Ultrascale MPSoCs. It is designed for maximum compute efficiency at 16-bit integer data type.

The design goal of CHaiDNN is to achieve best accuracy with maximum performance. The inference on CHaiDNN works in fixed point domain for better performance. All the feature maps and trained parameters are converted from single precision to fixed point based on the precison parameters specified by the user. The precision parameters can vary a lot depending upon the network, datasets, or even across layers in the same network. Accuracy of a network depends on the precision parameters used to represent the feature maps and trained parameters. Well-crafted precision parameters are expected to give accuracy similar to accuracy obtained from a single precision model.

How to Download the Repository

To get a local copy of the CHaiDNN repository, clone this repository to the local system with the following command:

git clone https://github.com/Xilinx/CHaiDNN.git CHaiDNN

Where CHaiDNN is the name of the directory where the repository will be stored on the local system. This command needs to be executed only once to retrieve the latest version of the CHaiDNN library.

Hardware and Software Requirements

The CHaiDNN library is designed to work with Zynq Ultrascale+ MPSoCs. The library has been verified on zcu102 board. Xilinx SDSoC 2017.4 Development Environment is required to work with the library.

Run Inference Using Pre-built binaries

Follow the below steps to run inference on example networks.

  1. Download the example networks that need to to be benchmarked and place the unzipped contents at SD_Card/models directory.

  2. Verify the contents of SD_Card with the below checklist.

    • _sds folder
    • models folder
    • lib folder
    • opencv_arm64 folder
    • protobuf_arm64 folder
    • BOOT.BIN file
    • image.ub file
    • .elf file(s)

  3. Copy the contents of SD_Card folder into a SD-Card.

  4. Insert the SD-Card and power on the board.

Note: A serial port emulator (Teraterm/Minicom) is required to interface the user commands to the board

  1. Attach a USB-UART cable from the ZCU102 board to the host PC. Set the UART serial port to

    Baud rate: 115200
Data: 8 bit
Parity: none
Stop: 1 bit
Flow control: none

  2. After boot, go to /mnt directory.

    cd /mnt

  3. Export library paths.

    export LD_LIBRARY_PATH=lib/:opencv_arm64/lib/:protobuf_arm64/lib

  4. Execute the commands to run inference for the networks

  • AlexNet

    ./alexnet.elf

  • GoogleNet

     ./googlenet.elf

  • VGG16

     ./vgg.elf

  • AlexNetFCN

     ./alexnetfcn.elf

  • SSD

     ./ssd.elf

  • Advanced ResNet50

     ./adv_resnet50.elf
  1. Sync after execution 
    cd /
sync
umount /mnt
  2. Output will be written into text file inside respective output folders. 
    Ex : models/<network>/out

Note: Failing to run sync might corrupt the file system and cause crash on subsequent runs.

Note: For running inference on a new network, please follow the instructions in Run new Network using CHaiDNN.

Build from Source

CHaiDNN can be built using Makefiles OR using SDx IDE. The below steps describe how to build CHaiDNN using Makefiles. For steps to build using SDx IDE, check the instructions in Build using SDx IDE.

1. Build CHaiDNN Hardware from source

Please follow the steps to build the design for zcu102

  1. Go to the local CHaiDNN folder.

  2. Go to design directory. This contains all the design files required to build the design.

  3. Go to scripts/kernel directory. This contains the Makefile to compile the design.

  4. Source the SDx-2017.4 path. Please check the example command below.

  • For BASH: 
     source <SDx Installation Dir>/installs/lin64/SDx/2017.4/settings64.sh
  • For CSH 
     source <SDx Installation Dir>/installs/lin64/SDx/2017.4/settings64.csh
  1. Run Makefile. This will build the design. 
    make ultraclean
make
  2. After the build is completed, copy the libxlnxdnn.so file and other build files (BOOT.BIN, image.ub and _sds directory) inside scripts/kernel/sd_card to SD_Card directory. 
    make copy
  3. The hardware setup is now ready.

2. Build CHaiDNN Software from source

Follow the steps to compile the software stack.

  1. Copy libxlnxdnn.so to SD_Card/lib directory. This will be present inside design/scripts/kernel/sd_card directory once the HW build is finished.

    Skip this step if already copied.

  2. Source SDx path.

  • CSH 
     source <SDx Installation Dir>/installs/lin64/SDx/2017.4/settings64.csh
  • BASH 
     source <SDx Installation Dir>/installs/lin64/SDx/2017.4/settings64.sh

  1. Go to the software directory. This contains all the files to generate software libraries (.so).

    cd <path to CHaiDNN>/software

  2. Go to scripts directory, open Makefile and update the SDx_BUILD_PATH variable. Check example below.

    SDx_BUILD_PATH = <SDx Installation Dir>/installs/lin64/SDx/2017.4

  3. Now run the following commands.

    cd scripts
make ultraclean
make

  4. Make will copy all executables to SD_Card directory and all .so files to SD_Card/lib directory.

  5. Now all the files are available to run inference on zcu102.

  • Download the example models and copy the unzipped contents to SD_Card/models folder.
  • Copy the contents of SD_Card directory to SD-Card

  1. Insert the SD-Card in the board card slot and switch it ON.

  2. Upon successful boot, execute the following commands.

    cd /mnt
export LD_LIBRARY_PATH=opencv_arm64/lib/:lib/:protobuf_arm64/lib/ 
./<name>.elf

  3. Make sure to sync before removing SD-Card from the board OR power off the board.

    cd /
sync
umount /mnt

CHaiDNN-2017.4 Performance

Network Images/ Second @ 200 MHz (Batch Size =1 on zcu102)
GoogleNet 72.97
AlexNet(with FC) 55.11
AlexNet(without FC) 160.03
VGG-16 9.68
AlexNet-FCN 7.59
SSD 4.64
AdvResNet 21.18
ResNet50 24.59
ResNet34 37.68

IMP : Only Performance Projections for Legacy ResNet34 and ResNet50.

CHaiDNN-2017.4 Resource Utilization

Parameter Available on zcu102 Used % Utilization
DSP 2520 1939 76.94
BRAM 912 828.5 90.84
LUT 274080 135033 49.27
FF 548160 176146 32.13

Other Information

For CHaiDNN API Document click CHaiDNN API document

For Running Inference on other Networks Running Inference for new networks.

For information on CHaiDNN quantization parameters check Quantization Parameters User Guide

For insturctions on creating the SDx GUI Project for CHaiDNN click Build using SDx IDE

For SDSoC User Guide click SDSoC User Guides

Important Notes

  • none

Support

For questions and to get help on this project or your own projects, visit the SDSoC Forums.

Licence and Contributing to the Repository

The source for this project is licensed under the Apache License 2.0

To contribute to this project, follow the guidelines in the Repository Contribution README

Acknowledgements

Revision History

Date Readme Version Release Notes Tool Version
Feb, 2018 1.0 Initial Xilinx release SDx-2017.4

Change Log

  1. First Release

Known Issues

  • none