Quantized Deep Neural Network for Defect Detection on Jetson AGX Xavier Using GPU Coder

How to create, train and quantize network, then integrate it into pre/post image processing and generate CUDA C++ code for targeting Jetson AGX Xavier

Deep Learning is really powerful approach to solve difficult problems(e.g. image classification, segmentation and detection). However, performing inference using deep learning is computationally intensive, consuming significant amount of memory. Even networks that are small in size require a considerable amount of memory and hardware to perform these arithmetic operations. These restrictions can inhibit deloyment of deep learning networks to devices that have low computational power and smaller momory resources.

In this case, you can use Deep Learning Toolbox in tandem with the Deep Learning Toolbox Model Quantization Library support package to reduce the memory footprint of a deep neural network by quantizing the weights, biases, and activations of convolution layers to 8-bit scaled integer data types. And then you can use GPU Coder to generate optimized CUDA code for the quantized network.

This example shows how to create, train and quantize a simple convolutional neural network for defect detection, then demonstrate how to generate code for whole algorithms that includes pre/post image processing and convolutional neural network so that you can deploy it into NVIDIA GPUs such as Jetson AGX Xavier, Nano and Drive platforms.

This example demonstrates how to:

Load and explore image data
Define the network architecture and training options
Train the network and classify validation images
Quantize network to reduce memory footprint
Walk through whole algorithm that consist of pre-processing, CNN and post-processing
Generate CUDA C++ code(MEX) for whole algorithm
Deploy algorithms to NVIDIA hardware
Run the Executable on the Target

Prerequisites - MathWorks Products and Support Packages

MATLAB (R2020a or later)
MATLAB Coder^TM
GPU Coder^TM
Parallel Computing Toolbox^TM
Deep Learning Toolbox^TM
Image Processing Toolbox^TM
Computer Vision Toolbox^TM

Prerequisites - Development Host REquirements

NVIDIA® GPU enabled for CUDA with compute capability 3.2 or higher (6.1 or higher is required for quantization)
NVIDIA CUDA toolkit and driver
C/C++ Compiler
CUDA Deep Neura Network library (cuDNN)
Open Source Computer Vision Library v3.1.0
(Optional) NVIDIA TensorRT
The support package GPU Coder Interface for Deep Learning
GPU Coder Support Package for NVIDIA GPUs
The support package Deep Learning Toolbox Model Quantization Library. To install support packages, use the Add-On Explorer.
Environment variables for the compilers and libraries. For information on the supported versions of the compilers and libraries, see Third-party Products. For setting up the environment variables, see Setting Up the Prerequisite Products.

Prerequisites - Target Board REquirements

NVIDIA Jetson AGX Xavier
Ethernet crossover cable to connect the target board and host PC (if the target board cannot be connected to a local network)
C/C++ Compiler
CUDA Deep Neura Network library (cuDNN)
Open Source Computer Vision Library v3.1.0 or higher for reading and displaying images/video
Environment variables on the target for the compilers and libraries. For information on the supported versions of the compilers and libraries and their setup, see installing and setting up prerequisites for NVIDIA boards

Running the Example

Open and run the live script