Khronos OpenVX Tutorial Material

Khronos OpenVX is an open, royalty-free standard API for cross platform acceleration of computer vision applications. OpenVX enables performance and power-optimized computer vision functionality, especially important in embedded and real-time use cases. This course covers both the function-based API and the graph API that enable OpenVX developers to efficiently run computer vision algorithms on heterogeneous computing architectures. A set of example algorithms from computational photography and advanced driver assistance mapped to the graph API will be discussed. Also covered is the relationship between OpenVX and OpenCV, as well as OpenCL. The tutorial includes hands-on practice sessions that get the participants started on solving real computer vision problems using OpenVX.

NOTE: The OpenVX Neural Network acceleration tutorials from Embedded Vision Summit 2017 are now available. Refer to wiki page for details.

This tutorial material is based on OpenVX 1.1 and it can be used on a PC with recent 64-bit OS (Windows, OS X, or Linux). Tutorial exercises and build procedures on various platforms are explained in this document. An additional section explains tutorial exercises with a VirtualBox VM.

Tutorial Exercises Overview
Build on any PC using VirtualBox software and a pre-built VM (recommended)
Build on Mac or Linux PC
Build on Windows PC using Visual Studio

NOTE: The directory ~ in the text below refers to directory containing the openvx_tutorial sub-directory.

Refer to Wiki page for upcomming events and past event archives.

1. Tutorial Exercises Overview

It is best to start doing these exercises after going through the tutorial presentations that discuss OpenVX ecosystem overview and hands-on programming sessions.

All the tutorial exercises are kept in ~/openvx_tutorial/tutorial_exercises. The tutorial_exercises/CMakeLists.txt includes all the exercises as separate projects. All of the exercise folders contain only one .cpp file with main() as the entry point. All the include files are kept in the tutorial_exercises/include directory.

The tutorial_exercises sub-directory contains four exercises:

exercise1: framework basics, graph concepts, keypoint tracking
exercise2: user kernels, build a wrapper kernel to OpenCV function
exercise3: intro to tensor object, build a cosine activation user kernel for tensor objects
exercise4: build OpenCL based user kernel for neural network cosine activation and tensor objects

There are additional folders with full solutions:

solution_exerciseN: complete solution of exerciseN. Just for reference.

Each exercise requires you to modify exerciseN/exerciseN.cpp file. Here are few helpful instructions:

Look for TODO keyword in exerciseN/exerciseN.cpp comments for instructions for code snippets that you need to create.
The steps are numbered, do them in that order.
All header files are kept in tutorial_exercises/include. To open a header file, move the cursor to corresponding #include statement in exerciseN/exerciseN.cpp and press F2.
- The "opencv_camera_display.h" is a wrapper that imports media and displays results using OpenCV library.
  - #define DEFAULT_WAITKEY_DELAY 1 is used to specify wait time in milliseconds after each frame processing; to slowdown use larger numbers; or use 0 to wait for a key after each frame.
- The "VX/vx.h" & "VX/vxu.h" files are part of OpenVX header files downloaded from https://www.khronos.org/registry/vx/
To view the definition of any OpenVX API or data type, simply move the cursor to the name and press F2.
The video sequence PETS09-S1-L1-View001.avi will be in ~/openvx_tutorial/tutorial_videos folder.
- All the exercises in this tutorial use this video sequence as input.
- Optionally, you can also specify you own video sequence on command-line as an argument to exercises.
Once you finish all the exercises, try using Release Build to see better performance.

2. Build on any PC using VirtualBox software and a pre-built VM

We have prepared a VirtualBox VM with this new course material. Make sure to setup your computer before starting the tutorial.

Choose a laptop with a recent 64-bit OS.
Download and install a recent VirtualBox from https://www.virtualbox.org/wiki/Downloads.
Download virtual machine "Ubuntu-64-OpenVX.zip" (2 GB) from https://goo.gl/YfcTLh and extract files into a local folder (~6 GB extracted).
- This VM image includes all the necessary tools and packages required to run the tutorial, including the following two OpenVX implementations options:
  - Open-source OpenVX on GitHub from AMD (default)
  - Khronos OpenVX sample implementation from khronos.org/registry/vx
Run VirtualBox and add "Ubuntu-64-OpenVX" virtual machine [Machine -> Add] from the local folder.
- If you cannot install 64-bit VM, even though you have a 64-bit Windows, you need to enable virtualization in the BIOS.
- In the Security section, enable Virtualization Technology and VT-d Feature.
- On Windows 8.1, you also need to turn Hyper-V off (search for Turn Windows features on or off).
Start the "Ubuntu-64-OpenVX" virtual machine ([username: openvx][password: openvx]).
Run "Qt Creator" (click Qt icon on left) and open exercises project.
- Open Project: CMakeLists.txt in /home/openvx/openvx_tutorial/tutorial_exercises
- click "Configure Project" to open CMake Wizard
- click "Run CMake" and "Finish"
Select exercise1 as active sub-project.
- click "Open Build and Run Kit Selector" under the "Build" menu
- select Run "exercise1" under the Build "Default" and press ESCAPE
- expand "exercise1" folder and click "exercise1.cpp"
- you are going to modify this file during the first practice session
Build the project and run.
- click "Run" under the "Build" menu (or use keyboard shortcut Ctrl+R)
- you should see video in a window (you can move the window for better view)
- press ESCAPE or 'q' to exit the app

Please note that the VirtualBox VM might not have the latest version of tutorial exercises. You can copy files from this GitHub project into /home/openvx/openvx_tutorial directory when new updates are available.

3. Build on Mac or Linux PC

In order to build these tutorial exercises, you need the following:

Laptop with a recent 64-bit OS (OS X or Linux)
Download and install OpenVINO or OpenCV
Download and install Qt Creator (optional)
Download and install an OpenVX implementation:
- Open-source OpenVX on GitHub from AMD
  - The source code will be in ~/openvx_tutorial/tutorial_exercises/amdovx-modules directory.
  - CPU only build will be used for this tutorial.
- Khronos OpenVX sample implementation from khronos.org/registry/vx.
  - Follow the instructions in openvx_sample/README to create pre-built OpenVX libraries
- See Khronos OpenVX Resources for available commertial implementations.
Download PETS09-S1-L1-View001.avi and keep it in ~/openvx_tutorial/tutorial_videos folder.

3.1 Build using open-source OpenVX

Create an empty folder ~/openvx_tutorial/build-open-source. The directory structure should be like:

~/openvx_tutorial/
  ├── LICENSE
  ├── README.md
  ├── build-open-source/
  ├── scripts/
  ├── tutorial_exercises/
  │   ├── amdovx-modules/
  │   ├── CMakeLists.txt
  │   ├── exercise1/
  │   ├── exercise2/
  │   ├── exercise3/
  │   ├── exercise4/
  │   ├── include/
  │   ├── solution_exercise1/
  │   ├── solution_exercise2/
  │   ├── solution_exercise3/
  │   ├── solution_exercise4/
  └── tutorial_videos/
      └── PETS09-S1-L1-View001.avi

To prepare for build: % cd ~/openvx_tutorial/build-open-source; cmake ../tutorial_exercises; make
To build and run an example: % cd ~/openvx_tutorial/build-open-source/exercise1; make; ./exercise1
- You should see video in a window. Press ESCAPE or 'q' to exit the app.

3.2 Build using open-source OpenVX in Qt Creator

Open Project ~/openvx_tutorial/tutorial_exercises/CMakeLists.txt.
Click "Configure Project" to open CMake Wizard
CPU only build will be used for this tutorial, unless CMake build flag ENABLE_OPENCL=TRUE is selected.
Click "Run CMake" and "Done"
Compile and run the project by clicking the higher of the green triangles at left bottom, or with CTRL-R.
You should see video in a window. Press ESCAPE or 'q' to exit the app.

3.3 Build using pre-built OpenVX library in Qt Creator

Open Project ~/openvx_tutorial/tutorial_exercises/CMakeLists.txt.
Click "Configure Project" to open CMake Wizard
- Specify arguments: -DOpenVX_LIBS_DIR=<path-to-openvx-libraries> -DOpenVX_LIBS=<list-of-openvx-libraries>
- Click "Run CMake" and "Done"
Compile and run the project by clicking the higher of the green triangles at left bottom, or with CTRL-R.
You should see video in a window. Press ESCAPE or 'q' to exit the app.

4. Build on Windows PC using Visual Studio