Genral Questions on NVDLA

[minjae-park]

We have the following inquiries about NVDLA. Would appreciate feedback.

1. How do we access the full training infrastructure when we build our own ConvNet model ?

2. Can NVDLA parser (compilation tool) can read Caffe2, TensorFlow and pyTorch frameworks?

3. It seems that running parallel multiple independent layers is not feasible for “Headless Implementation”. Is “Fused (Pipelined) operation” possible in this environment?

4. If there is a companion macrocontroller, does KMD run on this companion microcontroller (and UMD runs on main system CPU)?

5. If the companion microcontroller runs on RTOS (say, Nucleus), do we need to reprogram the entire Kernel Mode Driver that is currently accessing Linux Kernel?

6. Can NVDLA’s 2D convolution support “1x1 convolution” (for ResNet implementation)?

7. If we need a softmax activations in the output layer, is there a way to implement it in DLA (using application program)?
   Are there any KMD (or UMD) APIs available for this implementation?

8. In the following paragraph in http://nvdla.org, “multiple DLA devices” mean multiple DLA IP cores? or multiple independent layers in one DLA core?

   “Runtime driver supports submitting inference jobs to multiple DLA devices”

9. Application program should be written in C/C++?, not Java?

10. How does the application program submit a inference job to KMD?

11. Does UMD also include Linux Kernel Driver?

12. What are the portability layers supposed to do in UMD and KMD, respectively? Licensee is supposed to program these?

13. Is KMD stack mainly composed of the firmware (scheduler) and Linux Kernel Driver?

14. Does KMD stack have system calls for power management in the Liux Kernel?

15. Suppose that runtime environment is going to be running on Android (instead of Linux), what do we need to do in this case?

• 1. Modify the Linux kernel driver in KMD into Android Kernel driver (such as IPC, power management, etc.)?
• 2. If necessary, also modify the Linux kernel driver in UMD into Android kernel driver?
• 3. If necessary, move some system calls/functionalities from KMD to UMD (due to discrepancy between Linux kernel and android kernel)?
• 4. Does UMD need to be connected to Android HAL?
• 5. Application program doesn’t need to be passed to Java framework in android?
• 6. What else do we need to modify?

[jwise]

Oop, sorry for the delay on this -- I missed this one. I've asked our software folks to have a look.

[jwise]

We have a release coming up in the next few days that will change some of the answers to some of these questions; we'll plan to get you up to date answers then! Thanks for your patience.

[prasshantg]

Question: How do we access the full training infrastructure when we build our own ConvNet model ?
Answer: We have released nvdla_compiler binary which can be used to convert the trained model to DLA loadable for runtime execution. For training, you still need to use CUDA Dev kit.

Question: Can NVDLA parser (compilation tool) can read Caffe2, TensorFlow and pyTorch frameworks?
Answer: Currently it supports only Caffe

Question: It seems that running parallel multiple independent layers is not feasible for “Headless Implementation”. Is “Fused (Pipelined) operation” possible in this environment?
Answer: It is possible to run multiple independent layers if those are submitted in single task. Fused operation is possible in this environment.

Question: If there is a companion macrocontroller, does KMD run on this companion microcontroller (and UMD runs on main system CPU)?
Answer: Firmware part from KMD runs on companion microcontroller and UMD + KMD runs on main system CPU. In this case, KMD works as communication channel between UMD and firmware running on companion microcontroller

Question: If the companion microcontroller runs on RTOS (say, Nucleus), do we need to reprogram the entire Kernel Mode Driver that is currently accessing Linux Kernel?
Answer: In this case, firmware part from KMD can be used as is and just need to program portability layer for RTOS.

Question: Can NVDLA’s 2D convolution support “1x1 convolution” (for ResNet implementation)?
Answer: Yes

Question: If we need a softmax activations in the output layer, is there a way to implement it in DLA (using application program)?
Are there any KMD (or UMD) APIs available for this implementation?
Answer: Yes, it is implemented in runtime test application. There are options to enable it.

Question: In the following paragraph in http://nvdla.org, “multiple DLA devices” mean multiple DLA IP cores? or multiple independent layers in one DLA core?
“Runtime driver supports submitting inference jobs to multiple DLA devices”
Answer: It means multiple DLA IP cores

Question: Application program should be written in C/C++?, not Java?
Answer: You can add JNI and write application program in Java on top of it.

Question: How does the application program submit a inference job to KMD?
Answer: Through IOCTL. Refer to https://github.com/nvdla/sw/blob/master/umd/port/linux/nvdla.c

Question: Does UMD also include Linux Kernel Driver?
Answer: UMD is more like a userspace library providing interface between user application program and KMD.

Question: What are the portability layers supposed to do in UMD and KMD, respectively? Licensee is supposed to program these?
Answer: Portability layers implement OS specific functionality. Any OS functions or service implementation has to be in portability layer and OS agnostic code in UMD/KMD uses these interfaces. We have defined interface which has to be implemented in portability layer. Please refer to http://nvdla.org/sw/runtime_environment.html#kmd-layer and http://nvdla.org/sw/runtime_environment.html#umd-layer

Question: Is KMD stack mainly composed of the firmware (scheduler) and Linux Kernel Driver?
Answer: Yes, firmware includes engine scheduler + HW programming

Question: Does KMD stack have system calls for power management in the Liux Kernel?
Answer: No, as it is vendor implementation specific

Question: Suppose that runtime environment is going to be running on Android (instead of Linux), what do we need to do in this case?
Modify the Linux kernel driver in KMD into Android Kernel driver (such as IPC, power management, etc.)?
If necessary, also modify the Linux kernel driver in UMD into Android kernel driver?
If necessary, move some system calls/functionalities from KMD to UMD (due to discrepancy between Linux kernel and android kernel)?
Does UMD need to be connected to Android HAL?
Application program doesn’t need to be passed to Java framework in android?
What else do we need to modify?
Answer: As such nothing, you will have to port the Makefiles to Android build system but from functional point of view no change should be required. It depends on how you want to access the runtime interfaces. If you have some program in Java framework then you need to either implement Android HAL or connect it through native interface.