Table of Contents
Overview of opae-xilinx
The purpose of this project is to port OPAE to Xilinx FPGA devices.
The original OPAE and FPGA accelerator:
The ported OPAE and FPGA accelerator:
FIM and AFU design
To be discoverable and managable by the OPAE PCIe driver, the design of FIU(FPGA Interface Unit, the PCIe interface logic) should be compliant with the OPAE specification. So a major design effort is put into the FIM(FPGA Interface Manager) part, which is a static 'Shell' that resides persistantly in the FPGA.
Based on the 'Shell', a design flow of the AFU(Accelerator Function Unit) part, which is a dynamic 'Role' that can be swapped in and out of the FPGA, is setup to utilize open-source projects supporting HLS, such as FINN-HLS, GEMM_HLS, hlslib, BISMO, VTA, HeteroCL and Vitis Library L1 modules.
Using AXI instead of CCI-P
Although the OPAE specification mandates the use of CCI-P interface between FIM and AFU when targeting Intel MCP and DCP platforms, this project uses AXI interface instead. The inclusion of an industry standard interface makes the OPAE ecosystem truly Vendor Neutral and makes the many IPs targeting ASIC designs available to FPGA designers, such as MatchLib and HLSLibs.
Demo: Full-System Simulation with QEMU
A full-system simulation, which involves application/driver software code and FIM/AFU hardware logic, not only speeds up the development and debugging process of the SW/HW interface, but also enables the evaluation of this full-stack solution without a physical FPGA acceleration card. Please take a look at the README for details.
OPAE-scan results in QEMU-HDL co-simulation:
OPAE-scan AXI-bus transaction waveform in QEMU-HDL co-simulation:
OPAE Sample Applications: Hello_FPGA with Xilinx CDMA
Application of Xilinx CDMA IP in C.
Application of Xilinx CDMA IP in Python.
The above sample applications have been validated in the QEMU-HDL co-simulation environment.
Docker script to setup an OPAE run-time environment
Build OPAE on Ubuntu 16.04 from source code.
The original script is here. Thank you so much akirajoeshoji, for the inspiration!
Directory Structure
.
├── doc
│ ├── dmesg
│ └── pics
├── hw
│ ├── prj
│ │ ├── afu
│ │ ├── blue_bs
│ │ ├── fim
│ │ ├── green_bs
│ │ └── qemu_hdl_cosim
│ └── src
│ ├── afu_customize
│ ├── constraints
│ ├── hdl
│ ├── ip
│ ├── ipi
│ ├── qemu_hdl_cosim
│ └── sim
└── sw
├── Docker
├── OPAE
│ ├── driver
│ └── sdk
│ └── opae-sdk-1.3.0-2
│ ├── pyopae
│ │ └── samples
│ └── samples
└── QEMU
└── qemu_hdl_cosim
Development Tools
The FPGA projects are designed with 2018.3 release of Vivado and Vivado HLS.
The FPGA platform currently supported is the KCU105 development board from Xilinx.
Getting Started
To get started with the design of FIM and AFU, or the generation of Blue and Green bitstreams, follow the README in ecah of the directories under ./hw/prj.
ToDo List
- [Done] Add Container scripts to install OPAE driver/sdk/pyopae - Clean environment.
- Use Verilator/GtkWave in the QEMU-HDL cosimulation - Truely open source tools based; Mixed C/Verilog simulation.
- Port Xilinx PR driver to OPAE - FME functionality enhancement.
- Add Ethernet interface to the FIM - FIM functionality enhancement/AFU BBB optional component logic.
- Add DDR interface to the FIM - FIM functionality enhancement/AFU BBB optional component logic.
- Add AFU BBB logic components and compilation flow scripts - A synthesis flow for automatic HLS IP integration.
- Optimize FIM pyhsical constraints - Available area estimation for AFU.
Wish List
- Support SR-IOV.
- Support dual PF.
- Add AFU BBB logic components for supporting OpenCL/SyCL/oneAPI - ?.
- PYNQ/XRT compliant - ?.
- VirtIO NIC compatible with ixy user space driver - Network attached accelerator (SmartNIC).
- Add a RISC-V Core as an AXI-Lite master in the FIM - Autonomous task scheduling and hardware microservice.