/aws-fpga

Official repository of the AWS EC2 FPGA Hardware and Software Development Kit

Primary LanguageVHDLOtherNOASSERTION

Table of Contents

  1. Overview of AWS EC2 FPGA Development Kit
  2. Amazon EC2 F1 platform features
  3. Getting Started
  4. Documentation Overview
  5. Developer Support

Overview of AWS EC2 FPGA Development Kit

AWS EC2 FPGA Development Kit is a set of development and runtime tools to develop, simulate, debug, compile and run hardware accelerated applications on Amazon EC2 F1 instances. It is distributed between this github repository and FPGA Developer AMI provided by AWS with no cost of development tools.

⚠️ NOTE: The developer kit is supported for Linux operating systems only.

Development Flow

After creating an FPGA design (also called CL - Custom logic), developers can create an Amazon FPGA Image (AFI) and easily deploy it to an F1 instance. AFIs are reusable, shareable and can be deployed in a scalable and secure way.

Alt text

Development Environments

Development Environment Description Accelerator Language Hardware Interface Debug Options Typical Developer
Software Defined Accelerator Development using Vitis/SDAccel Development experience leverages an optimized compiler to allow easy new accelerator development or migration of existing C/C++/openCL, Verilog/VHDL to AWS FPGA instances C/C++/OpenCL, Verilog/VHDL (RTL) OpenCL APIs and XRT SW/HW Emulation, Simulation, GDB, Virtual JTAG (Chipscope) SW or HW Developer with zero FPGA experience
Hardware Accelerator Development using Vivado Fully custom hardware development experience provides hardware developers with the tools required for developing AFIs for AWS FPGA instances Verilog/VHDL XDMA Driver, peek/poke Simulation, Virtual JTAG HW Developer with advanced FPGA experience
IP Integrator/High Level Design(HLx) using Vivado Graphical interface development experience for integrating IP and high level synthesis development Verilog/VHDL/C XDMA Driver, peek/poke Simulation, Virtual JTAG HW Developer with intermediate FPGA experience

For on-premise development, SDAccel/Vitis/Vivado must have the correct license and use one of the supported tool versions.

FPGA Developer AMI

The FPGA Developer AMI is available on the AWS marketplace without a software charge and includes tools needed for developing FPGA Designs to run on AWS F1.

Given the large size of the FPGA used inside AWS F1 Instances, Xilinx tools work best with 32GiB Memory. z1d.xlarge/c5.4xlarge and z1d.2xlarge/c5.8xlarge instance types would provide the fastest execution time with 30GiB+ and 60GiB+ of memory respectively. Developers who want to save on cost, could start coding and run simulations on low-cost instances, like t2.2xlarge, and move to the aforementioned larger instances to run the synthesis of their acceleration code.

AWS marketplace offers multiple versions of the FPGA Developer AMI. The following compatibility table describes the mapping of currently supported developer kit versions to AMI versions:

Developer Kit Version Tool Version Supported Compatible FPGA Developer AMI Version
1.4.16+ 2020.1 v1.9.0-v1.9.X (Xilinx Vivado/Vitis 2020.1)
1.4.13+ 2019.2 v1.8.0-v1.8.X (Xilinx Vivado/Vitis 2019.2)
1.4.11+ 2019.1 v1.7.0-v1.7.X (Xilinx Vivado/SDx 2019.1)
1.4.8 - 1.4.15a 2018.3 v1.6.0-v1.6.X (Xilinx Vivado/SDx 2018.3)
1.4.3 - 1.4.15a 2018.2 v1.5.0-v1.5.X (Xilinx Vivado/SDx 2018.2)
1.3.7 - 1.4.15a 2017.4 v1.4.0-v1.4.X (Xilinx Vivado/SDx 2017.4)

⚠️ Developer kit release v1.4.16 will remove support for Xilinx 2017.4, 2018.2, 2018.3 toolsets. While developer kit release v1.4.16 onwards will not support older Xilinx tools, you can still use them using HDK releases v1.4.15a or earlier. Please checkout the latest v1.4.15a release tag from Github to use Xilinx 2017.4, 2018.2, 2018.3 toolsets.

⚠️ Developer kit versions prior to v1.3.7 and Developer AMI prior to v1.4 (2017.1) reached end-of-life. See AWS forum announcement for additional details.

For software-defined development please look at the runtime compatibility table based on the Xilinx toolset in use: SDAccel or Vitis

Hardware Development Kit (HDK)

The HDK directory contains documentation, examples, simulation, build and AFI creation scripts to start building Amazon FPGA Images (AFI).
The HDK can be installed on any on-premises server or an EC2 instance. The developer kit is not required if you plan to use a pre-built AFI shared from another developer.

Software-defined Development Environment

The software-defined development environment allows customers to compile their C/C++/OpenCL code into the FPGA as kernels, and use OpenCL APIs to pass data to the FPGA. Software developers with no FPGA experience will find a familiar development experience that supercharges cloud applications.

In addition, this development environment allows for a mix of C/C++ and RTL accelerator designs into a C/C++ software based development environment. This method enables faster prototyping using C/C++ while supporting manual optimization of critical blocks within RTL. This approach is similar to optimizing time critical functions using software compiler optimization methods.

To get started with Xilinx SDAccel, review the Software-defined development environment readme. To get started with Xilinx Vitis, review the Vitis unified development environment readme.

Runtime Tools (SDK)

The SDK directory includes the runtime environment required to run on EC2 FPGA instances. It includes the drivers and tools to manage the AFIs that are loaded on the FPGA instance. The SDK isn't required during the AFI development process; it is only required once an AFI is loaded onto an EC2 FPGA instance. The following sdk resources are provided:

  • Linux Kernel Drivers - The developer kit includes three drivers:
    • XDMA Driver - DMA interface to/from HDK accelerators.
  • FPGA Libraries - APIs used by C/C++ host applications.
  • FPGA Management Tools - AFI management APIs for runtime loading/clearing FPGA image, gathering metrics and debug interface on the F1 instance.

Amazon EC2 F1 Platform Features

  • 1-8 Xilinx UltraScale+ VU9P based FPGA slots
  • Per FPGA Slot, Interfaces available for Custom Logic(CL):
    • One x16 PCIe Gen 3 Interface
    • Four DDR4 RDIMM interfaces (with ECC)
    • AXI4 protocol support on all interfaces
  • User-defined clock frequency driving all CL to Shell interfaces
  • Multiple free running auxiliary clocks
  • PCI-E endpoint presentation to Custom Logic(CL)
    • Management PF (physical function)
    • Application PF
  • Virtual JTAG, Virtual LED, Virtual DIP Switches
  • PCI-E interface between Shell(SH) and Custom Logic(CL).
    • SH to CL inbound 512-bit AXI4 interface
    • CL to SH outbound 512-bit AXI4 interface
    • Multiple 32-bit AXI-Lite buses for register access, mapped to different PCIe BARs
    • Maximum payload size set by the Shell
    • Maximum read request size set by the Shell
    • AXI4 error handling
  • DDR interface between SH and CL
    • CL to SH 512-bit AXI4 interface
    • 1 DDR controller implemented in the SH (always available)
    • 3 DDR controllers implemented in the CL (configurable number of implemented controllers allowed)

Getting Started

Getting familiar with AWS

If you have never used AWS before, we recommend you start with AWS getting started training, and focus on the basics of the AWS EC2 and AWS S3 services. Understanding the fundamentals of these services will make it easier to work with AWS F1 and the FPGA Developer Kit.

FPGA Image generation and EC2 F1 instances are supported in the us-east-1 (N. Virginia), us-west-2 (Oregon), eu-west-1 (Ireland) and us-gov-west-1 (GovCloud US) regions.

⚠️ NOTE: By default, your AWS Account will have an EC2 F1 Instance launch limit of 0. Before using F1 instances, you will have to open a Support Case to increase the EC2 Instance limits to allow launching F1 instances.

Setting up development environment for the first time

You have the choice to develop on AWS EC2 using the FPGA Developer AMI or on-premise.

ℹ️ INFO: We suggest starting with the FPGA Developer AMI with build instances on EC2 as it has Xilinx tools and licenses setup for you to be able to quickly get into development.

ℹ️ INFO: For on-premise development, you will need to have Xilinx tools and licenses available for you to use

  1. Start a Build Instance first to start your development.

    💡 TIP: This instance does not have to be an F1 instance. You only require an F1 instance to run your AFI's(Amazon FPGA Image) once you have gone through your design build and AFI creation steps.

    ℹ️ INFO: If you need to follow GUI Development flows, please checkout our Developer Resources where we provide Step-By-Step guides to setting up a GUI Desktop.

  2. Clone the FPGA Developer Kit on your instance. git clone https://github.com/aws/aws-fpga.git

  3. Follow the quickstarts from the next section.

Quickstarts

Before you create your own AWS FPGA design, we recommend that you go through one of the step-by-step Quickstart guides:

Description Quickstart Next Steps
Software Defined Accelerator Development using Xilinx Vitis Vitis hello_world Quickstart 60+ Vitis examples, Vitis Library Examples
Software Defined Accelerator Development using Xilinx SDAccel SDAccel hello_world Quickstart 60+ SDAccel examples
Custom Hardware Development(HDK) HDK hello_world Quickstart CL to Shell and DRAM connectivity example, Virtual Ethernet Application using the Streaming Data Engine
IP Integrator/High Level Design(HLx) IPI hello_world Quickstart IPI GUI Examples

ℹ️ INFO: For more in-depth applications and examples of using High level synthesis, Vitis Libraries, App Notes and Workshops, please refer to our Example List

How Tos

How To Description
Migrate Alveo U200 designs to F1 This application note shows the ease of migrating an Alveo U200 design to F1.

Documentation Overview

Documentation is located throughout this developer kit and the table below consolidates a list of key documents to help developers find information:

Topic Document Name Description
AWS setup Setup AWS CLI and S3 Bucket Setup instructions for preparing for AFI creation
Developer Kit RELEASE NOTES, Errata Release notes and Errata for all developer kit features, excluding the shell
Developer Kit Errata Errata for all developer kit features, excluding the shell
F1 Shell AWS Shell RELEASE NOTES Release notes for F1 shell
F1 Shell AWS Shell ERRATA Errata for F1 shell
F1 Shell AWS Shell Interface Specification Shell-CL interface specification for HDK developers building AFI
F1 Shell - Timeout and AXI Protocol Protection How to detect a shell timeout The shell will terminate transactions after a time period or on an illegal transaction. This describes how to detect and gather data to help debug CL issues caused by timeouts.
Vitis Debug Vitis Kernel Instructions on debugging Vitis Kernel
Vitis Create Runtime AMI Instructions on creating a runtime AMI when using Xilinx Vitis
Vitis XRT Instructions Instructions on building, installing XRT with MPD daemon considerations for F1
SDAccel Debug RTL Kernel Instructions on debugging RTL Kernel with SDAccel
SDAccel Create Runtime AMI Instructions on creating a runtime AMI when using Xilinx SDAccel
HDK - Host Application Programmer View Host application to CL interface specification
HDK - CL Debug Debug using Virtual JTAG Debugging CL using Virtual JTAG (Chipscope)
HDK - Simulation Simulating CL Designs Shell-CL simulation specification
HDK - Driver README Describes the DMA driver (XDMA) used by HDK examples and includes a link to an installation guide
AFI AFI Management SDK CLI documentation for managing AFI on the F1 instance
AFI - EC2 CLI copy_fpga_image, delete_fpga_image, describe_fpga_images, fpga_image_attributes CLI documentation for administering AFIs
AFI - Creation Error Codes create_fpga_image_error_codes CLI documentation for managing AFIs
AFI - Power FPGA Power, recovering from clock gating Helps developers with understanding FPGA power usage, preventing power violations on the F1 instance and recovering from a clock gated slot.
On-premise Development Tools, Licenses required for on-premise development Guidance for developer wanting to develop AFIs from on-premises instead of using the FPGA Developer AMI
Frequently asked questions FAQ Q/A are added based on developer feedback and common AWS forum questions

Developer Support

  • The Amazon FPGA Development User Forum is the first place to go to post questions, learn from other users and read announcements.

    • We recommend joining the AWS forums to engage with the FPGA developer community, AWS and Xilinx engineers to get help.
  • You could also file a Github Issue for support. We prefer the forums as this helps the entire community learn from issues, feedback and answers.

    • Click the "Watch" button in GitHub upper right corner to get regular updates.