Dremio Accelerated is an extension of Dremio's Sabot engine which enables the transparent offloading of subqueries to FPGA's. The evaluation of these subqueries is facilitated by Fletcher, which decreases the time required to design and implement the required FPGA kernels.
An additional FPGA Acceleration Planning phase is added to the Sabot engine, which facilitates rules to target nodes in the physical plan and substitute them by a Fletcher operator. This operator exposes it's underlying Arrow buffers to the FPGA and specifies the location to which the results should be written. These extensions can be seen in blue in the figure below.
This illustration is in the context of an example in which an FPGA kernel can evaluate a filter and perform both a project and stream aggregate operation. More detailed illustrations for specific usecases can be seen in the section below.
- Regular expression filter on taxi dataset - evaluating a filter operation on FPGA
Planned usecases:
- Regular expression filter on covid-19 tweets - (WIP) evaluating a regex filter on a dataset with larger strings
- Filter and StreamAggregate - (WIP) offloading multiple operators
- TPC-H Query 6 - (WIP) offloads hashaggregate, project, and filter operators
Follow the instructions in the installation guide to get Dremio Accelerated up and running.