aws/aws-fpga

LUT utilization target

useragdp1 opened this issue · 12 comments

useragdp1 commented

My design is consumes ~55% LUTs. The RAM and DSP utilization is pretty low < 10%. When I try add more logic it runs into congestion issues. So, I am trying to reduce some bus widths. However, I would like to know if there is a good rule of thumb on what LUT utilization can be target without hitting congestion issues. I am hoping you guys have such data from various customer datapoints. Thanks

AWScsaralay commented

Hello,

While the 55% LUT utilization may not be the primary reason for congestion, the tool may still struggle to close timing if there are multiple SLR crossings, datapaths with greater than 10 levels of logic and heavy usage of global asynchronous resets. Please try addressing these issues, and also add multiple pipeline stages to spread out the logic in entire SLRs.

Also, please try floorplanning the design based on SLRs to be used. An example to constrain a module to a specific SLR is as shown below:
https://github.com/aws/aws-fpga/blob/master/hdk/cl/examples/cl_dram_dma/build/constraints/cl_pnr_user.xdc#L3

Additional resources to help timing closure:
https://github.com/aws/aws-fpga/blob/master/hdk/docs/AWS_Shell_Interface_Specification.md#implementation-tips
https://www.xilinx.com/content/dam/xilinx/support/documentation/sw_manuals/xilinx2021_2/ug1292-ultrafast-timing-closure-quick-reference.pdf
https://www.xilinx.com/content/dam/xilinx/support/documentation/sw_manuals/xilinx2021_2/ug949-vivado-design-methodology.pdf#nameddest=TimingClosure

Please feel free to contact us if you have any questions.

Thanks!
Chakra

useragdp1 commented

Hi Chakra,
I am able to resolve congestion using some of techniques you pointed to. But, we want to push more logic into the FPGA. For example, it seems obvious 80% utilization would be impossible for us to route (I have tried). So, the number is somewhere between 55-80% which is a large range to run experiments with. In order to reduce our development time, I am looking for some guidance on how much more LUTs can we practically push into this FPGA. Some datapoints would be very helpful.
Thanks

AWScsaralay commented

Unfortunately, since each design is different in terms of pipeline stages, floorplanning, place and route, etc, it is difficult to pin-point a "safe" utilization number. We have seen designs with > 70% LUT utilization close timing at 250MHz, but with very efficient/optimized RTL datapaths and floorplan constraints.

Alternately, you could try running design at slower clock speeds (if that is an acceptable solution). Also, if your appllication does not rely on XDMA engine then I would recommend you to try our Small Shell which is 30% smaller in size and offers more resources to the Customer Logic:
https://github.com/aws/aws-fpga/tree/small_shell#small-shell

Please let me know if you would like any additional details.

Thanks!
Chakra

useragdp1 commented

Hi Chakra,
I would like to use the small_shell since we do not use XDMA. Do I really need to rollback the Vivado from 2021.1 to 2020.2?
Also, would it require change in SW driver? We have some working version with the current shell.
Thanks
Abhay

AWScsaralay commented

Hello Abhay,

Small Shell is supported on vivado tool versions 2020.2 and above. Therefore you don't need to roll back to 2020.2 if you are currently using 2021.1.

Small Shell has same interfaces between Shell and CL as provided by the full sized F1.X.1.4 Shell. Please note that the floorplan location of PCIM and PCIS interfaces have changed in Small Shell. Please refer to the guidelines here https://github.com/aws/aws-fpga/blob/small_shell/hdk/docs/AWS_Shell_Interface_Specification.md#multi-slr-fpga for optimal floorplanning of your CL. Since the PF/BAR ranges did not change, SW driver that worked with full sized F1.X.1.4 Shell should also work with Small Shell.

Please let us know if you see any issues or have any questions.

Thanks!
Chakra

useragdp1 commented

Thanks Chakra

useragdp1 commented

I bypasses the Vivado version check in hdk_setup for small_shell. But ran into these issues while synthesising.

WARNING: [IP_Flow 19-2162] IP 'vio_0' is locked:

  • IP definition 'VIO (Virtual Input/Output) (3.0)' for IP 'vio_0' (customized with software release 2017.4.op) has a different revision in the IP Catalog.
    WARNING: [IP_Flow 19-2162] IP 'src_register_slice' is locked:
  • IP definition 'AXI Register Slice (2.1)' for IP 'src_register_slice' (customized with software release 2017.4.op) has a different revision in the IP Catalog.
    WARNING: [IP_Flow 19-2162] IP 'ila_vio_counter' is locked:
  • IP definition 'ILA (Integrated Logic Analyzer) (6.2)' for IP 'ila_vio_counter' (customized with software release 2017.4.op) has a different revision in the IP Catalog.
    WARNING: [IP_Flow 19-2162] IP 'ila_0' is locked:
  • IP definition 'ILA (Integrated Logic Analyzer) (6.2)' for IP 'ila_0' (customized with software release 2017.4.op) has a different revision in the IP Catalog.
    WARNING: [IP_Flow 19-2162] IP 'dest_register_slice' is locked:
  • IP definition 'AXI Register Slice (2.1)' for IP 'dest_register_slice' (customized with software release 2017.4.op) has a different revision in the IP Catalog.
    WARNING: [IP_Flow 19-2162] IP 'axi_register_slice' is locked:
  • IP definition 'AXI Register Slice (2.1)' for IP 'axi_register_slice' (customized with software release 2017.4.op) has a different revision in the IP Catalog.
    WARNING: [IP_Flow 19-2162] IP 'cl_debug_bridge' is locked:
  • IP definition 'Debug Bridge (3.0)' for IP 'cl_debug_bridge' (customized with software release 2017.4.op) has a different revision in the IP Catalog.
    WARNING: [IP_Flow 19-2162] IP 'axi_register_slice_light' is locked:
  • IP definition 'AXI Register Slice (2.1)' for IP 'axi_register_slice_light' (customized with software release 2017.4.op) has a different revision in the IP Catalog.
    Attempting to get a license for feature 'Synthesis' and/or device 'xcvu9p'
    INFO: [Common 17-349] Got license for feature 'Synthesis' and/or device 'xcvu9p'
    INFO: [Vivado_Tcl 4-550] User specified maximum number of URAM that can be cascaded is 2
    INFO: [Device 21-403] Loading part xcvu9p-flgb2104-2-i
    INFO: [Synth 8-7079] Multithreading enabled for synth_design using a maximum of 4 processes.
    INFO: [Synth 8-7078] Launching helper process for spawning children vivado processes
    INFO: [Synth 8-7075] Helper process launched with PID 757825

INFO: [Synth 8-6157] synthesizing module 'sh_ddr' [/data//awssmall/hdk/common/shell_v04182104/design/sh_ddr/synth/sh_ddr.sv:32]
Parameter DDR_A_PRESENT bound to: 1 - type: integer
Parameter DDR_B_PRESENT bound to: 1 - type: integer
Parameter DDR_D_PRESENT bound to: 0 - type: integer
ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/data//awssmall/hdk/common/shell_v04182104/design/sh_ddr/synth/sh_ddr.sv:610]
ERROR: [Synth 8-5809] Error generated from encrypted envelope.

Finished RTL Elaboration : Time (s): cpu = 00:00:19 ; elapsed = 00:00:24 . Memory (MB): peak = 3275.777 ; gain = 453.438 ; free physical = 13192 ; free virtual = 26431

RTL Elaboration failed

czfpga commented

Hi abhaypelago,

Thank you for reaching out. Looks like we might need to add support for version 2021.1. I did bypass the vivado version check and build the example of CL_DRAM_DMA in 2021.1 and saw no error during RTL elaboration. Is this from building a CL examples or your own CL design? Thanks.

  • Chen
useragdp1 commented

thanks for the hint. I should have copied the build files from cl_dram_dma. I am able to make progress. When can vivado version 2021.1 be support be added? I am proceeding by skipping the check and try the image next week.

useragdp1 commented

The small_shell is unable to make under cl_dram_dma/verif/scripts
ERROR: [VRFC 10-2063] Module <axi_register_slice_v2_1_22_axi_register_slice> not found while processing module instance [/data/awssmall/aws-fpga/hdk/common/shell_v04182104/design/ip/axi_register_slice_axuser/sim/axi_register_slice_axuser.v:318]
Note that I had to remove xsc cmd from Makefile since --additional_options is deprecated in newer vivado version

czfpga commented

Hi abhaypelago,

Thank you for sharing the updates. These are very helpful data points that I'll bring back to the development team. I believe at this moment, the best option is to use Vivado 2020.2 for your project since that's the version officially supported and has been validated. The development team is looking at adding supports for newer tool versions. There is no ETA for that yet.

Thanks again,

Chen

czfpga commented

Hi abhaypelago,

I'll close this issue. Please feel free to open a new one if you have any other questions. Thanks.

Chen