Error in parent_model = model.transform(CreateDataflowPartition()
Ba1tu3han opened this issue · 8 comments
Prerequisites
Please make sure to check off these prerequisites before submitting a bug report.
- [+ ] Test that the bug appears on the current version of the dev-branch. Make sure to include the commit hash of the commit you checked out.
- [+] Check that the issue hasn't already been reported, by checking the currently open issues.
- If there are steps to reproduce the problem, make sure to write them down below.
- [+] If relevant, please include the ONNX files, which were created directly before and/or after the bug.
Quick summary
I trained the example model which is in Brevitas website with MNIST dataset. Then I imported it to FINN example "tfc-end2end-example" (in attachment). parent_model = model.transform(CreateDataflowPartition()) this line has an error in Creating a Dataflow Partion part. You can see the whole error in following screenshots.
I'm using current dev-branch.
Additional context
Could you help me to solve it?
Best Regards,
You probably have layers in the middle of your model which are not converted to HLS layers. Could you please share your ONNX graph right before running the CreateDataflowPartition transformation?
Briefly looking into your model graph (that is probably directly after export, right?), I suspect it might be due to exporting to QCDQ format instead of QONNX format. The QuantizeLinear and DequantizeLinear layers will not be turned into FINN MultiThresholds. If that is the case, you have two options: Either directly export to QONNX format via Brevitas, or you could try running the QCDQToQuant transformation from the QONNX package before doing the conversion to HLS layers.
Furthermore, as your model is a convolutional model, you should probably follow the cnv_end2end_example instead of the tfc_end2end_example, as the cnv variant contains steps to properly handle the convolution operations by lowering them to matmuls.
Hello Dear Christoph,
Thank you for your reply.
I will share the ONNX file you mentioned when I access to my computer today.
I could not find how to export directly to QONNX via Brevitas. I only found export_onnx_qcdq() function in the Getting Started webpage of Brevitas. Could you share with me any document about the exporting?
Additionally, thank you for your guidance about example notebooks.
Have a good day!
The QONNX export is from brevitas.export import export_qonnx, but you could just try the conversion pathway from QCDQ as well. You might also run into similar issues as the referenced #938, if you are using the bnn example notebooks for other than binarized models, as in your case 4-bit - not sure whether you have already adapted the notebook to account for this, just be cautious.
I thought that I had a problem with my model and changed it with Brevitas FC then exported it with export_qonnx as you said. I directly imported the QONNX file to the example notebook.
I set width of weight, activation and initial to 1 bit as you recommend in #938, then it is solved.
model = model.transform(to_hls.InferQuantizedMatrixVectorActivation("decoupled")) it is solved for other combination except for 1w1a. But I have another problem.
AssertionError: IPGen failed: /tmp/finn_dev_ba/code_gen_ipgen_StreamingDataflowPartition_1_Thresholding_Batch_0_hifo_j1w/project_StreamingDataflowPartition_1_Thresholding_Batch_0/sol1/impl/ip not found. Check log under /tmp/finn_dev_ba/code_gen_ipgen_StreamingDataflowPartition_1_Thresholding_Batch_0_hifo_j1w
in
model = model.transform(ZynqBuild(platform = pynq_board, period_ns = target_clk_ns))
QONNX file: QONNX_FC.zip
Torch model: QONNX_FC.zip
Python Code: custom-tfc_end2end_example 18.12.2023.zip
Additionally, while I am exporting the QONNX file just after I trained it, I get this warning:
Warning: The shape inference of onnx.brevitas::BipolarQuant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Thank you,
Regarding the failing IPGen, you should check the log under the specified path to see what is actually going on. There should be some lines with ERROR: towards the end, which typically tell you exactly what is wrong.
Regarding the seemingly missing shape inference for BipolarQuant: I think it is safe to ignore this. It is probably just an issue during export (when PyTorch is still involved), but as soon as you are relying solely on QONNX, the BipolarQuant should be fully supported by shape inference, if I recall correctly. So, as long as you do not see proper shape inference errors later, just ignore it.