[Error Report] Benchmarks do not run

Question

[Error Report] Benchmarks do not run

luojw-dwr opened this issue 3 years ago · 9 comments

Hi,

I am running benchmarks/CNN/13x10/runme.sh with Vivado HLS 2019.2/2020.1 and Python 3.6.1. When executing./ab_cnn_13x10.py it reports:

Using Python-MIP package version 1.8.1
Traceback (most recent call last):
  File "./ab_cnn_13x10.py", line 101, in <module>
    NaiveBalance = NaiveBalance)
TypeError: __init__() got an unexpected keyword argument 'column'
Detecting error, Rerun AutoBridge...
Using Python-MIP package version 1.8.1
Traceback (most recent call last):
  File "./ab_cnn_13x10.py", line 101, in <module>
    NaiveBalance = NaiveBalance)
TypeError: __init__() got an unexpected keyword argument 'column'

Similar problems also occur withbenchmarks/Stencil/3PE/runme.sh, benchmarks/LUDecomposition/12x12/u250/runme.sh and benchmarks/PageRank/runme.sh.

Please contact if more concrete information is needed to debug. Thanks for your time.

Answer 1 · 2021-03-12T17:17:26.000Z

Hi,
I'll check what's going on later today.
Probably some of my latest updates broke the previous scripts.
Sorry about that!

Answer 2 · 2021-03-12T22:08:57.000Z

Hi, can you try the latest commit?
We have a lot of problems with the naming convention of HLS.

Answer 3 · 2021-03-13T04:28:13.000Z

Hi, with the latest commit the 'column'-related problems have been resolved. However, CNN(13x2) reports missing file global.setting.tcl when running run_vitis.sh; by contrast, CNN(13x4) runs alright. Due to lack of computational resources, I have only retried some of the benchmarks. Please help to make sure whether this is a bug of Vivado or benchmark scripts.

By the way, many [reBalance] FIXME need to handle async_mmap are reported in benchmark PageRank. Do these warnings affect the correctness of the resulted functionality?

Thanks for your patience.

Answer 4 · 2021-03-13T07:35:03.000Z

Hi, the CNN 13x2 works fine on my side. Have you tried using the Gurobi solver? You need to register for a free trial, but it's easy and quick. I remember sometimes the interface with the default open-sourced ILP solver has some glitches.

The pagerank warning is fine. Also, right now I'm refactoring the entire code base and will release the new code soon, so I probably will not update on that warning in the current code base.

If you are interested in using the tool, you are welcome to contact me directly and I can help with your specific cases.

Answer 5 · 2021-03-13T08:04:51.000Z

Hi,

It seems that the mentioned problem about benchmark CNN 13x2 has nothing to do with the python world, I wonder whether it comes from tcl scripts. Whatever, this seems not be a problem that deserves this much concern.

In fact, I am an immature developer rather than an AB user at present. I am trying to split current AutoBridge pipeline after which I can plug-in my own frontend and optimization model. It would be my honor if we can discuss about the architecture design of a new code base (maybe in a new issue or through e-mail).

Answer 6 · 2021-03-15T06:43:52.000Z

Hi @luojw-dwr,

Seems that my previous message failed to reach your email.

Thanks for your detailed explanation. The new code base has better modularity than the existing one.

However, since I'm working on other new features, the original RTL generation (add additional pipelining) and constraint generation part have not been added to the new framework.

If you are interested, I could first publicize part of the new code base (corresponds to what's covered by the AutoBridge paper) if my advisor agrees. You are welcome to contribute to the repo to add your customization. Your suggestions surely sound promising :)

Best,
Licheng

Answer 7 · 2021-03-15T08:42:33.000Z

Hi @Licheng-Guo ,

It will be quite appreciated if you can publicize part of the under-development codes to allow external contribution. Keeping on a consistent track with you is an attractive choice for me. Let's make AutoBridge a great framework!

Thanks for your time and patience.

Yours sincerely,
Jianwen

p.s. About the communication problem, I am not quite sure which is your 'previous message'. Maybe it is my side which failed to send the message. For the convenience of future developers, please allow me to attach the technical part of my e-mail:

Hi Licheng,

First, I’d like to explain what I am going to do with AutoBridge.

I am primarily switching to another coarse-grained representation of the dies, namely slot vertices fully connected with edges weighted by abstract distances (e.g. Manhattan distances derived from your slot grids).

This will allow more flexible situations, e.g. some die in the middle has been almost fully occupied by some IPs, which seems not to be compatible with your original model.

The cost of my model is that the objective function is no longer linear, but gurobi can still solve it. (The resulted efficiency remains to be measured.)

The problems I have met with the current code base:

The data of dataflow graph, the temporary variables introduced for the optimization process and the floorplan representation are strongly coupled.

The floorplan pipelining input format is strongly coupled with the optimization model.

About the architecture for an ideal code base of AutoBridge, I think it should be split as the following:

Die graph: vertex(die), edge(connectivity * user weight)

Dataflow graph: vertex(Area), edge(FIFO info, e.g. width)

Slot graph: vertex(Area), edge(abstract distance)

Equipped with slot graph vertex -> die graph vertex

Equipped with slot graph edge -> die graph path

Floorplan: Dataflow graph (V, E) -> slot graph (V, E) (-> die graph (V, List[E]))

Dataflow graph extractor: source -> Dataflow graph.

Currently from VHLS-generated RTL

Ideally the source should be adequate to determine the mapping from dataflow graph vertex to resulted RTL. Otherwise, exceptions like “xxx_entryzz_U” may temporarily provided by user.

Making this stage standalone provides modularity to adapt to other frontends like HeteroCL to assure constraints on the resulted RTLs and make easier to extract dataflow topology.

Floorplanner: Dataflow graph (-> Optimization Dataflow graph) -> Floorplan; which may either statically load a slot graph or dynamically update the slot graph inside.

Pipeliner: Floorplan -> Pipelining constraints.

Other additional ideas come as the following:

Make use of yosys for uniquifing.

Make use of the open-sourced vitis-hls frontend for metadata insertion to aid automation on resolving naming conventions.

…

Hope that these ideas can help. Looking forward to your reply.

Yours sincerely,
Jianwen Luo

Answer 8 · 2021-03-15T22:36:29.000Z

Hi @luojw-dwr,

Please check the latest commit. Let me know if you have any suggestions about the organization or any issues. Thanks!

Licheng

Answer 9 · 2021-03-16T05:27:32.000Z

Hi @Licheng-Guo ,

About the HLSParser package, it is nice to see that tapa frontend has been added. It is quite a good beginning.

About the Device package, my idea is to leave the concrete device information to plain-text files, e.g. yaml or csv. We may have further discussion about how to represent the 'branches' on the resource variations induced by composition of instantiations of IPs, e.g. DDR, NoC infrastructures, etc. What choices come to me at this early stage:

record different resource situation as you've done in the old code base, but this may be a problem when the compositions are of a terrifying number.
let users pick the correct 'branch' in the resource file. (then supporting on comments can be a must for the file format)

About the Opt package, the components are still heavily coupled with the details of the proposed floorplanning algorithm. I'd like to start a new git branch to get them decoupled as I've mentioned in the last few e-mails. Hopefully I will keep the interfaces consistent (one exception is the coordination of floorplanner and slots, as the organizaton of slots may vary a lot depending on the floorplanner implementation).

About the Flow package, the Floorplan (optional) setion in def help seems to be confusing. It is really a problem for both of us to allow user to make partial assignment with a tidy interface without knowledge about the floorplanning implementation details on naming conventions.

This may be a problem if users want to connect vhls and autobridge directly as a whole workflow. (Some manual coding is not too big a realistic problem at this stage though).
The naming convention (the CLOCKREGION stuff) is not quite obvious even after I have read the Device package. This can be a real problem as normally user won't dive into the Opt package for namings (though the floorplanner itself and the constraint stuffs are there).
Maybe either
- expose the namings to somewhere more obvious, or
- let users bind to slot graphs and leave us to manage the name mappings. (It is not a choice if the slot graph is dynamically generated during floorplanning though. Several round trips between abstract coordinates and slot-during-iteration is otherwise needed.)

By the way, discussing in a closed issue seems not to be a correct way using GitHub. Maybe start a new issue ;-P

It is nice to be on a consistent track with you!

Best,
Jianwen