Simulation gets stuck with SIM=verilator 4.106
andreaskuster opened this issue · 10 comments
Thanks a lot for this great project! @alexforencich @rodrigomelo9
I am trying to run the axi-verilog designs using verilator, however, this library does not seem to be compatible with verilator. I use version 4.106 which should be compatible with cocotb (all their base examples work).
However, it seems as if the simulation tests are stuck here: https://github.com/alexforencich/cocotbext-axi/blob/master/cocotbext/axi/axi_master.py#L413
Commands for reproduction:
python3 -m pip install cocotbext-axi==0.1.18
git clone https://github.com/alexforencich/cocotbext-axi.git
cd cocotbext-axi/tests/axi/
git checkout v0.1.18
SIM=verilator make
running make without SIM=verilator works
The log for the axi test is:
make -f Makefile results.xml
make[1]: Entering directory '/home/andreas/Desktop/cocotbext-axi/tests/axi'
MODULE=test_axi TESTCASE= TOPLEVEL=test_axi TOPLEVEL_LANG=verilog \
sim_build/Vtop
-.--ns INFO cocotb.gpi ..mbed/gpi_embed.cpp:76 in set_program_name_in_venv Did not detect Python virtual environment. Using system-wide Python interpreter
-.--ns INFO cocotb.gpi ../gpi/GpiCommon.cpp:99 in gpi_print_registered_impl VPI registered
0.00ns INFO Running on Verilator version 4.106 2020-12-02
0.00ns INFO Running tests with cocotb v1.6.1 from /home/andreas/.local/lib/python3.8/site-packages/cocotb
0.00ns INFO Seeding Python random module with 1642601669
0.00ns INFO Found test test_axi.run_test_write_001
0.00ns INFO Found test test_axi.run_test_write_002
0.00ns INFO Found test test_axi.run_test_write_003
0.00ns INFO Found test test_axi.run_test_write_004
0.00ns INFO Found test test_axi.run_test_write_005
0.00ns INFO Found test test_axi.run_test_write_006
0.00ns INFO Found test test_axi.run_test_write_007
0.00ns INFO Found test test_axi.run_test_write_008
0.00ns INFO Found test test_axi.run_test_write_009
0.00ns INFO Found test test_axi.run_test_write_010
0.00ns INFO Found test test_axi.run_test_write_011
0.00ns INFO Found test test_axi.run_test_write_012
0.00ns INFO Found test test_axi.run_test_read_001
0.00ns INFO Found test test_axi.run_test_read_002
0.00ns INFO Found test test_axi.run_test_read_003
0.00ns INFO Found test test_axi.run_test_read_004
0.00ns INFO Found test test_axi.run_test_read_005
0.00ns INFO Found test test_axi.run_test_read_006
0.00ns INFO Found test test_axi.run_test_read_007
0.00ns INFO Found test test_axi.run_test_read_008
0.00ns INFO Found test test_axi.run_test_read_009
0.00ns INFO Found test test_axi.run_test_read_010
0.00ns INFO Found test test_axi.run_test_read_011
0.00ns INFO Found test test_axi.run_test_read_012
0.00ns INFO Found test test_axi.run_test_write_words_001
0.00ns INFO Found test test_axi.run_test_read_words_001
0.00ns INFO Found test test_axi.run_stress_test_001
0.00ns INFO running run_test_write_001 (1/27)
0.00ns INFO AXI master (write)
0.00ns INFO cocotbext-axi version 0.1.18
0.00ns INFO Copyright (c) 2020 Alex Forencich
0.00ns INFO https://github.com/alexforencich/cocotbext-axi
0.00ns INFO Reset de-asserted
0.00ns INFO Reset de-asserted
0.00ns INFO Reset de-asserted
0.00ns INFO AXI master configuration:
0.00ns INFO Address width: 32 bits
0.00ns INFO ID width: 8 bits
0.00ns INFO Byte size: 8 bits
0.00ns INFO Data width: 32 bits (4 bytes)
0.00ns INFO Max burst size: 2 (4 bytes)
0.00ns INFO Max burst length: 256 cycles (1024 bytes)
0.00ns INFO AXI master signals:
0.00ns INFO awaddr width: 32 bits
0.00ns INFO awburst width: 2 bits
0.00ns INFO awcache width: 4 bits
0.00ns INFO awid width: 8 bits
0.00ns INFO awlen width: 8 bits
0.00ns INFO awlock width: 1 bits
0.00ns INFO awprot width: 3 bits
0.00ns INFO awqos width: 4 bits
0.00ns INFO awready width: 1 bits
0.00ns INFO awregion width: 4 bits
0.00ns INFO awsize width: 3 bits
0.00ns INFO awuser width: 1 bits
0.00ns INFO awvalid width: 1 bits
0.00ns INFO wdata width: 32 bits
0.00ns INFO wlast width: 1 bits
0.00ns INFO wready width: 1 bits
0.00ns INFO wstrb width: 4 bits
0.00ns INFO wuser width: 1 bits
0.00ns INFO wvalid width: 1 bits
0.00ns INFO bid width: 8 bits
0.00ns INFO bready width: 1 bits
0.00ns INFO bresp width: 2 bits
0.00ns INFO buser width: 1 bits
0.00ns INFO bvalid width: 1 bits
0.00ns INFO Reset de-asserted
0.00ns INFO AXI master (read)
0.00ns INFO cocotbext-axi version 0.1.18
0.00ns INFO Copyright (c) 2020 Alex Forencich
0.00ns INFO https://github.com/alexforencich/cocotbext-axi
0.00ns INFO Reset de-asserted
0.00ns INFO Reset de-asserted
0.00ns INFO AXI master configuration:
0.00ns INFO Address width: 32 bits
0.00ns INFO ID width: 8 bits
0.00ns INFO Byte size: 8 bits
0.00ns INFO Data width: 32 bits (4 bytes)
0.00ns INFO Max burst size: 2 (4 bytes)
0.00ns INFO Max burst length: 256 cycles (1024 bytes)
0.00ns INFO AXI master signals:
0.00ns INFO araddr width: 32 bits
0.00ns INFO arburst width: 2 bits
0.00ns INFO arcache width: 4 bits
0.00ns INFO arid width: 8 bits
0.00ns INFO arlen width: 8 bits
0.00ns INFO arlock width: 1 bits
0.00ns INFO arprot width: 3 bits
0.00ns INFO arqos width: 4 bits
0.00ns INFO arready width: 1 bits
0.00ns INFO arregion width: 4 bits
0.00ns INFO arsize width: 3 bits
0.00ns INFO aruser width: 1 bits
0.00ns INFO arvalid width: 1 bits
0.00ns INFO rdata width: 32 bits
0.00ns INFO rid width: 8 bits
0.00ns INFO rlast width: 1 bits
0.00ns INFO rready width: 1 bits
0.00ns INFO rresp width: 2 bits
0.00ns INFO ruser width: 1 bits
0.00ns INFO rvalid width: 1 bits
0.00ns INFO Reset de-asserted
0.00ns INFO AXI slave model (write)
0.00ns INFO cocotbext-axi version 0.1.18
0.00ns INFO Copyright (c) 2021 Alex Forencich
0.00ns INFO https://github.com/alexforencich/cocotbext-axi
0.00ns INFO Reset de-asserted
0.00ns INFO Reset de-asserted
0.00ns INFO Reset de-asserted
0.00ns INFO AXI slave model configuration:
0.00ns INFO Address width: 32 bits
0.00ns INFO ID width: 8 bits
0.00ns INFO Byte size: 8 bits
0.00ns INFO Data width: 32 bits (4 bytes)
0.00ns INFO AXI slave model signals:
0.00ns INFO awaddr width: 32 bits
0.00ns INFO awburst width: 2 bits
0.00ns INFO awcache width: 4 bits
0.00ns INFO awid width: 8 bits
0.00ns INFO awlen width: 8 bits
0.00ns INFO awlock width: 1 bits
0.00ns INFO awprot width: 3 bits
0.00ns INFO awqos width: 4 bits
0.00ns INFO awready width: 1 bits
0.00ns INFO awregion width: 4 bits
0.00ns INFO awsize width: 3 bits
0.00ns INFO awuser width: 1 bits
0.00ns INFO awvalid width: 1 bits
0.00ns INFO wdata width: 32 bits
0.00ns INFO wlast width: 1 bits
0.00ns INFO wready width: 1 bits
0.00ns INFO wstrb width: 4 bits
0.00ns INFO wuser width: 1 bits
0.00ns INFO wvalid width: 1 bits
0.00ns INFO bid width: 8 bits
0.00ns INFO bready width: 1 bits
0.00ns INFO bresp width: 2 bits
0.00ns INFO buser width: 1 bits
0.00ns INFO bvalid width: 1 bits
0.00ns INFO Reset de-asserted
0.00ns INFO AXI slave model (read)
0.00ns INFO cocotbext-axi version 0.1.18
0.00ns INFO Copyright (c) 2021 Alex Forencich
0.00ns INFO https://github.com/alexforencich/cocotbext-axi
0.00ns INFO Reset de-asserted
0.00ns INFO Reset de-asserted
0.00ns INFO AXI slave model configuration:
0.00ns INFO Address width: 32 bits
0.00ns INFO ID width: 8 bits
0.00ns INFO Byte size: 8 bits
0.00ns INFO Data width: 32 bits (4 bytes)
0.00ns INFO AXI slave model signals:
0.00ns INFO araddr width: 32 bits
0.00ns INFO arburst width: 2 bits
0.00ns INFO arcache width: 4 bits
0.00ns INFO arid width: 8 bits
0.00ns INFO arlen width: 8 bits
0.00ns INFO arlock width: 1 bits
0.00ns INFO arprot width: 3 bits
0.00ns INFO arqos width: 4 bits
0.00ns INFO arready width: 1 bits
0.00ns INFO arregion width: 4 bits
0.00ns INFO arsize width: 3 bits
0.00ns INFO aruser width: 1 bits
0.00ns INFO arvalid width: 1 bits
0.00ns INFO rdata width: 32 bits
0.00ns INFO rid width: 8 bits
0.00ns INFO rlast width: 1 bits
0.00ns INFO rready width: 1 bits
0.00ns INFO rresp width: 2 bits
0.00ns INFO ruser width: 1 bits
0.00ns INFO rvalid width: 1 bits
0.00ns INFO Reset de-asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
2.00ns INFO Reset de-asserted
10.00ns INFO length 1, offset 0
10.00ns INFO Write start addr: 0x00001000 awid: 0x0 prot: AxiProt.NONSECURE data: 00
10.00ns INFO Write burst start awid: 0x0 awaddr: 0x00001000 awlen: 0 awsize: 2 awprot: AxiProt.NONSECURE
Hi @andreaskuster Only to clarify, I am simply a recent user of cocotbext-axi.
I can't right now, but I will try to reproduce it and let you know. Let me say that I ran verilator --version in my Ubuntu 20.04 and it reported Verilator 4.028 2020-02-06 rev v4.026-92-g890cecc1 (are you sure that you are trying 1.106? seems extremely old).
Thanks for your reply. And yes you were right, i meant version 4.106 (fixed it in the description), and the compatibility issue with newer versions i mentioned is according to the cocotb github issues and the comment here: https://docs.cocotb.org/en/stable/simulator_support.html
Furthermore, i tested different (older) versions of cocotbext-axi, with the same behaviour, so this problem seems to be present for a while.
Ok, I didn't know about the cocotb limitation using Verilator. I generally use cocotb with GHDL or iverilog.
Sure, ghdl and iverilog are fine, as long as you work with vhdl or verilog. However, I am currently extending a project written in SystemVerilog, which is why I would like to get verilator up to speed.
Well, it is not really a limitation by design, but rather the devil that comes with the fact that verilator is extending the VPI infrastructure. This should eventually become compatible again.
However, I am not sure if verilator is really to blame here, since the latest (v1.6.1) cocotb basic examples seem to work with verilator v4.106. However, it is difficult for me to debug the issue, since this co-routine based code seems to be stuck in an infinity loop, but I do not understand the cause yet. @alexforencich Please let me know if you have any insights or clues that I can help with.
I'm not sure what the problem is here. But I'm also not going to do anything with verilator until verilator/verilator#2778 is resolved so that cocotb will work correctly with current versions of verilator.
Yeah sure I can understand that. Thanks for your clarification. Let's keep this open and come back as soon as verilator has been fixed.
BTW: cocotbext-axi + questasim works well for systemverilog.
I mean, it's also possible that I introduced some sort of bug recently that only affects verilator. So it's probably still worth getting to the bottom of what's going on here. Previously, I think I have tested this on verilator 4.106, and it worked fine. But there have been some changes since then, although I can't think of any change in particular that might have affected verilator specifically.
I think that the problem either exists for a longer time, or it is already broken in verilator 4.106, since I tried different, older version combinations of cocotbext-axi and cocotb, but with the very same result of getting stuck.
@andreaskuster I was also facing this issue, where after I upgraded several libraries the simulation for my previously working project got stuck. I'm using Verilator 4.106, and was using a previous release of cocotb and cocotbext-axi. It turns out the issue was with cocotb-bus, where earlier I was using version 0.1.1 and after the upgrade, it was 0.2.1.
Reverting cocotb-bus to version 0.1.1worked for me. And this works with even the newest cocotb and cocotb-ext versions.
pip3 install cocotb-bus==0.1.1
When a bus version > 0.1.1 is used the reset never gets asserted. I don't know why reset is affected. But hopefully, this solution helps.