VUnit/vunit

Question: Pass/Fail criteria checked in Python Script

bryankerr1986 opened this issue · 3 comments

bryankerr1986 commented

What if I want to a Python Script controlling the Simulation.

For example, what if I want to compare results of two .txt files produced by TB in python and use that knowledge to determine if the testbench passed. Can I do this?

Or what if I want to use Python to control the run of the testbench. I.E. Loop through the same testbench two times with different parameters. Can I do this?

LarsAsplund commented

Yes, this is possible. If you read the latest blog https://vunit.github.io/blog/2023_08_26_vhdl_configurations.html you will find information about VUnit configurations which is the tool to run the same test with different generics/parameters. If you look at the documentation for add_config you will also find an option to have a post_check Python function executed after the test exits simulation. That function can compare the text files. If the function returns False, the test will fail.

bryankerr1986 commented

Thank you, this was very helpful. I was able to accomplish both of these tasks with ease using VUnit.

Follow up question...
If I changed testbench parameters 30 times to run 30 different senarios, then I now have 30 separate testbenches under 'python3 run.py --list'. Is it possible to log a comment about each scenario in the python run script so when I type 'python3 run.py --list', then all 30 scenarios will be printed with a short description of what the testbench does? If this is possible, is it also possible to save this in html, or some other, format?

LarsAsplund commented

What I usually do is to insert the parameter(s) in the name of the configuration. Something like:

for parameter in range(30):
    testbench.add_config(name=f"parameter={parameter}", generics=dict(parameter=parameter))

You can also use the attributes parameter to add_config. For example attributes={".description", "This is a description for my configuration"}. See also https://vunit.github.io/py/ui.html#attributes

The attributes for all tests can be exported with the --export-json option. That JSON file can then be parsed to build a HTML page for example.