Unable to run Benchmarks

[Abraham-Castaneda]

I have compiled all of the Benchmark source files, but when I try to run the size and speed tests, I come across some issues. When running ./benchmark_size.py, I get the following:

Traceback (most recent call last):
File "./benchmark_size.py", line 26, in
import lief
ModuleNotFoundError: No module named 'lief'

When running ./benchmark_speed.py, I get:

benchmark_speed.py: error: the following arguments are required: --target-module

For benchmark_speed, what module should I be targeting exactly? For the record I built the benchmarks with the following supplied arguments:

./build_all.py --arch risc32 --board ri5cyverilator --cc riscv64-unknown-elf-gcc --clfags=-c --user-libs="-lm"

[sobuch]

Hi, I havent used embench in a while and this lief import is more recent change but I would expect you just need to run something like pip install lief.

For benchmarking speed, you want to take a look at the scripts in https://github.com/embench/embench-iot/tree/master/pylib. For example, to run the benchmarks on your native machine, I believe the required option is --target-module="native", the functions instructing the embench core how to run the benchmark code are then in https://github.com/embench/embench-iot/blob/master/pylib/run_native.py.

[Abraham-Castaneda]

Thanks, installing lief with pip seemed to fix the size testing. For speed however, when I set it to use the module "run-native", I get the following results:

Benchmark Speed
aha-mont64
crc32
cubic
edn
huffbench
matmult-int
minver
nbody
nettle-aes
nettle-sha256
nsichneu
picojpeg
qrduino
sglib-combined
slre
st
statemate
ud
wikisort
ERROR: Failed to compute speed benchmarks

Looking into the log, I can see that for every single benchmark, the timing seems to be failing, like so:

**Warning: Failed to find timing
Args to subprocess:
sh '-c' 'time -p ./aha-mont64; echo RET=$?'
RET=126

./aha-mont64: ./aha-mont64: cannot execute binary file
real 0.00
user 0.00
sys 0.00

Warning: Failed to find timing
Args to subprocess:
sh '-c' 'time -p ./crc32; echo RET=$?'
RET=126

./crc32: ./crc32: cannot execute binary file
real 0.00
user 0.00
sys 0.00**

This goes on in the log for the remaining 17 benchmarks. Not sure exactly what the issue with the timing is.

[sobuch]

the problem there is that to run on your native machine you also need to compile the benchmarks for that architecture (probably x86 and not rv32). Running natively is the easier bit and should work right away, running on some other target may be more complicated. Possibly https://github.com/embench/embench-iot/blob/master/pylib/run_gdbserver_sim.py could help you here, but I havent used that yet.

[Abraham-Castaneda]

I gave run_gdbserver_sim a shot and the times are still coming up empty unfortunately. Taking a peek at the logs shows this:

Warning: Failed to find return code
Args to subprocess:
gdb -ex 'set confirm off' -ex 'set style enabled off' -ex 'set height 0' -ex 'file aha-mont64' -ex 'target remote | gdbserver -c ri5cy --stdin' -ex 'stepi' -e
x 'stepi' -ex 'load' -ex 'break start_trigger' -ex 'break stop_trigger' -ex 'break _exit' -ex 'jump *_start' -ex 'monitor cyclecount' -ex 'continue' -ex 'moni
tor cyclecount' -ex 'continue' -ex 'print $a0' -ex 'detach' -ex 'quit'
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Reading symbols from /home/acastaneda/envs/embench-iot/bd/src/aha-mont64/aha-mont64...(no debugging symbols found)...done.
Remote debugging using | gdbserver -c ri5cy --stdin
Breakpoint 1 at 0x104e8
Breakpoint 2 at 0x104ec
Breakpoint 3 at 0x10978
$1 = void

No symbol table is loaded. Use the "file" command.
sh: gdbserver: command not found
Remote communication error. Target disconnected.: Connection reset by peer.
The program is not being run.
The program is not being run.
You can't do that when your target is `exec'
The program is not being run.
"monitor" command not supported by this target.
The program is not being run.
"monitor" command not supported by this target.
The program is not being run.
The program is not being run.

This series of messages just repeats 18 more times for the rest of the benchmarks in the log. It says that the monitor command is not supported by the target, which I take to mean that riscv32 won't work for this module either (please correct me if I'm wrong)? If so, is there a workaround for RISC-V processor testing for embench (i.e. how are others going about it)?

[rawan305]

Hi,

I'm having the same issue while trying to run the speed benchmark.
Did you find a solution? could you run the speed benchmark successfully?

[Abraham-Castaneda]

Hi rawan305,

Unfortunately I have not been able to run it as of yet.