Using xfuzz to assess hardware coverage but there is no output.

[Codemaker-1]

Hi, there!

I am using xfuzz and trying to build a fuzzer for FIRRTL coverage detection with the following command: make emu XFUZZ=1 REF=$SPIKE_HOME/difftest/build/riscv64-spike-so FIRRTL_COVER=mux,control,line,toggle,ready_valid -j16.

After successfully building, I run ./build/fuzzer -f -c firrtl.mux --max-runs 100 --corpus-input ./riscv-arch-test/riscv-test-suite/build -- --max-cycles 10000 but there is no output related to coverage, and it seems no binary is executed. The entire log is as follows:

The image is wim@0x55a76a99d010+0x97c8
[Objective #0] run time: 0h-0m-0s, clients: 1, corpus: 0, objectives: 1, executions: 0, exec/sec: 0.000

Could you provide more information on how to use xfuzz correctly? If I missed any details, please let me know. Thank you very much!

My environment is as follows:

OS: Ubuntu 18.04
gcc: 7.5.0
verilator: v5.024
commit (xfuzz): 8e14b3
commit (rocket): 82c0f3
commit (difftest): 57742e

[poemonsense]

I just tested the commands again and confirmed they are working perfectly as follows. They are expected to look similar to your outputs.

In stdout:

The image is wim@0x5c6395c2a550+0x3866
Core 0: ^[[31mUnknown trap code: 7
^[[35minstrCnt = 3, cycleCnt = 123, IPC = 0.024390
^[[0mSimMemory: img_size 1804, req_all 8, req_in_range 8
^[[32mCOVERAGE: toggle, 13990, 2030, 2030
^[[0m^[[32mCOVERAGE: mux, 2772, 498, 498
^[[0m^[[32mCOVERAGE: ready_valid, 473, 292, 292
^[[0m^[[32mCOVERAGE: control, 1679, 239, 239
^[[0m^[[32mCOVERAGE: line, 2244, 606, 606
^[[0m^[[32mExitCode: 3
^[[0mTotal Coverage:       17.965%
[Stats #0] run time: 0h-0m-0s, clients: 1, corpus: 0, objectives: 0, executions: 0, exec/sec: 0.000
[Testcase #0] run time: 0h-0m-0s, clients: 1, corpus: 1, objectives: 0, executions: 1, exec/sec: 0.000
The image is wim@0x5c6395c89c80+0xba7
Core 0: ^[[31mUnknown trap code: 7
^[[35minstrCnt = 3, cycleCnt = 98, IPC = 0.030612
^[[0mSimMemory: img_size 372, req_all 8, req_in_range 8
^[[32mCOVERAGE: toggle, 13990, 2166, 2274
^[[0m^[[32mCOVERAGE: mux, 2772, 510, 535
^[[0m^[[32mCOVERAGE: ready_valid, 473, 291, 292
^[[0m^[[32mCOVERAGE: control, 1679, 234, 242
^[[0m^[[32mCOVERAGE: line, 2244, 631, 661
^[[0m^[[32mExitCode: 3
^[[0mTotal Coverage:       19.300%
...... // repeated

In stderr:

thread '<unnamed>' panicked at src/harness.rs:128:9:
Exit due to max_runs == 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The README may be missing some explanation words on the relationship across different submodules. There are some possible reasons. Basically the failure may be due to some missing components of the system

1. Did you build rocket-chip right with FIRRTL-instrumented coverage? It will output some warnings like:

WARN: [RoundAnyRawFNToRecFN_6] found no clock input, skipping ...
WARN: [RoundRawFNToRecFN_3] found no clock input, skipping ...
WARN: [CSRFile] found more than one clock, picking the first one: clock, io.ungated_clock
WARN: [CSRFile] found more than one clock, picking the first one: clock, io_ungated_clock
WARN: Does not find the Input:ClockType PIN in module MulAddRecFNToRaw_preMul. Use global inputs instead.

2. Did you build the spike-so right? You can check it by determining whether the file $SPIKE_HOME/difftest/build/riscv64-spike-so exists. Besides, since LLVM_COVER=1 is not set when make emu, your spike-so is assumed to be built without SANCOV=1. This information is missing in the README, and I'm fixing it now.

3. Did you check the output stderr? Did you try using gdb to debug the emu?

[Codemaker-1]

Thank you very much for your detailed response.

Based on your logs and the revised README file, I have successfully identified my issue.

Thanks again, and I will close this issue now.