Trouble Running Benchmarks
aroelke opened this issue · 25 comments
I'm having some issues running the benchmarks--when I run "make check," all of the benchmarks fail with execution scores of 0. Going into the BEEBS log, I see that they never ran because they are expecting to connect to a GDB server. Assuming I have compiled a Verilator emulator of my design, how do I start a GDB server with it? From what I can see in the Rocket Chip debug guide, I have to run a specific program on the emulator to debug with GDB, but that doesn't seem like what embench is trying to do.
This is an area where Embench needs improving. It still uses DejaGnu to run the benchmarks. The Expect script controlling this for ri5cy Verilator model is in testsuite/config/ri5cyverilator.exp. You will see that it attempts to connect to a GDBserver via stdin.
You should replace this by something appropriate for your target. For example the command to fire up the rocket gdbserver, or to run the emulator with GDB enabled.
Hope this gives you some pointers to get it working. If you want to supply a pull request for the changes to support Rocket as a board that would be appreciated.
Okay, I'll try that, thanks.
I'm trying to just run the binaries on my Rocket emulator (no DejaGNU yet), but when I try to load aha-mont64 after connecting GDB to it I get "Load failed" when loading section .rodata. Do you know why this might be happening?
@aroelke Sorry - missed the notification on this. It sounds like your emulator is not allowing you to write to that region of memory. If it is genuinely a read-only piece of memory, it may be emulating you not being able to write to it. Are you able to load the .text section?
Is your compiler/linker putting the code/data at the correct address - do you need a different linker script for your emulator?
HTH
I did some digging into this issue, and you're right, that's what was happening. I was able to load the .text section, but not anything else, most likely because all I was doing was passing the linker -Ttext=0x80000000 to put .text at the start of Rocket Chip's memory.
I started playing around with compiling my own bare-metal code, and learned that I need a full linker script to put the text and data sections into the right places and on top of that I need startup code to initialize things like bss and the stack pointer and register a trap handler (and set up .tohost for compatibility with the frontend server). I notice that your build system doesn't include any of that and instead uses libtool to create a static library which is then compiled into the final binary to run. How do you get around needing this initialization code and linker script?
Hi @aroelke
You don't get round it. You can't run any program without loading it into the right location in memory and setting up key registers. How to do that is a property of the target board, not the program.
Usually linker scripts and crt0 are provided for any board. How do you run other programs? You can just reuse the linker script and crt0.
HTH,
Jeremy
Oh, I see. So, since test programs run over riscv-fesvr, which the Rocket Chip emulator uses (and so does spike), should use the startup code and linker script from riscv-tests, I basically just copy those files into config/[arch]/boards/[board]? How do I then use those files? I can add the linker script into config/[arch]/chips/[chip]/chip.cfg and include any headers that are necessary, but I don't see a step in the build process where the actual code files are compiled so I can add crt.S there.
I suggest you define a new board in config/riscv32/boards. Look at the ri5cyverilator subdirectory for an example. In that directory you'll need boardsupport.c and boardsupport.h, for which you can probably just use the code in ri5cyverilator. In addition you can add a file board.cfg when you can use shell commands to define some compiler and/or linker flags. For example
export BOARD_LDFLAGS="-Wl,-T=myscript.ld"
Would pass in the linker script myscript.ld.
There is also BOARD_CFLAGS and BOARD_CPPFLAGS to define board specific C and preprocessor flags.
I'm not convinced your compiler/linker is not automatically picking up crt0.o. It is more likely that it doesn't match what you need. IIRC the standard Newlib/libgloss assumes you are running programs as a Linux process, not as bare metal.
The current DejaGnu scripts assume you are connecting to a GDB server to load and run programs. If you want to use Spike, then you'll need to write a version of testsuite/config/ri5cyverilator.exp for your board which either fires up Spike and OpenOCD, or invokes Spike directly instead of using GDB.
One of the areas where we need to improve Embench is to replace autotools and DejaGnu with Python scripts to build and run the benchmarks. We really don't need all the features of a POSIX 1003.3 test environment to build and run them.
HTH
Jeremy
Oh, I had been putting those into chip.cfg; I didn't realize there's a board.cfg option as well.
I've been passing GCC -nostartfiles and -nostdlib to make sure it doesn't use its own crt0.o, actually, so I can use my own (or, rather, crt.S copied from riscv-tests). The problem is that it isn't compiling my copied crt.S, so when it goes to link it warns that there's no _start symbol and doesn't set a start address. I can't load this using either GDB or riscv-fesvr, since there isn't a valid entry point. It sounds like I would want to use boardsupport.c and boardsupport.h somehow to handle this, but I'm not sure how to use them to include other code files (i.e. not headers). When you did this with RI5CY, how did you handle initialization and other things crt0.o is supposed to do?
I've been able to connect GDB to spike before with other code, but I haven't gotten to that point here so I haven't tried messing with the DejaGNU scripts yet.
Usually _start is the entry point in crt0.S. For execution you should just use your standard libraries - using -nostartfiles and -nostdlib seems like a mistake to me. I just use the standard crt0.o and libraries when running for RI5CY.
One of the reasons we use GDB is that the timing points are not the start and end of the program, but the start_trigger and stop_trigger functions. This allows us to run the code a bit to warm up caches etc first.
For completeness, you can also have a file arch.cfg in the config/riscv32 directory to set architecture specific flags (in ARCH_CFLAGS etc. They are applied in the order architecture, then chip, then board. So far we have not defined arch.cfg for RV32.
The problem with using Newlib's crt0.o is that, as you said, it expects to be running on top of a kernel which has already initialized the processor. This causes it to not set up the stack pointer, so when it calls its first function it subtracts from it, causing it to point to an illegal address, and then triggers a fault when it tries to save registers (which also doesn't have a handler set up). Are there configuration flags I should be setting when I build my toolchain that enable this? The only other way I know to do it is to use my own startup script, but it sounds like that's not possible here.
Sorry - missed this one. You should be able to use your own startup script- just specify it using the various FLAGS options.
This should soon become obsolete. We don't need the complexity of autotools for compiling a benchmark suite, so it will be replaced by a python script in the near future. Similarly we don't need the complexity of DejaGnu to run the tests (which restricts us to Linux only) and that will be replaced by python scripts.
I have work in progress in my own fork of this directory. Have a look at https://github.com/jeremybennett/embench-iot and the py-build branch. The scripts currently named build-all.py and benchmark-size.py will respectively build the benchmarks and report on the size statistic. Still under development (notably documentation), but ./build-all.py --help will tell you the options and you should be able to specify your own libraries (including startup binary) for linking. Once I'm satisfied it's working well, I'll merge into this repo.
@aroelke I have updated the documentation, which I hope makes your life easier. Thanks for your contribution and please raise other issues if you continue to encounter issues.
Best wishes,
Jeremy
Hi Jeremy,
can you please help me? I am running following command
./build_all.py --arch riscv32 --chip generic --board ri5cyverilator --cc riscv64-unknown-elf-gcc --clean
and after this I got the error attached in the file. What should I do now?
@Prince-Joy can you look in the log file and post the detail here.
@jeremybennett thanks for your response, I have attached the log.
@Prince-Joy. OK, I see the problem - I made this mistake myself earlier. You need to ensure the CFLAGS include '-c', so that the compilation generates an object file for subsequent linking. Increasingly we realize that the options are better set on the command line, rather than in the config files. It is more flexible.
Here is how I run a build, where I am interested in execution speed:
./build_all.py --arch riscv32 --chip generic --board ri5cyverilator --cc riscv32-unknown-elf-gcc --cflags="-c -march=rv32imc -mabi=ilp32 -O2 -ffunction-sections -fdata-sections" --ldflags="-Wl,-gc-sections" --user-libs="-lm" --clean
And here i show I run a build with dummy libraries to measure code size.
./build_all.py --arch riscv32 --chip generic --board ri5cyverilator --cc riscv32-unknown-elf-gcc --cflags="-c -march=rv32imc -mabi=ilp32 -Os -msave-restore -ffunction-sections" --ldflags="-nostartfiles -nostdlib -Wl,-gc-sections" --dummy-libs="crt0 libc libgcc libm" --clean
This was discussed at yesterday's monthly call. One of the goals for Embench 0.6 is to make configuration much simpler.
@jeremybennett I have solved that issue, command was :
./build_all.py --clean --arch riscv32 --chip generic --board ri5cyverilator --cc riscv64-unknown-elf-gcc --cflags="-c -march=rv32i -static -std=gnu99 -mabi=ilp32 -O2 -ffunction-sections -fdata-sections" --ld riscv64-unknown-elf-ld --ldflags="-march=rv32i -mabi=ilp32 -m elf32lriscv" --dummy-libs="crt0 libc libgcc libm"
Now i want to build with custom linker.ld(memory region definitions) and start.s(initialize register) file please help me about this.
It is just standard command options. You can specify libraries with --user-libs and you can add additional object files in --ldflags. You can put custom code for your own board if you wish.
Dear @jeremybennett Sir,
I think I didn't get it.
Actually I want to define my start PC value and RAM, ROM information and also want to initialize all the registers through loading an assembly file (start.s)
something like attached dump file (program_dump.txt).
How can i do this, actually I have very little knowledge about python so its difficult for me to find out.
Remarks: I have tried --user-libs="-lm" but linker doesn't get it only --dummy-libs works.
@Prince-Joy Use --ldflags to specify anything you want on the start of the link command line and --user-libs to specify anything you want on the end of the link line. You should be able to construct any link command in this way.
If you have custom code to be compiled assembled, create a new configuration, put the code there, and specify it on the command line. For example copy config/riscv32/boards/ri5cyverilator to a new directory, such as config/riscv32/boards/prince-joy, put start.s` in there alongside the existing files. and modify your build line to specify this as the board file:
./build_all.py --clean --arch riscv32 --chip generic --board prince-joy --cc riscv64-unknown-elf-gcc --cflags="-c -march=rv32i -static -std=gnu99 -mabi=ilp32 -O2 -ffunction-sections -fdata-sections" --ld riscv64-unknown-elf-ld --ldflags="-march=rv32i -mabi=ilp32 -m elf32lriscv" --dummy-libs="crt0 libc libgcc libm
The script should find the file, compile it and add it to the linker line.
HTH
Thanks @jeremybennett sir, It helped me a lot.
Hi,
Am not able to run the embench on Riscv64, Is it possible to run on Riscv64, Could you help me to come out of this issue
@indumathiraju I see you also asked this on #78. I've answered there.