pangine/disasm-benchmark

ddisasm results?

jrmuizel opened this issue · 4 comments

The README.md suggests that ddisasm is supported but the results for ddisasm are not included in the paper. Are ddisasm results available someplace else?

As specified in our ddisasm build Dockerfile, we tried to disassemble our dataset with the ddisasm version 4a8ae6a commited at Jun 1st, 2020.

Unfortunately, by executing ddisasm BIN --debug --asm BIN.asm, ddisasm failed to finish on multiple binaries.
For example, when running it on x64 vim compiled with ICC -O0, the error message was:

Printing assembler ddisasm: /root/gtrib-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp:166: void gtirb_pprint::ElfPrettyPrinter::printSymbolHeader(std::ostream&, const gtirb::Symbol&): Assertion `!"unknown visibility in elfSymbolInfo!"' failed.
  ddisasm execution failed
failed

When running it on x64 exim compiled with ICC -O2, the error message was:

The conflicts between the following code blocks could not be resolved:
4f0bac - 4f0bb2
4f0bb3 - 4f0bb6
4f0d13 - 4f0d20
Aborting

Since many binaries cannot be disassembled by the version of ddisasm we tested (almost 1/3 of the binaries compiled by ICC + several other binaries compiled by GCC and Clang), and the version of ddisasm did not support Windows binaries anyway, we believed that it was not yet approparate to include ddisasm in our evaluation.

If you are interested in the ddisasm results, you may follow our instructions and run only ddisasm on our released dataset by following our instructions in the README.

Do note that if ddisasm returns an error code after complaining that The conflicts between the following code blocks could not be resolved, then it will still produce a valid GTIRB file. So what you could do in the meantime is have it produce GTIRB via the --ir option, and have gtirb-pprinter produce your assembly file.

In addition, as of a recent new release, ddisasm now supports WIndows binaries.

I tried both exim and vim on ICC and they both now disassemble without errors and seem to work after reassembly. Perhaps it's worth reevaluating ddisasm now?

pangine/disasm-eval-disasms#1 fixes ddisasm integration. I should have results on ddisasm soon.