This repository contains a collection of benchmarks and seed inputs to make fuzzing research more readily comparable and reproducable. For instructions on installing and running a given benchmark, refer to the readme file in its respective directory.
We welcome any suggestions for improving this fuzzing benchmark corpus! Our only criteria for additional benchmarks is that they are compatible with AFL-Clang, AFL-QEMU, and AFL-Dyninst tracing (as all 8 below are).
Presented in our paper Full-speed Fuzzing: Reducing Fuzzing Overhead through Coverage-guided Tracing (to appear in the 2019 IEEE Symposium on Security and Privacy). |
|
| Citing this repository: |
@inproceedings{nagy:fullspeedfuzzing,title = {Full-speed Fuzzing: Reducing Fuzzing Overhead through Coverage-guided Tracing},author = {Stefan Nagy and Matthew Hicks},booktitle = {{IEEE} Symposium on Security and Privacy (Oakland)},year = {2019},}
|
| Developers: | Stefan Nagy (snagy2@vt.edu) and Matthew Hicks (mdhicks2@vt.edu) |
| License: | MIT License |
| Disclaimer: | This software is strictly a research prototype. |
We utilized Dyninst to compute the following bechmark statistics for the 8 binaries evaluated in our paper. Note that we compiled all with Clang/Clang++.
| benchname | libname | type | basic blocks | basic block edges |
|---|---|---|---|---|
| bsdtar | libarchive | archiv | 31379 | 43390 |
| cert-basic | libksba | crypto | 9958 | 14120 |
| cjson | cjson | web | 1447 | 2038 |
| djpeg | libjpeg | img | 4844 | 6776 |
| pdftohtml | poppler | doc | 54596 | 71945 |
| readelf | binutils | dev | 21249 | 31086 |
| sfconvert | audiofile | audio | 5603 | 7403 |
| tcpdump | tcpdump | net | 33743 | 48791 |
For our paper, we collected statistics on the 24hr fuzzing testcase corpora for each benchmark. Note that these numbers reflect corpora generated using AFL with QEMU-based tracing.
| benchname | libname | type | 24hr corpus size | testcases/24hr | 100ms timeouts |
|---|---|---|---|---|---|
| bsdtar | libarchive | archiv | 90.9G | 25.6M | 4 |
| cert-basic | libksba | crypto | 7.5G | 10.7M | 6 |
| cjson | cjson | web | 4.5G | 14.5M | 221K |
| djpeg | libjpeg | img | 30.1G | 21.0M | 656 |
| pdftohtml | poppler | doc | 0.2G | 1.2M | 107 |
| readelf | binutils | dev | 3.8G | 14.9M | 7 |
| sfconvert | audiofile | audio | 3.7G | 10.1M | 373K |
| tcpdump | tcpdump | net | 2.7G | 27.1M | 5 |
Before you attempt to build the benchmarks, there are several dependencies that you must resolve:
sudo apt install build-essential clang libgpg-error-dev libfontconfig1-dev libpcap-dev
We provide the script buildAll.sh to compile all benchmarks from source.
Edit the following parameters to reflect the desired C and C++ compilers, and any assembler parameters:
compiler=""
compilerXX=""
passToAS=""
Before building using AFL-modified compilers, make sure to upate PATH and AFL-PATH system variables to point to where you have AFL installed.
We also provide the script collectAll.py to copy all compiled benchmark binaries to the current directory and append them with a use-specific (as specified on the command line) postfix:
Run as follows:
python /path/to/FoRTE-FuzzBench/collectAll.py [binaryPostfix]
All benchmarks will be copied to the current directory and appended the specified postfix (leave blank if none).
