K-Path Coverage Evaluation

This is a replication package for the paper "Systematically Covering Input Structure" published at ASE 2019 (preprint).

You can download the entire dataset as reported in the paper from Google Drive (574MB download, 16GB uncompressed).

You can also run the experiments yourself:

Prerequisites

You need Python >= 3.6 and Java >= 1.8.

Clone this repo with submodules (namely the input generator tribble and the subjects):

git clone --recurse-submodules https://github.com/havrikov/covering-input-structure.git

The rest of these instructions assume you are in the cloned directory:

cd covering-input-structure

Install python dependencies:

pip3 install -r requirements.txt

Edit luigi.cfg to set the amount of RAM available and the experiment directory.

Start the luigi daemon (installed as part of the prerequisites).

luigid --background --pidfile tools/luigi/pid --logdir tools/luigi/logs --state-path tools/luigi/state

Run the experiments. (Substitute the number of CPUs below)

python3 ./experiments.py --k-params "[1,2,3,5]" --runs 50 --workers <number-of-CPUs>

Navigate to http://localhost:8082 to monitor the progress.

NOTE The process will take a long time for a high number of runs, so preferably launch it on some sort of compute server.

After the experiment pipeline has finished, the results will be available as report-50-runs.html in your experiment directory.