rohanpadhye/JQF

Percentage of coverage and data from corpuses

manobaneru opened this issue · 5 comments

manobaneru commented

Good evening! During my study of the tool and fuzzing in general, I had two questions.

  1. I am running a program using Maven:
    mvn jqf:fuzz -Dclass=ClassName -Dmethod=MethodName
    Zest displays the Total coverage and Valid coverage parameters, and I'm not quite sure what the map parameter is? I add a .jar file to pom.xml to invoke the class method from it, but what map percentage do I get displayed? The percentage covered from the whole file, the class in question, or something else?

  2. A corpus of data is generated during the JQF+Zest run. Can I realistically use them in the future? By using the jqf-afl-fuzz command? Or am I misunderstanding the concept of the workflow?

Thanks for your answers!

rohanpadhye commented

Thanks for your interest in JQF! I agree we need better documentation or actually a better status screen.

The "Map" is the hashmap used to store coverage. So higher % of map covered means higher coverage. For efficiency reasons, we don't always resolve hash collisions, so the idea is that if the % coverage is very high (more than 40-50%), it would be better to increase the size of the hash map for better accuracy at the expense of lower performance. If the map coverage % is lower than 10-20%, I would just ignore it.

The corpus can be used for re-running tests using mvn jqf:repro (see docs) or by using them as seeds for another round of fuzzing. The files themselves are not human readable; they only ensure that the same corresponding inputs will be produced assuming the test method has the same signature and that the generator code hasn't changed.

rohanpadhye commented

The mvn jqf:repro workflow is very important, because it allows you to re-run tests for debugging if you find a failure, or if you just want to see what behavior your program is exercising. We often use the repro runs along with a pretty-coverage tool like JaCoCo to generate HTML reports of coverage.

manobaneru commented

Thank you for your detailed response and help in researching the issue!

enzocardeal commented

Hi, I hope you are doing well.

How does this % of map compares to the code coverage percentage I get from intelliJ? I'm asking because I'm using as seeds the same inputs I use in my Junit tests, but the % code coverage I get from intelliJ is quite different from the % of map coverage.

rohanpadhye commented