rohanpadhye/JQF

Wrong Classpath for the ASM Version Update

cc-21 opened this issue · 1 comments

cc-21 commented

Hello,

The recent update of the ASM version leads to zero coverage when running jqf-zest. In order to fix the issue, the classpath declared in scripts/jqf-driver.sh (line 22) needs to be changed to version 9.4 instead of 9.1:
INST_CLASSPATH="${INST_DIR}/classes:${INST_JAR}:${INST_DIR}/dependency/asm-9.4.jar"

Steps to Reproduce:
Follow the tutorial to run JQF on MacOS.

cd $JQF_DIR
mvn install
# go to the folder of programs under test and compile the java files
javac -cp .:$($JQF_DIR/scripts/classpath.sh) CalendarLogic.java CalendarGenerator.java CalendarTest.java
$JQF_DIR/bin/jqf-zest -c .:$($JQF_DIR/scripts/classpath.sh) CalendarTest testLeapYear

The instrumentation does not work properly so that running JQF will not generate coverage information at all (i.e., Total coverage: 0 branches (0.00% of map) ):

Semantic Fuzzing with Zest
--------------------------

Test name:            CalendarTest#testLeapYear
Instrumentation:      Janala
Results directory:    /Users/madonnahuang/master_projects/fall2022/research/zest-tut/fuzz-results
Elapsed time:         1s (no time limit)
Number of executions: 85,058 (no trial limit)
Valid inputs:         59 (0.07%)
Cycles completed:     0
Unique failures:      0
Queue size:           0 (0 favored last cycle)
Current parent input: <seed>
Execution speed:      95,733/sec now | 47,045/sec overall
Total coverage:       0 branches (0.00% of map)
Valid coverage:       0 branches (0.00% of map)
Fuzzing stopped due to guidance exception: Too many trials without coverage; likely all assumption violations
E
Time: 1.698
There was 1 failure:
1) testLeapYear(CalendarTest)
edu.berkeley.cs.jqf.fuzz.guidance.GuidanceException: Too many trials without coverage; likely all assumption violations
	at edu.berkeley.cs.jqf.fuzz.ei.ZestGuidance.lambda$getInput$0(ZestGuidance.java:666)
	at edu.berkeley.cs.jqf.fuzz.ei.ZestGuidance.conditionallySynchronize(ZestGuidance.java:1054)
	at edu.berkeley.cs.jqf.fuzz.ei.ZestGuidance.getInput(ZestGuidance.java:652)
	at edu.berkeley.cs.jqf.fuzz.junit.quickcheck.FuzzStatement.evaluate(FuzzStatement.java:117)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at edu.berkeley.cs.jqf.fuzz.junit.GuidedFuzzing.run(GuidedFuzzing.java:213)
	at edu.berkeley.cs.jqf.fuzz.junit.GuidedFuzzing.run(GuidedFuzzing.java:159)
	at edu.berkeley.cs.jqf.fuzz.junit.GuidedFuzzing.run(GuidedFuzzing.java:123)
	at edu.berkeley.cs.jqf.fuzz.ei.ZestDriver.main(ZestDriver.java:77)

FAILURES!!!
Tests run: 1,  Failures: 1

The coverage information will load normally after fixing the ASM version specified in the classpath.

Thank you for your time.

rohanpadhye commented

Thanks for the report. Yes, this is just a matter of bumping the classpath. Most users just run the Maven plugin mvn jqf:fuzz, which does not require setting the classpath explicitly, so this fell under the radar.

  • 👍1