pholser/junit-quickcheck

maven assembly plugin + quickcheck bug

yevgenypats opened this issue · 6 comments

hi @pholser !:) I'm not sure if it's a bug or I'm doing something wrong. I'm trying to use assembly plugin to package the junit-quickcheck tests together with the code so I can run them but I'm getting some strange exception even that I included the generator package:

1) fuzz(dev.fuzzit.examples.ParseComplexTest)
java.lang.IllegalArgumentException: Cannot find generator for dev.fuzzit.examples.ParseComplexTest.fuzz:arg0 of type java.lang.String
        at com.pholser.junit.quickcheck.internal.generator.GeneratorRepository.matchingGenerators(GeneratorRepository.java:256)
        at com.pholser.junit.quickcheck.internal.generator.GeneratorRepository.generatorFor(GeneratorRepository.java:237)
        at com.pholser.junit.quickcheck.internal.generator.GeneratorRepository.produceGenerator(GeneratorRepository.java:220)
        at com.pholser.junit.quickcheck.internal.sampling.TupleParameterSampler.decideGenerator(TupleParameterSampler.java:97)
        at com.pholser.junit.quickcheck.internal.generator.PropertyParameterGenerationContext.<init>(PropertyParameterGenerationContext.java:65)
        at com.pholser.junit.quickcheck.runner.PropertyStatement.lambda$evaluate$50(PropertyStatement.java:101)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at com.pholser.junit.quickcheck.runner.PropertyStatement.evaluate(PropertyStatement.java:108)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        at dev.fuzzit.examples.QuickCheckMain.main(QuickCheckMain.java:20)

FAILURES!!!
Tests run: 1,  Failures: 1

here is the link to the example repository https://github.com/fuzzitdev/quickcheck-assembly

and here are the exact steps to reproduce:

git clone https://github.com/fuzzitdev/quickcheck-assembly
cd quickcheck-assembly
 docker run -v `pwd`:/app -it maven:3.6.1-jdk-12 /bin/bash
cd /app
mvn test # this works
mvn package
java -jar /app/target/quickcheck-1.0-SNAPSHOT-fat-tests.jar # this won't work and will throw the exception

maybe I'm doing something wrong with the initialisation https://github.com/fuzzitdev/quickcheck-assembly/blob/master/src/test/java/dev/fuzzit/examples/QuickCheckMain.java#L13

will appreciate any help!

Ccing also @rohanpadhye as JQF is also affected by the same bug/issue.

@yevgenypats Thanks for this -- I'll investigate.

@pholser maybe I found something that can help investigate. For some reason when calling

java -jar /app/target/quickcheck-1.0-SNAPSHOT-fat-tests.jar this will cause the exception I described.

But when I essentially do the same thing + add junit-quickhceck-generators to the classpath (even that it is in the fat jar anyway) it works!

java -ea -cp junit-quickcheck-generators-0.8.jar:/app/target/quickcheck-1.0-SNAPSHOT-fat-tests.jar dev.fuzzit.examples.QuickCheckMain

Another workaround is to explicitly mention the string generator :

diff --git a/src/test/java/dev/fuzzit/examples/ParseComplexTest.java b/src/test/java/dev/fuzzit/examples/ParseComplexTest.java
index 8af8797..e0043e4 100644
--- a/src/test/java/dev/fuzzit/examples/ParseComplexTest.java
+++ b/src/test/java/dev/fuzzit/examples/ParseComplexTest.java
@@ -4,11 +4,12 @@ import com.pholser.junit.quickcheck.Property;
 import com.pholser.junit.quickcheck.runner.JUnitQuickcheck;
 import org.junit.runner.RunWith;
 +import com.pholser.junit.quickcheck.From;
 
 @RunWith(JUnitQuickcheck.class)
 public class ParseComplexTest {
 
-    @Property public void fuzz(String data) {
+    @Property public void fuzz(@From(com.pholser.junit.quickcheck.generator.java.lang.StringGenerator.class) String data) {
         ParseComplex.parse(data);
     }
 }

In the packaged version, file META-INF/services/com.pholser.junit.quickcheck.generator.Generator from junit-quickcheck-generators got overwritten by the one from junit-quickcheck-core

So, junit-quickcheck-generators should be listed before junit-quickcheck-core in pom.xml dependencies

@yevgenypats @catenacyber It appears that the service loader file for com.pholser.junit.quickcheck.generator.Generator from core is the one that survived the Maven assembly. Ideally, you want one such file in your fat jar, that is the concatenation of any such files from any of the original jars.

I wonder if this feature of the Maven assembly plugin would help: https://maven.apache.org/plugins/maven-assembly-plugin/examples/single/using-container-descriptor-handlers.html

@pholser I think the direction that you suggested is the right one. in the meanwhile I just swapped the order of dependency. The full solution probably would be using the container-descriptor feature. I guess we can close this issue as it's more related to assembly plugin and less to quickcheck. Thanks for helping with that!