@Fuzz tests don't run on `mvn test` on JUnit5

Question

@Fuzz tests don't run on `mvn test` on JUnit5

robbiesumner opened this issue a year ago · 8 comments

I wrote fuzz tests and they run great with the plugin (mvn jqf:fuzz <flags>). However, when I run the tests as part of JUnit, they succeed, but do not run one single time:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

In the documentation, it states that tests included in the JUnit suite would be run with 100 random inputs. What am I doing wrong?

Answer 1 · 2023-11-29T11:59:45.000Z

there was another issue in my code base

Answer 2 · 2023-11-29T14:45:53.000Z

The fuzz tests are not run on mvn test, but if i call them directly with java they run. Anyone know how I can make them run on mvn test?

Answer 3 · 2023-11-29T16:00:31.000Z

Hi. JQF @Fuzz tests should run by default with mvn test, unless you have some other configuration in your pom.xml file that changes the configuration for the surefire plugin. See the example repository here: https://github.com/rohanpadhye/jqf-zest-example and run mvn test. You should see the PatriciaTrieTest run (although you might get an exception about assumption not being met enough number of times, which is expected). You will have to identify what is the difference in configuration between that example project and your project. The number of possibilities are too large for anyone else to debug unless you can provide a minimal repository where this fails.

Answer 4 · 2023-11-29T16:06:46.000Z

Thank you, that is really helpful, I will look into the differences and figure out why this does not run

Answer 5 · 2023-11-30T08:25:25.000Z

JQF extends Quickcheck which extends the JUnit4 runner. My project uses JUnit5. Could that be the issue at hand?

Answer 6 · 2023-11-30T08:56:44.000Z

Is there a way to make JQF tests compatible with a JUnit5 (jupiter) runner?

Answer 7 · 2023-11-30T18:11:51.000Z

It's possible that is the issue. You may be able to configure your surefire plugin to run JUnit4 tests by default with mvn test.

I don't have a JUnit5 runner for JQF at the moment, because that would require modifications to the underlying junit-quickcheck library. I'm happy to accept contributions if anyone wants to improve compatibility.

Answer 8 · 2023-11-30T19:18:18.000Z

awesome, thank you, I'll think of looking into that and maybe contributing