Does Pitest work with the junit-quickcheck property test tool?
Opened this issue · 2 comments
ricardosm commented
I am trying to use the junit-quickcheck property testing tool together with Pirtest 1.6.4. However, I am encountering a problem. Pitest doesn't seem to be able to find the tests in my test class.
The message "PIT >> INFO: Found 0 tests" is always returned.
I believe that this problem occurs because the junit-quickcheck tool requires @RunWith (JUnitQuickcheck.class).
Does anyone have a solution to this problem?
Below is my code, test code and pom.xml:
package sort_selection_sort;
public class SelectionSort {
public int[] sort(int arr[]) {
int n = arr.length;
// One by one move boundary of unsorted subarray
for (int i = 0; i < n - 1; i++) {
// Find the minimum element in unsorted array
int min_idx = i;
for (int j = i + 1; j < n; j++)
if (arr[j] < arr[min_idx])
min_idx = j;
// Swap the found minimum element with the first
// element
int temp = arr[min_idx];
arr[min_idx] = arr[i];
arr[i] = temp;
}
return arr;
}
}
package sort_selection_sort;
import static org.junit.Assert.assertTrue;
import java.util.Arrays;
import org.junit.runner.RunWith;
import com.pholser.junit.quickcheck.Property;
import com.pholser.junit.quickcheck.runner.JUnitQuickcheck;
@RunWith(JUnitQuickcheck.class)
public class SelectionSortPropertiesTest {
@Property(trials = 10)
public void property(int arr[]) {
SelectionSort ob = new SelectionSort();
int[] copyOfArr = Arrays.copyOf(arr, arr.length);
Arrays.sort(copyOfArr);
int[] sort = ob.sort(arr);
assertTrue(Arrays.equals(copyOfArr, sort));
}
}
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
</plugin>
<!-- Configuration of Maven Surefire Plugin for support to jUnit 5 (Unit
tests) -->
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
</plugin>
<!-- Configuration of Pitest and Pitest jUnit 5 plugin -->
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.6.4</version>
<configuration>
<targetClasses>
<param>sort_selection_sort.*</param>
</targetClasses>
<targetTests>
<param>sort_selection_sort.*</param>
</targetTests>
<mutators>
<mutator>ALL</mutator>
</mutators>
<verbose>true</verbose>
</configuration>
</plugin>
</plugins>
</build>
<!-- ... -->
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.pholser</groupId>
<artifactId>junit-quickcheck-core</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.pholser</groupId>
<artifactId>junit-quickcheck-generators</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.pholser</groupId>
<artifactId>junit-quickcheck-guava</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.30</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.30</version>
</dependency>
<!-- -->
</dependencies>
16:55:16 PIT >> FINE : Maximum available memory is 3612 mb
16:55:16 PIT >> FINE : MINION : Installing PIT agent
16:55:16 PIT >> INFO : Sending 2 test classes to minion
16:55:16 PIT >> INFO : Sent tests to minion
16:55:17 PIT >> INFO : MINION : 16:55:16 PIT >> FINE : Expecting 2 tests classes from parent
16:55:17 PIT >> INFO : MINION : 16:55:16 PIT >> FINE : Tests classes received
16:55:17 PIT >> INFO : MINION : 16:55:16 PIT >> INFO : Checking environment
16:55:17 PIT >> INFO : MINION : 16:55:17 PIT >> INFO : Found 0 tests
16:55:17 PIT >> INFO : MINION : 16:55:17 PIT >> INFO : Dependency analysis reduced number of potential tests by 0
16:55:17 PIT >> INFO : MINION : 16:55:17 PIT >> INFO : 0 tests received
16:55:17 PIT >> INFO : MINION : 16:55:17 PIT >> FINE : Running 0 units
16:55:17 PIT >> FINE : Coverage generator Minion exited ok
16:55:17 PIT >> INFO : Calculated coverage in 0 seconds.
16:55:17 PIT >> FINE : Used memory after coverage calculation 23 mb
16:55:17 PIT >> FINE : Free Memory after coverage calculation 219 mb
16:55:17 PIT >> FINE : Incremental analysis set 153 mutations to a status of NOT_STARTED
16:55:17 PIT >> INFO : Incremental analysis reduced number of mutations by 0
16:55:17 PIT >> INFO : Created 1 mutation test units
16:55:17 PIT >> FINE : Used memory before analysis start 47 mb
16:55:17 PIT >> FINE : Free Memory before analysis start 136 mb
16:55:17 PIT >> FINE : Running 1 units
16:55:17 PIT >> INFO : Completed in 0 seconds
================================================================================
- Mutators
================================================================================
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOR3Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOR2Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOR1Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOR4Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.RemoveConditionalMutator_ORDER_IF
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.MathMutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR3Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.ConditionalsBoundaryMutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.IncrementsMutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.experimental.RemoveIncrementsMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 1
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR4Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR2Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.NullReturnValsMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 1
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOD1Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.InlineConstantMutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOD2Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 2
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.ReturnValsMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 1
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR6Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR1Mutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 1
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR5Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ABSMutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 18
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.RemoveConditionalMutator_ORDER_ELSE
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR3Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR5Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR1Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR2Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR4Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.UOI1Mutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 18
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.NegateConditionalsMutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 3
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.UOI3Mutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 18
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.UOI2Mutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 18
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.UOI4Mutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 18
--------------------------------------------------------------------------------
================================================================================
- Timings
================================================================================
> scan classpath : < 1 second
> coverage and dependency analysis : < 1 second
> build mutation tests : < 1 second
> run mutation analysis : < 1 second
--------------------------------------------------------------------------------
> Total : < 1 second
--------------------------------------------------------------------------------
================================================================================
- Statistics
================================================================================
>> Generated 153 mutations Killed 0 (0%)
>> Mutations with no coverage 153. Test strength 0%
>> Ran 0 tests (0 tests per mutation)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.596 s
[INFO] Finished at: 2021-04-20T16:55:17-03:00
[INFO] ------------------------------------------------------------------------
pholser commented
@hcoles Any advice you can offer? TBH I haven't tried running junit-quickcheck tests under pitest.
tj-byun commented
Pitest + junit-quickcheck will be super cool..! Strong test generator & oracle meets strong test adequacy criterion.
@ricardosm Did you get it to work yet, by any chance?