Design Your Test as Code: The JCUnit 0.10.x

is an open source model-based testing (MBT) framework powered by combinatorial interaction testing (CIT) technique. It supports various models to provide a uniformed testing style. It employs CIT to balance the coverage and test suite size. The documentation is found here.

Maven Coordinate for JCUnit

<dependency>
    <groupId>com.github.dakusui</groupId>
    <artifactId>jcunit</artifactId>
    <version>{jcunit-version}</version>
</dependency>

JCUnit 0.10.x is under development. The most recent stable major version of JCUnit is 0.8.x. Please check

JCUnit 0.10.x: Key Features to Come

0.10.x will be the last version line previous to 1.0.x, which is intended for the "true" public usage and broader audience.

The project board is found here.

Enhanced Regex Support: (#152, #155)
JUnit5: support (#142)
Temporary removal of FSM support: (It will be brought back in 1.0.x, #169).
PICT integration: (#179)
Simplified Internal Structure for integrating external CIT tools: (Stretch goal, #175).
Fault Localizer: (Stretch goal, #144)

Limitations

Issue-185: JCUnit test fails with 4.13.1: This is coming from a design change introduced in JUnit’s side. A workaround is described in the ticket.
Issue-125: Provide a way to execute a test method for a test case or all the generated test cases: If you have multiple test methods in your test class from an IDE such as IntelliJ, you cannot run only one of them. JCUnit has a helper class to work around the situation.

References

PICT

PICT is the most popular tool for Combinatorial Interaction Testing (CIT). It can generate a covering array from a text file described in its own human-readable notation. It was developed by Jacek Czerwonka and now an open source project from Microsoft.

JCUnit 0.10.x (or later) uses PICT for covering array generation for the part (factors) it can be used and then combine it with another covering array generated by JCUnit’s internal covering array generator for the rest (#179).

"Pairwise Testing in Real World", J Czerwonka, 24th Pacific Northwest Software Quality Conference, 2006
PICT github project, Microsoft

JCUnit

We have published some key ideas of JCUnit as academic papers and a blog.

Publications

Test Design as Code: JCUnit; A paper on this product; Presented at IEEE International Conference on Software Testing 2017
Reduce Test Cost by Reusing Test Oracles through Combinatorial Join; A paper about a technique this product utilizes for building a covering array; Presented at 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)
Accelerating covering array generation by combinatorial join for industry scale software testing; A paper about a technique to construct a covering array from existing ones;
JCUnit blog; Japanese; 日本語

JCUnit Project

Quick Start Guide for JCUnit (t.b.d.)
JCUnit 0.10.x Project Board

Build Instructions

The JCUnit requires Java SE 8 to build. Please don’t forget installing and using it for building. This is intentional decision to make the testing compatible for as many product as possible.

Build

$ mvn clean compile

Conduct Unit Test

$ mvn clean compile test

Generating Site

$ mvn clean package site

Generates the site under target/site in the following structure.

target/
  site/
    apidocs/     - JavaDoc
    jacoco/      - JaCoCo report (code coverage)
    pit-reports/ - PIT report (mutation coverage)

The mutation testing is configured to cover only small amount of classes to save build time. To configure the coverage, refer to [mutationTesting].

Deploying Site

$ mvn clean package site

Misc

Conduct Mutation Test

$ mvn clean compile test org.pitest:pitest-maven:mutationCoverage

Currently, this may take very long time (≥ 2 hours), if you include all the classes of this product. By default, the pom.xml is configured to exercise the mutation testing over small number of classes so that you customize the intended coverage.

Under the plugin configuration for org.pitest:pitest-maven, you can find a following element.

pom.xml configuration for org.pitest:pitest-maven plugin

    <targetClasses>
        <param>com.github.dakusui.jcunit8.exceptions.*</param>
    </targetClasses>

To cover all the JCUnit classes, you can modify it like following.

pom.xml configuration for org.pitest:pitest-maven plugin

    <targetClasses>
        <param>com.github.dakusui.*</param>
    </targetClasses>

After a successful execution, it generates a pitest report under a directory target/pit-reports/{yyyyMMddHHmmss}.

Build Javadoc

$ mvn clean javadoc:javadoc

This generates JavaDoc under target/site/apidocs.

Build the Site

$ mvn clean package site

Publish the Site

$ mvn clean package site site-deploy

Copyright and license

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this work except in compliance with the License. You may obtain a copy of the License in the LICENSE file, or at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

792401/jcunit