Toradocu: automated generation of test oracles from Javadoc documentation

Toradocu generates assertions from the Javadoc documentation of a class.

Overview

Toradocu takes the source code of a class as input and produces a set of specifications in JSON format. Toradocu can translate all kind of specifications at the method level:

preconditions;
normal postconditions;
exceptional postconditions.

The tool combines different translations approaches: pattern match, syntax match and semantic match. All the generated specifications are executable and guaranteed to compile.

Toradocu can also generate aspects that can be used as test oracles.

Learn more

Toradocu is described in the following publications:

Translating code comments to procedure specifications by Arianna Blasi, Alberto Goffi, Konstantin Kuznetsov, Alessandra Gorla, Michael D. Ernst, Mauro Pezzè and Sergio Delgado (presented at ISSTA 2018).

In this publication we refer to our tool as Jdoctor, which is the last official version of Toradocu (aka Toradocu 3.0).
Automatic Generation of Oracles for Exceptional Behaviors by Alberto Goffi, Alessandra Gorla, Michael D. Ernst, and Mauro Pezzè (presented at ISSTA 2016).

This publication describes the original work. In origin, Toradocu was processing only exceptional postconditions and was missing many of the current features.

Aspects tutorial

To see how to use Toradocu to automatically add test oracles to your test suite follow the tutorial instructions.

Building Toradocu

To compile Toradocu run the command: ./gradlew shadowJar

This will create the file build/libs/toradocu-1.0-all.jar. The size of file is approximately 1 GB.

Building Toradocu requires Java JDK 1.8+.

Running Toradocu

Toradocu is a command-line tool. The list of all Toradocu command-line options is available on the wiki. To get a list of available command-line parameters, execute

java -jar build/libs/toradocu-1.0-all.jar --help

A typical Toradocu invocation looks like this:

java -jar toradocu-1.0-all.jar \
   --target-class mypackage.MyClass \
   --test-class mypackage.MyTest \
   --source-dir project/src \
   --class-dir project/bin \
   --aspects-output-dir aspects

Toradocu + Randoop integration

Toradocu's assertions are integrated in Randoop, to augment its generated test cases with semantically meaningful oracles. Follow this link to see how the integration works:

https://github.com/ariannab/toyproject

Using Toradocu Aspects

With those options, Toradocu generates AspectJ aspects in the directory specified with the option --aspects-output-dir. In the aspects output directory, Toradocu places the source code of the aspects and an aop.xml that lists the generated aspects and that is used by the AspectJ compiler.

Aspects generated by Toradocu are standard AspectJ aspects and can be used to instrument an existing test suite. This is done by using the AspectJ compiler to weave the source files under test with the aspects generated by Toradocu.

To augment an existing test suite with Toradocu's oracles you have to:

Generate the aspects with Toradocu.
Compile the generated aspects.
Weave the existing test suites and the system under test.
Run the weaved test suite.

To compile the generated aspects you can use javac. Just be sure that JUnit, the AspectJ weaver, and your system under test are on the classpath.

To weave the existing test suite and the system under test, you can use the AspectJ compiler.

Run the weaved test suite as as you would normally.

Please refer to the AspectJ documentation for more information.

Contributing to Toradocu

Information for contributors can be found on the wiki pages.

albertogoffi/toradocu