Scott Test Reporter

Scott provides failure messages containing all state changes for tests written in Java to get meaningful errors even without assertion libraries. All information is presented on the source code of the test method as comments.

myTest(hu.advancedweb.example.ListTest) Time elapsed: 0.028 sec  <<< FAILURE!
java.lang.AssertionError: 

  22|      @Test
  23|      public void myTest() {
  24|          Integer[] myArray = new Integer[] { 1, 4, 2, 4 };  // myArray=[1, 4, 2, 4]
  25|          List<Integer> myList = Arrays.asList(myArray);  // myList=[1, 4, 2, 4]
  26|
  27|          Set<Integer> mySet = new HashSet<>(myList);  // mySet=[1, 2, 4]
  28|          mySet.remove(4);  // mySet=[1, 2]
  29|
  30|*         assertTrue(mySet.contains(4));  // AssertionError
  31|      }
  
        at hu.advancedweb.example.ListTest.myTest(ListTest.java:25)

Scott automatically tracks the internal state of the tests to provide important details for a failing scenario.

Although it plays nicely with other testing tools and frameworks, all information is present in the report even without using sophisticated assertions.

Besides the usual assertion error, Scott reports the changes and assignments related to

local variables
input parameters
fields that the test accesses

Supports JUnit 4, JUnit 5, and Cucumber Java on Java 7+ (up to Java 12).

How to use

Just drop it into your project, and Scott will automatically enhance your test reports. You don't have to use its API or modify your existing tests to make it work.

Scott: All systems automated and ready. A chimpanzee and two trainees could run her.

Kirk: Thank you, Mr. Scott. I'll try not to take that personally.

Gradle

Add hu.advanceweb.scott-gradle-plugin to your build.gradle:

plugins {
  id "hu.advanceweb.scott-gradle-plugin" version "3.4.1"
}

Example projects:

Maven

Add the following to your pom.xml:

<build>
	<plugins>
		<!-- Add the Scott Plugin. -->
		<plugin>
			<groupId>hu.advancedweb</groupId>
			<artifactId>scott-maven-plugin</artifactId>
			<version>3.4.1</version>
			<executions>
				<execution>
					<goals>
						<goal>prepare-agent</goal>
					</goals>
				</execution>
			</executions>
		</plugin>
	</plugins>
</build>
<dependencies>
	<!-- Add Scott as a dependency -->
	<dependency>
		<groupId>hu.advancedweb</groupId>
		<artifactId>scott</artifactId>
		<version>3.4.1</version>
		<scope>test</scope>
	</dependency>
</dependencies>

Example projects:

Cucumber

Scott for Cucumber tracks whole scenarios, and in case of a failure it prints the details of every step involved.

This feature provides valuable information if a test fails in a CI environment, as it can make it much easier to reproduce and fix browser-based tests, especially for flaky tests.

Example projects with Cucumber tests:

Wire it up manually

If you can't use the Gradle or Maven Plugin for some reason, you can do the necessary steps manually.

Configuration

In case you are not satisfied with the default tracking behavior, the Scott Maven Plugin and Gradle Plugin provides configuration options to fine-tune its behaviour.

Changelog

See Releases.

Highlights from the latest releases:

Java 12 support
Gradle Plugin
Maven Plugin
Scott Cucumber Java now records whole scenarios, supports io.cucumber:cucumber-java and info.cukes:cucumber-java
Customizable tracking behavior

Contributing

Contributions are welcome! Please make sure to visit the contribution and development guide for some important notes on how to build and debug Scott. If you are looking for issues that can get you started with the development, see Issues marked with the help-wanted tag.

adnovum/scott