Gradle plugin that visualises tests execution schedule

Usage

build.gradle.kts:

plugins {
    id("io.github.platan.tests-execution-chart") version "0.6.1"
}

build.gradle:

plugins {
    id "io.github.platan.tests-execution-chart" version "0.6.1"
}

./gradlew test createTestsExecutionReport --rerun-tasks

#...

Tests execution schedule report saved to /my-project/build/reports/tests-execution/mermaid/test.txt file.
Tests execution schedule report saved to /my-project/build/reports/tests-execution/json/test.json file.
Tests execution schedule report saved to /my-project/build/reports/tests-execution/html/test.html file.

Example result (HTML report):

Reports

createTestsExecutionReport task creates reports in HTML, JSON and Mermaid formats.

HTML

Located in build/reports/tests-execution/html/*.html.

JSON

Located in build/reports/tests-execution/json/*.json.

startTime - in milliseconds since the epoch
endTime - in milliseconds since the epoch

{
  "results": [
    {
      "className": "Test1Spec",
      "durationMs": 120,
      "endTime": 1667936777974,
      "startTime": 1667936777854,
      "resultType": "SUCCESS",
      "testName": "test 2"
    },
    {
      "className": "Test1Spec",
      "durationMs": 213,
      "endTime": 1667936778067,
      "startTime": 1667936777854,
      "resultType": "SUCCESS",
      "testName": "test 1"
    }
  ]
}

Mermaid

Located in build/reports/tests-execution/mermaid/*.txt.

gantt
dateFormat YYYY-MM-DDTHH:mm:ss.SSSZZ
axisFormat %H:%M:%S.%L
section Test1Spec
test 2 - 120 ms :active, 2022-11-08T20:46:17.854+0100, 2022-11-08T20:46:17.974+0100
test 1 - 213 ms :active, 2022-11-08T20:46:17.854+0100, 2022-11-08T20:46:18.067+0100

rendered: (info):

gantt
    dateFormat YYYY-MM-DDTHH:mm:ss.SSSZZ
    axisFormat %H:%M:%S.%L
    section Test1Spec
        test 2 - 120 ms: active, 2022-11-08T20:46:17.854+0100, 2022-11-08T20:46:17.974+0100
        test 1 - 213 ms: active, 2022-11-08T20:46:17.854+0100, 2022-11-08T20:46:18.067+0100

Configuration

Options:

Key	Type	Description	Default
`formats.html.enabled`	boolean	Generate report in html format	`true`
`formats.html.script.embed`	boolean	If true mermaid source will be downloaded and used locally in html	`false`
`formats.html.script.src`	url	Url to mermaid which should be used to generate html report	`https://cdn.jsdelivr.net/npm/mermaid@11.3.0/dist/mermaid.min.js`
`formats.html.script.config.maxTextSize`	int	Limit on the size of text used to generate diagrams	`50000`
`formats.json.enabled`	boolean	Generate report in json format	`true`
`formats.mermaid.enabled`	boolean	Generate report in mermaid text format	`true`
`components.suites.enabled`	boolean	Add suites/classes to report	`false`
`shiftTimestampsToStartOfDay`	boolean	Adjust the earliest timestamp to the start of the day	`false`
`marks.totalTimeOfAllTests.enabled`	boolean	Enable mark showing total time of all tests	`false`
`marks.totalTimeOfAllTests.name`	string	Label used for mark	`total time of all tests`

build.gradle.kts:

configure<io.github.platan.tests_execution_chart.CreateTestsExecutionReportExtension> {
    formats {
        html {
            enabled.set(true)
            script {
                embed.set(false)
                config {
                    maxTextSize.set(110000)
                }
                src.set("https://cdn.jsdelivr.net/npm/mermaid@11.3.0/dist/mermaid.min.js")
            }
        }
        json {
            enabled.set(true)
        }
        mermaid {
            enabled.set(true)
        }
    }
    components {
        suites {
            enabled.set(false)
        }
    }
    shiftTimestampsToStartOfDay.set(true)
    marks {
        totalTimeOfAllTests {
            enabled.set(true)
            name.set("total time of all tests")
        }
    }
}

build.gradle:

createTestsExecutionReport {
    formats {
        html {
            enabled = true
            script {
                embed = false
                config {
                    maxTextSize = 110000
                }
                src = "https://cdn.jsdelivr.net/npm/mermaid@11.3.0/dist/mermaid.min.js"
            }
        }
        json {
            enabled = true
        }
        mermaid {
            enabled = true
        }
    }
    components {
        suites {
            enabled = false
        }
    }
    shiftTimestampsToStartOfDay = true
    marks {
        totalTimeOfAllTests {
            enabled = true
            name = 'total time of all tests'
        }
    }
}

Option `components.suites.enabled`

When enabled (set to true) it allows to add suites/classed to report.

Troubleshooting

I see Maximum text size in diagram exceeded in an HTML report instead of a chart.

Mermaid has limit of maximum allowed size of the text diagram. Default value of this size is 50000. This can be overridden using formats.html.script.config.maxTextSize option.

Development

You can publish plugin locally:

cd tests-execution-chart
./gradlew publish

Artifacts are published to ../local-plugin-repository directory.

Configure your project:

settings.gradle (set proper path instead of /path-to):

pluginManagement {
    repositories {
        maven {
            url '/path-to/local-plugin-repository'
        }
        gradlePluginPortal()
    }
}

build.gradle (replace X.X.X-SNAPSHOT with the value returned by ./gradlew cV -q -Prelease.quiet):

plugins {
    id 'io.github.platan.tests-execution-chart' version 'X.X.X-SNAPSHOT'
}

Release

./gradlew createRelease -Prelease.versionIncrementer=incrementPatch/incrementMinor/incrementMinor - creates a new tag and updates examples in README (but does not commit changes in README)
git push origin release-[X.X.X]
Prepare ~/.gradle/gradle.properties (gradle.publish.key, gradle.publish.secret) and run ./gradlew clean :tests-execution-chart-commons:publishToSonatype closeSonatypeStagingRepository
Verify tests-execution-chart-commons module in staging repository (https://s01.oss.sonatype.org)
Run ./gradlew findSonatypeStagingRepository releaseSonatypeStagingRepository
Prepare ~/.gradle/gradle.properties (sonatypeUsername, sonatypePassword, signing.keyId, signing.password, signing.secretKeyRingFile) and run ./gradlew clean build publishPlugins
Edit Changelog in README.md and git commit -m "Document release [X.X.X]"
Update io.github.platan.tests-execution-chart plugin version in this project

Modules

tests-execution-chart-commons

tests-execution-chart-commons module contains common code which is used by the plugin. This library can be also used to build other tools.

Motivation

JUnit and Spock support parallel execution. Both frameworks allow configuring execution mode for classes/specifications and methods/features. JUnit/Spock documentation illustrates how this configuration affects the execution schedule of tests.

Unfortunately, I did not find existing tool which allows to visualise tests executions schedule for particular project. This Gradle plugin tries to address this need.

Gradle can generate reports in JUnit XML format. But such reports cannot be used to generate charts, because they do not have start timestamps for test methods, only for test classes (with seconds precision).

Changelog

Unreleased

0.6.1 (16 October 2024)

Changed

Mermaid library updated to version 11.3.0

0.6.0 (17 October 2023)

Added

Add option to include suites/classes in report

0.5.0 (15 October 2023)

Changed

Mermaid library updated to version 10.5.0

0.4.0 (1 June 2023)

Added

A new option marks.totalTimeOfAllTests allows to add mark showing total time of all tests

Changed

Do not remove # character from names of tasks/sections

0.3.1 (29 March 2023)

Added

A new option shiftTimestampsToStartOfDay allows to adjust the earliest timestamp to the start of the day

0.3.0 (12 March 2023)

Changed

Use dateFormat compatible with Mermaid 9.4.2
Link to Mermaid 9.4.3 instead of the latest version

0.2.1 (07 March 2023)

(fix) Render chart in HTML report when the test name contains a backtick character

0.2.0 (02 February 2023)

(feature) Allow to set maxTextSize config option in Mermaid script in HTML report

0.1.0 (09 November 2022)

Add option to create a report that visualises the tests execution schedule

License

This project is licensed under the MIT license.

hannahms/tests-execution-chart

Usage

Reports

HTML

JSON

Mermaid

Configuration

Option components.suites.enabled

Troubleshooting

Development

Release

Modules

tests-execution-chart-commons

Motivation

Changelog

Unreleased

0.6.1 (16 October 2024)

Changed

0.6.0 (17 October 2023)

Added

0.5.0 (15 October 2023)

Changed

0.4.0 (1 June 2023)

Added

Changed

0.3.1 (29 March 2023)

Added

0.3.0 (12 March 2023)

Changed

0.2.1 (07 March 2023)

0.2.0 (02 February 2023)

0.1.0 (09 November 2022)

License

Option `components.suites.enabled`