Talaiot 2.0 - Compatibility with Configuration Cache

[cdsap]

Overview
Configuration Cache is a Gradle Feature that allows skipping the configuration phase bringing better build times, on a massive modularize Gradle project this could reduce the overall user experience dramatically.
If there are no changes in the build.gradle files, the configuration will be reused. Additionally, version.catalogs, gradle.properties or settings.gradle are involved in the configuration cache.
In order to be compatible with Configuration Cache, plugins must not register any build listeners:
https://docs.gradle.org/current/userguide/configuration_cache.html#config_cache:requirements:build_listeners
Talaiot is registering the plugins:

• TalaiotListener
• BuildCacheOperationListener

Builds are not failing, however, at the time to reuse the configuration, the listener interface is not called not publishing the information of the build. This issue was already discussed in #313

This document explains the steps required to make the Plugins compatible with Configuration Cache and the impact on the current project. We have a snapshot available for testing the new Talaiot 2.0 version.

BuildService
It’s the recommended approach to interact with the build listener events in the plugin: https://docs.gradle.org/current/userguide/build_services.html#build_services. A build service implementation is treated as a custom Gradle type and can use any of the features available to custom Gradle types. The new Talaiot implementation uses the TalaiotBuildService to register the task information.

val serviceProvider: Provider<TalaiotBuildService> =
    target.gradle.sharedServices.registerIfAbsent(
        "talaiotService", TalaiotBuildService::class.java
    ) { spec ->
        spec.parameters.publisher.set(talaiotPublisher)
        spec.parameters.startParameters.set(parameters)
    }
target.serviceOf<BuildEventsListenerRegistry>().onTaskCompletion(serviceProvider)

The parameters required for the TalaiotBuildServce are a TalaiotPublisher and Gradle parameters.
More details https://github.com/cdsap/Talaiot/blob/talaiot_2_0/library/core/talaiot/src/main/kotlin/io/github/cdsap/talaiot/TalaiotBuildService.kt

Object Serialization
Gradle uses an optimised serialization mechanism to store the configuration cache entries(https://docs.gradle.org/current/userguide/configuration_cache.html#config_cache:intro:how_does_it_work). For this reason, we need to provide Serialization compatibility to the types used in the TalaiotBuildService.
The way to make compatible the types involved is to extend from the java interface Serializable. The scope of the Talaiot Plugin is reduced to the collection of task duration + metrics collection, reducing the possible performance impact. Most of the changes proposed for Talaiot 2.0 are related to the serialization compatibility of the objects involved. Some of the previous functionality has been affected by this requirement, and we will be forced to drop some of the plugins or metrics used in Talaiot 1.x.

Breaking Changes Talaiot 2.0

Build Metrics removed

Metric	Reason
TotalRamAvailableBytes	Oshi library incompatibility
OsManufacturer	Oshi library incompatibility
PublicIp	Oshi library incompatibility
GradleScanLink	BuildScan Listener not valid
LocalCacheHit	Cache Build Listener removed
LocalCacheMiss	Cache Build Listener removed
RemoteCacheHit	Cache Build Listener removed
RemoteCacheMiss	Cache Build Listener removed

Tasks Metrics Removed

Metric	Reason
WorkerId	Project object can’t be serialized
Critical	Project object can’t be serialized
LocalCacheHit	Cache Build Listener removed
LocalCacheMiss	Cache Build Listener removed
RemoteCacheHit	Cache Build Listener removed
RemoteCacheMiss	Cache Build Listener removed

Custom Publishers
Custom Publishers defined in the build scripts are not valid anymore.
We didn’t test the extension of custom publishers in buildSrc convention folder. The extension with new publishers wrapped in new plugins is still available.

Plugins
Graph Plugin: Removed due to the impossibility to retrieve the dependency tasks with the build listener.
Base Plugin:
— TimeLine publisher: Workers information not available.

Migration Strategy
Today we have released the first Snapshot for Talaiot 2.0. You can download it with:

buildscript {
    repositories {
        mavenCentral()
        maven(url = uri("https://s01.oss.sonatype.org/content/repositories/snapshots/"))
    }
    dependencies {
        classpath("io.github.cdsap:talaiot:2.0.0-SNAPSHOT")
    }
}

The idea is to collect feedback, bugs, suggestions and release the plugin in the next development cycle(end of june).
Please add the label 2.0 in the tickets created.

Please, @mokkun @MyDogTom @Malinskiy @MarinaShaposhnikova if you have time could you test the snapshot in your projects if you are using the library.

Info
Branch talaiot_2_0 has been setup with protected state, requiring CI checks to pass.
The PR create contains the diff with main: #348

[MyDogTom]

@cdsap Great to see configuration cache support! I'll test snapshot during the weekend.

[mokkun]

Nice work! I will try to give it a go this weekend as well. 👍
Now that the Cache Build Listener is removed, do we have any planned alternative to measure the local and remote cache hit metrics?

[cdsap]

hi @mokkun good point.
The assumption is: because build listeners don't work we are dropping the functionality. We provide cache state info but we don't know if is from local or remote.
Said that, the current master/main implementation is not using the general build listener:

val buildOperationListener = BuildCacheOperationListener()
target.gradle.buildOperationListenerManager().addListener(buildOperationListener)

The doc doesn't mention about the usage of BuildOperationListenerManager with (this as GradleInternal).services[BuildOperationListenerManager::class.java], so I'm not sure if we can try to find a workaround here. I did a quick test but didn't work throwing a StackOverflow exception.
I guess we could get the origin of the cache status in the first build, but the second one(hitting the configuration cache) could have a unexpected behavior(not serialized in the BuildService)

[MyDogTom]

Hi @cdsap ,
Sorry for this enormous delay. Only now I was able to try out the snapshot. I use influxdb-publisher and publishing works fine 🎉 . I checked when configuration cache is not available and when configuration cache is present.

I found two unexpected behaviours:

• configuration field is always zero. Even when configuration phase took some time due to missing configuration cache.
• I have two custom metrics, but it seems they are completely ignored. I extend GradleMetric.

What's the next step for me? Do you prefer that I check some other scenarios or dig deeper into any of these issues?

[cdsap]

@MyDogTom thanks for the feedback.

Configuration time should be addressed in this issue #354.
I don't remember details about GradleMetrics but let me check and I'll reply you