Gradle Analytics generates JSON-delimited data files about Gradle activity in the build/analytics
directory (by default). Custom pipelines can be built for processing these data files, but Gradle Analytics provides capabilities to write the data from these generated files to a few known target locations, called sinks.
In most cases, Gradle Analytics tries to create destination buckets, topics, datasets, etc. An exception is Kinesis Firehose, which involves several custom steps during the configuration.
Read the API docs.
Configuration of supported sinks is covered here.
This plugin is published to the Gradle Plugin portal.
Gradle Analytics automatically generates files for builds, tasks, and tests.
It generates a unique ID for each build, called the buildid, and generates a buildtag for each CI/CD build/job/pipeline so that multiple buildid's from the same CI/CD workflow can be grouped together.
It uses the CI Detect Plugin to automatically generate the buildtag, but is overridden using the Gradle property analytics.buildTag
, usually with a -Panalytics.buildTag
in the CD/CD configuration file.
Gradle Analytics can be extended in custom plugins to generate additional activity. An example is the Gradle Confluent plugin which generates a custom JSON file containing every KSQL statement executed by the plugin.
plugins {
id 'com.redpillanalytics.gradle-analytics' version '<current-version>'
}
analytics {
ignoreErrors = false
// write to Google BigQuery
bq {
// can configure multiple locations or "environments"
// used in generating the task name
test {
// Files are first staged in a GCS bucket
bucket = 'rpa-gradle'
// then they are loaded to a BigQuery dataset
dataset = 'gradle_analytics'
}
}
// Write to Amazon Kinesis Firehose
firehose {
test {
// prefix for naming sinks, such as streams.
// For instance: 'gradle.build' for the 'build' analytics file.
prefix = 'gradle'
}
}
// write to a Kafka cluster
kafka {
prod {
bootstrapServers = 'PLAINTEXT://localhost:32857'
acks = 'all'
}
}
// write to an S3 bucket
s3 {
dev {
// the bucket name to write to
bucket = 'rpa-gradle'
// a suffix to add to the end of the default entity names
suffix = 'dev'
}
}
// write to a Google Cloud Storage bucket
gcs {
prod {
// the bucket name to write to
bucket = 'rpa-gradle'
// a prefix to add to the beginning of the default entity names
prefix = 'prod'
}
}
}
-> gradle tasks --group analytics
------------------------------------------------------------
Tasks runnable from root project
------------------------------------------------------------
Analytics tasks
---------------
bqTestSink - Process data files using the 'test' delivery sink. Each generated analytics file is written to a corresponding BigQuery table based on 'dataset', 'suffix' and 'prefix' values.
firehoseTestSink - Process data files using the 'test' delivery sink. Each generated analytics file is written to a corresponding Kinesis stream based on 'suffix' and 'prefix' values.
gcsProdSink - Process data files using the 'prod' delivery sink. Each generated analytics file is written to a corresponding GCS path based on 'suffix' and 'prefix' values.
kafkaProdSink - Process data files using the 'prod' delivery sink. Each generated analytics file is written to a corresponding Kafka topic based on 'suffix' and 'prefix' values.
producer - Analytics workflow task for producing data to all configured sinks.
s3DevSink - Process data files using the 'dev' delivery sink. Each generated analytics file is written to a corresponding S3 path based on 'suffix' and 'prefix' values.
-> gradle producer
> Configure project :
> Task :bqTestSink
Datset 'gradle_analytics' already exists.
2 analytics files processed.
> Task :firehoseTestSink
3 analytics files processed.
> Task :gcsProdSink
3 analytics files processed.
> Task :kafkaProdSink
3 analytics files processed.
> Task :s3DevSink
3 analytics files processed.
> Task :producer
BUILD SUCCESSFUL in 18s
6 actionable tasks: 6 executed
BUILD SUCCESSFUL in 33s
I will provide better, more comprehensive documentation soon. This includes Gradle Analytics support for generating custom JSON files for custom Gradle Plugins.