A distributed build cache for Bazel declarative build system.
Bazel is a great tool for having reproducible builds. However, if you run a large CI/CD pipeline using it, you end up rebuilding many artifacts over and over across different agents (or containers).
Bazel has experimental support for gRPC-based remote workers/caches.
The goal of the project is to leverage it and build:
localcache
- local daemon thatbazel
talks to, providing low latency-first caching layerdistcache
- remote server thatlocalcache
talks to in case it encounters a miss
Because Bazel supports hermetic builds, this cache will be usable for all languages and build targets: Java, C++, Go, Python, protobuf, Scala... you name it.
This is pre-alpha, basically a localcache
proof of concept. It works, and bazel
happily uses it as a cache.
To build:
go install github.com/mwitkow/bazel-distcache/cmd/localcache
To start:
bin/localcache --blobstore_ondisk_path=/tmp/localcache-blobstore
At this point an HTTP debug interface (including metrics) is running on http://localhost:10100. The default gRPC address
for bazel is localhost:10101
. You can use it for example:
bazel --host_jvm_args=-Dbazel.DigestFunction=SHA1 build --spawn_strategy=remote --remote_cache=localhost:10101 ...
- you can enable gRPC tracing on https://localhost:10100/debug/requests with
--grpc_tracing_enabled
for easier debugging remote_protocol.proto
contains the protocolGrpcActionCache.java
is the Bazel-side of the protocol, with some interesting assumptionsRemoteOptions.java
contains the CLI parameters for caching in Bazel- if you want to rebuild protos, use
proto/protogen.sh
bazel-distcache
is released under the Apache 2.0 license. See the LICENSE file for details.