Bazel Buildfarm

This repository hosts the Bazel remote caching and execution system.

This project is just getting started; background information on the status of caching and remote execution in bazel can be found in the bazel documentation.

Read the meeting notes. Get involved by joining the general api discussion on the remote execution apis mailing list.

Usage

All commandline options override corresponding config settings.

Bazel Buildfarm Server

Run via

bazel run //src/main/java/build/buildfarm:buildfarm-server <configfile> [<-p|--port> PORT]

configfile has to be in Protocol Buffer text format, corresponding to a BuildFarmServerConfig definition.

For an example, see the examples directory, which contains the working example examples/worker.config.example. For format details see here. Protocol Buffer structure at src/main/protobuf/build/buildfarm/v1test/buildfarm.proto
PORT to expose service endpoints on

Bazel Buildfarm Worker

Run via

bazel run //src/main/java/build/buildfarm:buildfarm-operationqueue-worker <configfile> [--root ROOT] [--cas_cache_directory CAS_CACHE_DIRECTORY]

configfile has to be in Protocol Buffer text format, corresponding to a WorkerConfig definition.

For an example, see the examples directory, which contains the working example examples/server.config.example. For format details see here. Protocol Buffer structure at src/main/protobuf/build/buildfarm/v1test/buildfarm.proto
ROOT base directory path for all work being performed.
CAS_CACHE_DIRECTORY is (absolute or relative) directory path to cached files from CAS.

Bazel Client

To use the example configured buildfarm with bazel (version 1.0 or higher), you can configure your .bazelrc as follows:

$ cat .bazelrc
build --spawn_strategy=remote --genrule_strategy=remote --strategy=Javac=remote --strategy=Closure=remote --remote_executor=grpc://localhost:8980

Then run your build as you would normally do.

Debugging

Buildfarm uses Java's Logging framework and outputs all routine behavior to the NICE Level.

You can use typical Java logging configuration to filter these results and observe the flow of executions through your running services. An example logging.properties file has been provided at examples/debug.logging.properties for use as follows:

bazel run //src/main/java/build/buildfarm:buildfarm-server --jvm_flag=-Djava.util.logging.config.file=examples/debug.logging.properties ...

and

bazel run //src/main/java/build/buildfarm/buildfarm-operationqueue-worker --jvm_flag=-Djava.util.logging.config.file=examples/debug.logging.properties ...

To attach a remote debugger, run the executable with the --debug=<PORT> flag. For example:

bazel run src/main/java/build/buildfarm/buildfarm-server --debug=5005 \
    $PWD/config/server.config

Developer Information

Setting up intelliJ

Follow the instructions in https://github.com/bazelbuild/intellij to install the bazel plugin for intelliJ
Import the project using ij.bazelproject

Third-party Dependencies

Most third-party dependencies (e.g. protobuf, gRPC, ...) are managed automatically via bazel-deps. After changing the dependencies.yaml file, just run this to regenerate the 3rdparty folder:

git clone https://github.com/johnynek/bazel-deps.git ../bazel-deps
cd ../bazel-deps
bazel build //src/scala/com/github/johnynek/bazel_deps:parseproject_deploy.jar
cd ../bazel-buildfarm
../bazel-deps/gen_maven_deps.sh generate -r `pwd` -s 3rdparty/workspace.bzl -d dependencies.yaml

Things that aren't supported by bazel-deps are being imported as manually managed remote repos via the WORKSPACE file.

agoulti/bazel-buildfarm