/rules_proto_grpc

Bazel rules for building Protocol Buffers ± gRPC code and libraries

Primary LanguageStarlarkApache License 2.0Apache-2.0

Protobuf and gRPC rules for Bazel

Bazel rules for building Protocol Buffers ± gRPC code and libraries from proto_library targets

Announcements

  • 2019/12/19: Bazel 2.0.0 has been released and appears to be fully supported by rules_proto_grpc version 1.0.0+. If you discover any issues with Bazel 2.x.x support, please open a new issue.

  • 2019/12/10: Bazel 1.0.0+ support has been added in rules_proto_grpc version 1.0.0. There have been a number of changes necessary to add support, which may require updating how you load rules_proto_grpc and its dependencies. Please see the 1.0.0 release notes for further information.

Contents:

Overview

These rules provide Protocol Buffers (Protobuf) and gRPC rules for a range of languages and services.

Each supported language ({lang} below) is generally split into four rule flavours:

  • {lang}_proto_compile: Provides generated files from the Protobuf protoc plugin for the language. e.g for C++ this provides the generated *.pb.cc and *.pb.h files.

  • {lang}_proto_library: Provides a language-specific library from the generated Protobuf protoc plugin outputs, along with necessary dependencies. e.g for C++ this provides a Bazel native cpp_library created from the generated *.pb.cc and *pb.h files, with the Protobuf library linked. For languages that do not have a 'library' concept, this rule may not exist.

  • {lang}_grpc_compile: Provides generated files from both the Protobuf and gRPC protoc plugins for the language. e.g for C++ this provides the generated *.pb.cc, *.grpc.pb.cc, *.pb.h and *.grpc.pb.h files.

  • {lang}_grpc_library: Provides a language-specific library from the generated Protobuf and gRPC protoc plugins outputs, along with necessary dependencies. e.g for C++ this provides a Bazel native cpp_library created from the generated *.pb.cc, *.grpc.pb.cc, *.pb.h and *.grpc.pb.h files, with the Protobuf and gRPC libraries linked. For languages that do not have a 'library' concept, this rule may not exist.

Therefore, if you are solely interested in the generated source code artifacts, use the {lang}_{proto|grpc}_compile rules. Otherwise, if you want a ready-to-go library, use the {lang}_{proto|grpc}_library rules.

These rules are derived from the excellent stackb/rules_proto and add aspect-based compilation to all languages, allowing for all proto_library options to be expressed in user code.

Installation

Add rules_proto_grpc your WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_proto_grpc",
    urls = ["https://github.com/rules-proto-grpc/rules_proto_grpc/archive/1.0.2.tar.gz"],
    sha256 = "5f0f2fc0199810c65a2de148a52ba0aff14d631d4e8202f41aff6a9d590a471b",
    strip_prefix = "rules_proto_grpc-1.0.2",
)

load("@rules_proto_grpc//:repositories.bzl", "rules_proto_grpc_toolchains", "rules_proto_grpc_repos")
rules_proto_grpc_toolchains()
rules_proto_grpc_repos()

It is recommended that you use the tagged releases for stable rules. Master is intended to be 'ready-to-use', but may be unstable at certain periods. To be notified of new releases, you can use GitHub's 'Watch Releases Only' on the repository.

Important: You will also need to follow instructions in the language-specific README.md for additional workspace dependencies that may be required.

Rules

Language Rule Description
Android android_proto_compile Generates an Android protobuf .jar artifact (example)
Android android_grpc_compile Generates Android protobuf+gRPC .jar artifacts (example)
Android android_proto_library Generates an Android protobuf library using android_library from rules_android (example)
Android android_grpc_library Generates Android protobuf+gRPC library using android_library from rules_android (example)
Closure closure_proto_compile Generates Closure protobuf .js files (example)
Closure closure_proto_library Generates a Closure library with compiled protobuf .js files using closure_js_library from rules_closure (example)
C++ cpp_proto_compile Generates C++ protobuf .h & .cc artifacts (example)
C++ cpp_grpc_compile Generates C++ protobuf+gRPC .h & .cc artifacts (example)
C++ cpp_proto_library Generates a C++ protobuf library using cc_library, with dependencies linked (example)
C++ cpp_grpc_library Generates a C++ protobuf+gRPC library using cc_library, with dependencies linked (example)
C# csharp_proto_compile Generates C# protobuf .cs artifacts (example)
C# csharp_grpc_compile Generates C# protobuf+gRPC .cs artifacts (example)
C# csharp_proto_library Generates a C# protobuf library using core_library from rules_dotnet (example)
C# csharp_grpc_library Generates a C# protobuf+gRPC library using core_library from rules_dotnet (example)
D d_proto_compile Generates D protobuf .d artifacts (example)
D d_proto_library Generates a D protobuf library using d_library from rules_d (example)
Go go_proto_compile Generates Go protobuf .go artifacts (example)
Go go_grpc_compile Generates Go protobuf+gRPC .go artifacts (example)
Go go_proto_library Generates a Go protobuf library using go_library from rules_go (example)
Go go_grpc_library Generates a Go protobuf+gRPC library using go_library from rules_go (example)
Java java_proto_compile Generates a Java protobuf srcjar artifact (example)
Java java_grpc_compile Generates a Java protobuf+gRPC srcjar artifact (example)
Java java_proto_library Generates a Java protobuf library using java_library (example)
Java java_grpc_library Generates a Java protobuf+gRPC library using java_library (example)
Node.js nodejs_proto_compile Generates Node.js protobuf .js artifacts (example)
Node.js nodejs_grpc_compile Generates Node.js protobuf+gRPC .js artifacts (example)
Objective-C objc_proto_compile Generates Objective-C protobuf .m & .h artifacts (example)
Objective-C objc_grpc_compile Generates Objective-C protobuf+gRPC .m & .h artifacts (example)
Objective-C objc_proto_library Generates an Objective-C protobuf library using objc_library (example)
PHP php_proto_compile Generates PHP protobuf .php artifacts (example)
PHP php_grpc_compile Generates PHP protobuf+gRPC .php artifacts (example)
Python python_proto_compile Generates Python protobuf .py artifacts (example)
Python python_grpc_compile Generates Python protobuf+gRPC .py artifacts (example)
Python python_grpclib_compile Generates Python protobuf+grpclib .py artifacts (supports Python 3 only) (example)
Python python_proto_library Generates a Python protobuf library using py_library (example)
Python python_grpc_library Generates a Python protobuf+gRPC library using py_library (example)
Python python_grpclib_library Generates a Python protobuf+grpclib library using py_library (supports Python 3 only) (example)
Ruby ruby_proto_compile Generates Ruby protobuf .rb artifacts (example)
Ruby ruby_grpc_compile Generates Ruby protobuf+gRPC .rb artifacts (example)
Ruby ruby_proto_library Generates a Ruby protobuf library using ruby_library from rules_ruby (example)
Ruby ruby_grpc_library Generates a Ruby protobuf+gRPC library using ruby_library from rules_ruby (example)
Rust rust_proto_compile Generates Rust protobuf .rs artifacts (example)
Rust rust_grpc_compile Generates Rust protobuf+gRPC .rs artifacts (example)
Rust rust_proto_library Generates a Rust protobuf library using rust_library from rules_rust (example)
Rust rust_grpc_library Generates a Rust protobuf+gRPC library using rust_library from rules_rust (example)
Scala scala_proto_compile Generates a Scala protobuf .jar artifact (example)
Scala scala_grpc_compile Generates Scala protobuf+gRPC .jar artifacts (example)
Scala scala_proto_library Generates a Scala protobuf library using scala_library from rules_scala (example)
Scala scala_grpc_library Generates a Scala protobuf+gRPC library using scala_library from rules_scala (example)
Swift swift_proto_compile Generates Swift protobuf .swift artifacts (example)
Swift swift_grpc_compile Generates Swift protobuf+gRPC .swift artifacts (example)
Swift swift_proto_library Generates a Swift protobuf library (example)
Swift swift_grpc_library Generates a Swift protobuf+gRPC library (example)
Go (gogoprotobuf) gogo_proto_compile Generates gogo protobuf .go artifacts (example)
Go (gogoprotobuf) gogo_grpc_compile Generates gogo protobuf+gRPC .go artifacts (example)
Go (gogoprotobuf) gogo_proto_library Generates a Go gogo protobuf library using go_library from rules_go (example)
Go (gogoprotobuf) gogo_grpc_library Generates a Go gogo protobuf+gRPC library using go_library from rules_go (example)
Go (gogoprotobuf) gogofast_proto_compile Generates gogofast protobuf .go artifacts (example)
Go (gogoprotobuf) gogofast_grpc_compile Generates gogofast protobuf+gRPC .go artifacts (example)
Go (gogoprotobuf) gogofast_proto_library Generates a Go gogofast protobuf library using go_library from rules_go (example)
Go (gogoprotobuf) gogofast_grpc_library Generates a Go gogofast protobuf+gRPC library using go_library from rules_go (example)
Go (gogoprotobuf) gogofaster_proto_compile Generates gogofaster protobuf .go artifacts (example)
Go (gogoprotobuf) gogofaster_grpc_compile Generates gogofaster protobuf+gRPC .go artifacts (example)
Go (gogoprotobuf) gogofaster_proto_library Generates a Go gogofaster protobuf library using go_library from rules_go (example)
Go (gogoprotobuf) gogofaster_grpc_library Generates a Go gogofaster protobuf+gRPC library using go_library from rules_go (example)
grpc-gateway gateway_grpc_compile Generates grpc-gateway .go files (example)
grpc-gateway gateway_swagger_compile Generates grpc-gateway swagger .json files (example)
grpc-gateway gateway_grpc_library Generates grpc-gateway library files (example)
gRPC-Web closure_grpc_compile Generates Closure *.js protobuf+gRPC files (example)
gRPC-Web commonjs_grpc_compile Generates CommonJS *.js protobuf+gRPC files (example)
gRPC-Web commonjs_dts_grpc_compile Generates commonjs_dts *.js protobuf+gRPC files (example)
gRPC-Web ts_grpc_compile Generates CommonJS *.ts protobuf+gRPC files (example)
gRPC-Web closure_grpc_library Generates protobuf closure library *.js files (example)

Example Usage

These steps walk through the steps to go from a raw .proto file to a C++ library:

Step 1: Write a Protocol Buffer file (example: thing.proto):

syntax = "proto3";

package example;

import "google/protobuf/any.proto";

message Thing {
    string name = 1;
    google.protobuf.Any payload = 2;
}

Step 2: Write a BAZEL.build file with a native proto_library rule:

proto_library(
    name = "thing_proto",
    srcs = ["thing.proto"],
    deps = ["@com_google_protobuf//:any_proto"],
)

In this example we have a dependency on a well-known type any.proto, hence the proto_library to proto_library dependency ("@com_google_protobuf//:any_proto")

Step 3: Add a cpp_proto_compile rule (substitute cpp_ for the language of your choice).

NOTE: In this example thing.proto does not include service definitions (gRPC). For protos with services, use the cpp_grpc_compile rule instead.

# BUILD.bazel
load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_compile")

cpp_proto_compile(
    name = "cpp_thing_proto",
    deps = [":thing_proto"],
)

But wait, before we can build this, we need to load the dependencies necessary for this rule (from cpp/README.md):

Step 4: Load the workspace macro corresponding to the build rule.

# WORKSPACE
load("@rules_proto_grpc//cpp:repositories.bzl", "cpp_repos")

cpp_repos()

We're now ready to build the rule:

Step 5: Build it!

$ bazel build //example/proto:cpp_thing_proto
Target //example/proto:cpp_thing_proto up-to-date:
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc

If we were only interested in the generated file artifacts, the cpp_grpc_compile rule would be fine. However, for convenience we'd rather have the outputs compiled into an *.so file. To do that, let's change the rule from cpp_proto_compile to cpp_proto_library:

# BUILD.bazel
load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_library")

cpp_proto_library(
    name = "cpp_thing_proto",
    deps = [":thing_proto"],
)
$ bazel build //example/proto:cpp_thing_proto
Target //example/proto:cpp_thing_proto up-to-date:
  bazel-bin/example/proto/libcpp_thing_proto.a
  bazel-bin/example/proto/libcpp_thing_proto.so  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc

This way, we can use //example/proto:cpp_thing_proto as a dependency of any other cc_library or cc_binary rule as per normal.

NOTE: the cpp_proto_library implicitly calls cpp_proto_compile, and we can access that rule by adding _pb at the end of the rule name, like bazel build //example/proto:cpp_thing_proto_pb.

Migration

For users migrating from the stackb/rules_proto rules, please see the help at MIGRATION.md

Developers

Code Layout

Each language {lang} has a top-level subdirectory that contains:

  1. {lang}/README.md: Generated documentation for the language rules.

  2. {lang}/repositories.bzl: Macro functions that declare repository rule dependencies for that language.

  3. {lang}/{rule}.bzl: Rule implementations of the form {lang}_{kind}_{type}, where kind is one of proto|grpc and type is one of compile|library.

  4. {lang}/BUILD.bazel: proto_plugin() declarations for the available plugins for the language.

  5. example/{lang}/{rule}/: Generated WORKSPACE and BUILD.bazel demonstrating standalone usage of the rules.

  6. {lang}/example/routeguide/: Example routeguide example implementation, if possible.

The repository root directory contains the base rule defintions:

  • plugin.bzl: A build rule that defines the name, tool binary, and options for a particular proto plugin.

  • aspect.bzl: Contains the implementation of the compilation aspect. This is shared by all rules and is the heart of rules_proto_grpc; it calls protoc with a given list of plugins and generates output files.

Additional protoc plugins and their rules are scoped to the github repository name where the plugin resides.

Rule Generation

To help maintain consistency of the rule implementations and documentation, all of the rule implementations are generated by the tool //tools/rulegen. Changes in the main README.md should be placed in tools/rulegen/README.header.md or tools/rulegen/README.footer.md. Changes to generated rules should be put in the source files (example: tools/rulegen/java.go).

How-it-works

Briefly, here's how the rules work:

  1. Using the proto_library graph, an aspect walks through the ProtoInfo providers on the deps attribute to {lang}_{proto|grpc}_compile. This finds all the directly and transitively required proto files, along with their options.

  2. At each node visited by the aspect, protoc is invoked with the relevant plugins and options to generate the desired outputs. The aspect uses only the generated proto descriptors from the ProtoInfo providers.

  3. Once the aspect stage is complete, all generated outputs are optionally gathered into a final output tree.

  4. For {lang}_{proto|grpc}_library rules, the generated outputs are then aggregated into a language-specific library. e.g a .so file for C++.

Developing Custom Plugins

Generally, follow the pattern seen in the multiple language examples in this repository. The basic idea is:

  1. Load the plugin rule: load("@rules_proto_grpc//:plugin.bzl", "proto_plugin").
  2. Define the rule, giving it a name, options (not mandatory), tool, and outputs.
  3. tool is a label that refers to the binary executable for the plugin itself.
  4. Choose your output type (pick one!):
    • outputs: a list of strings patterns that predicts the pattern of files generated by the plugin. For plugins that produce one output file per input proto file
    • out: the name of a single output file generated by the plugin.
    • output_directory: Set to true if your plugin generates files in a non-predictable way. e.g. if the output paths depend on the service names.
  5. Create a compilation rule and aspect using the following template:
load("@rules_proto_grpc//:plugin.bzl", "ProtoPluginInfo")
load(
    "@rules_proto_grpc//:aspect.bzl",
    "ProtoLibraryAspectNodeInfo",
    "proto_compile_aspect_attrs",
    "proto_compile_aspect_impl",
    "proto_compile_attrs",
    "proto_compile_impl",
)

# Create aspect
example_aspect = aspect(
    implementation = proto_compile_aspect_impl,
    provides = [ProtoLibraryAspectNodeInfo],
    attr_aspects = ["deps"],
    attrs = dict(
        proto_compile_aspect_attrs,
        _plugins = attr.label_list(
            doc = "List of protoc plugins to apply",
            providers = [ProtoPluginInfo],
            default = [
                Label("//<LABEL OF YOUR PLUGIN>"),
            ],
        ),
        _prefix = attr.string(
            doc = "String used to disambiguate aspects when generating outputs",
            default = "example_aspect",
        )
    ),
    toolchains = ["@rules_proto_grpc//protobuf:toolchain_type"],
)

# Create compile rule to apply aspect
_rule = rule(
    implementation = proto_compile_impl,
    attrs = dict(
        proto_compile_attrs,
        deps = attr.label_list(
            mandatory = True,
            providers = [ProtoInfo, ProtoLibraryAspectNodeInfo],
            aspects = [example_aspect],
        ),
    ),
)

# Create macro for converting attrs and passing to compile
def example_compile(**kwargs):
    _rule(
        verbose_string = "{}".format(kwargs.get("verbose", 0)),
        merge_directories = True,
        **{k: v for k, v in kwargs.items() if k != "merge_directories"}
    )

License

This project is derived from stackb/rules_proto under the Apache 2.0 license and this project therefore maintains the terms of that license. An overview of the changes can be found at MIGRATION.md.

Contributing

Contributions are very welcome. Please see CONTRIBUTING.md for further details.