dprogm/ngx-grpc

Angular gRPC framework

ngx-grpc

Angular gRPC framework.

Changelog

Features

• two-way-binding-friendly protobuf messages implementation (instead of Java-like setters / getters / builder pattern in original google-protobuf)
• client services are wired to Angular's dependency injection
• typescript first-class support
• rxjs first-class support
• client & bidirectional streaming (only possible with @improbable-eng/grpc-web)
• interceptors
• logger
• support for well-known types, including Any
• support for JSON mappings
• 3 different client implementations: grpc-web, @improbable-eng/grpc-web and web worker
• easy to install, update and support thanks to npm packages

Example

The example requires docker & docker-compose to be installed

Clone this repository and run

npm ci
npm run build
npm run examples

On m1 chips replace npm ci with npm ci --target_arch=x64 --no-optional.

Now open your browser at http://localhost:4200/. The source code could be found at examples directory.

Installation

The documentation uses @ngx-grpc/grpc-web-client by default, however is applicable to any client you choose.

First ensure that you

• installed protoc as a binary or as a part of grpc-tools npm package
• configured your backend / grpc-web proxy according to grpc-web documentation or follow documentation of the alternative client if you use one

Then in your Angular project's root directory run

npm i -S @ngx-grpc/common @ngx-grpc/core @ngx-grpc/grpc-web-client @ngx-grpc/well-known-types google-protobuf grpc-web
npm i -D @ngx-grpc/protoc-gen-ng @types/google-protobuf

Where:

• @ngx-grpc/common contains common reusable types for other ngx-grpc packages
• @ngx-grpc/core contains angular specific implementation
• @ngx-grpc/grpc-web-client provides a client based on grpc-web
• @ngx-grpc/protoc-gen-ng generates the code based on your proto files
• @ngx-grpc/well-known-types contains well-known types
• google-protobuf is required to encode / decode the messages
• grpc-web implements the transport between the browser and grpc proxy

Also you can choose between alternative client implementations:

• @ngx-grpc/grpc-web-client - based on grpc-web
• @ngx-grpc/improbable-eng-grpc-web-client - alternative client implementation based on @improbable-eng/grpc-web
• @ngx-grpc/worker-client - similar to @ngx-grpc/grpc-web-client but running in worker

Generate the code

MacOS / Linux

Add proto:generate script to your package.json scripts section:

{
  "scripts": {
    "proto:generate": "protoc --plugin=protoc-gen-ng=$(which protoc-gen-ng) --ng_out=<OUTPUT_PATH> -I <PROTO_DIR_PATH> <PROTO_FILES>"
  }
}

Where:

• OUTPUT_PATH - the directory your code will be generated at (please ensure the directory exists, otherwise you'll have a protoc error)
• PROTO_DIR_PATH - the root path of your proto files
• PROTO_FILES - list of proto files to use

Example:

{
  "scripts": {
    "proto:generate": "protoc --plugin=protoc-gen-ng=$(which protoc-gen-ng) --ng_out=./src/proto -I ../proto $(find ../proto -iname \"*.proto\")"
  }
}

Finally, run npm run proto:generate every time you want to (re)generate the code

Advanced generator config

You can have more control on what and how is being generated. Create a ngx-grpc.conf.js (.json format also supported) file in your project root.

E.g. to generate well-known types in your project instead of using @ngx-grpc/well-known-types, use this config:

module.exports = {
  embedWellKnownTypes: true,
};

More details on the configuration properties and their default values see here.

Then update your package.json command with path to this file config=./ngx-grpc.conf.js:

{
  "scripts": {
    "proto:generate": "protoc --plugin=protoc-gen-ng=$(which protoc-gen-ng) --ng_out=config=./ngx-grpc.conf.js:./src/proto -I ../proto $(find ../proto -iname \"*.proto\")"
  }
}

Windows

Unfortunately the way to generate files on Windows slightly differs. Here is a sophisticated example that shows how to scan windows folder with proto files and pass it to protoc-gen-ng.

{
  "scripts": {
    "proto:generate:win": "for /f %G in ('dir /b ..\\proto\\*.proto') do grpc_tools_node_protoc --plugin=protoc-gen-ng=.\\node_modules\\.bin\\protoc-gen-ng.cmd --ng_out=.\\output\\path -I ..\\proto ..\\proto\\%G",
  }
}

Usage

Import the required modules

In general, you need to import at least

• GrpcCoreModule
• GrpcWebClientModule (or another client implementation, see below)

@NgModule({
  imports: [
    GrpcCoreModule.forRoot(),
    GrpcWebClientModule.forRoot({
      settings: { host: 'http://localhost:8080' },
    }),
  ],
})
export class AppModule {}

You also can define them in child modules by using forChild() methods instead of forRoot().

Per-service clients configuration

Instead of configuring the client settings globally you can configure them per-service. Every service has an injected configuration which could be found e.g. in the corresponding *.pbconf.ts file.

E.g. for a service TestServiceClient you need to provide the GRPC_TEST_SERVICE_CLIENT_SETTINGS:

@NgModule({
  providers: [
    { provide: GRPC_TEST_SERVICE_CLIENT_SETTINGS, useValue: { host: 'http://localhost:8080' } as GrpcWebClientSettings },
  ],
})
export class AppModule {}

To set grpcweb / binary proto format use

{ provide: GRPC_TEST_SERVICE_CLIENT_SETTINGS, useValue: { host: 'http://localhost:8080', format: 'binary' } as GrpcWebClientSettings },

From now on this particular service is set.

Service client methods

Concept

Every client call accepts a message to be sent and returns an Observable of response message(s). However, the request is not being executed until the Observable gets subscribed, so it can be safely used at any place.

To cancel the request / close the connection simply unsubscribe from that Subscription. Of course, if connection is closed by the server or the error happens, the Observable gets terminated.

class MyService {

  constructor(private client: EchoClient) {}

  sendOne() {
    this.client.echo(new EchoRequest({ message: 'text' })).subscribe(res => console.log(res));
    
    // or if you want to terminate it, e.g. it is a server stream or you navigate away and do not need to wait
    const sub = this.client.echo(new EchoRequest({ message: 'text' })).subscribe(res => console.log(res));

    setTimeout(() => sub.unsubscribe(), 1000); // this closes connection
  }

}

The behaviour above is possible due to the Observable's natural laziness and ability to be terminated.

The server streaming has the same signature as the unary requests, but the returned Observable can emit more than one message and the connection is kept open.

The client streaming is differrent, because it accepts an Observable (and any of its derivatives, such as Subject, BehaviourSubject, etc.) of messages

class MyService {

  constructor(private client: EchoClient) {}

  sendMany() {
    const stream = from(['message 1', 'message 2', 'message 3']);

    this.client.echoMany(stream).subscribe(res => console.log(res));
  }

}

The bidirectional streaming has the same signature as the client's one and is a combination of server and client streaming.

Implementation details

Each RPC has two corresponding methods.

• the first, that emits messages, is a direct method of the service client.
• the second, that emits events, is scoped into service client property $raw.

E.g. for rpc Echo(...) there would be the following:

• echo(...) - returns Observable of messages and throws errors in case of non-zero status codes. This is the most common use-case
• .$raw.echo(...) - returns Observable of GrpcEvents. Events could be of two kinds: GrpcDataEvent containing the message inside and GrpcStatusEvent containing gRPC status response. Apart from the returned data type there is important difference in the behaviour. There are no errors thrown in this stream (by design). All errors are considered to be normal GrpcStatusEvents. Furthermore, this method is the only one where it is anyhow possible to read the gRPC status code 0 (OK) metadata. This method is not that comfortable to use in every place, but it can do things that are not achievable with the method above.

There are two custom RxJS operators that could be used on the stream to make it easier:

• throwStatusErrors - searches for the non-zero status codes and throws them as errors
• takeMessages - searches for the messages

For usage example look at any of your generated .pbsc.ts file. In fact, those two operators turn the raw method into a 'normal' one.

Messages

To create a new message just pass its initial values to the constructor: new Message(myInitialValues). Here is some information on the message's methods:

• constructor - accepts a message or an object with initial values. All values are safely / deeply cloned.
• toObject() - casts message as is to a normal JavaScript object
• toJSON() - convenience method to be able to pass message to JSON.stringify(msg)
• toProtobufJSON() - constructs a protobuf-defined JSON. Accepts an optional GrpcMessagePool (see google.protobuf.Any section) which is required only if the message or some of its descendants embeds google.protobuf.Any

Int64

JavaScript does not support int64, out of the box, that's why all of its kinds are generated as string by default.

You can however override this behavior by passing JS_NUMBER or JS_STRING option to the appropriate field. Example:

message Message { 
  int64 bigInt = 1 [jstype = JS_NUMBER];
  uint64 bigUint = 2 [jstype = JS_NUMBER];
}

Well-known types

The well-known types are served as a separate package. You can also configure generation of the well-known types together with your proto definitions (like older versions did).

Some types have additional functionality, see below.

google.protobuf.Any

The google.protobuf.Any has additional methods pack and unpack.

Unpacking the message requires a special message pool GrpcMessagePool where the expected message types are listed; otherwise the unpacking would not be possible.

Example of type-safe unpacking:

// we expect one of 3 message types to be packed into Any
const myAny: Any;
const pool = new GrpcMessagePool([Empty, Timestamp, MyMessage]);

try {
  switch(myAny.getPackedMessageType(pool)) {
    case Empty: console.log('Empty found', myAny.unpack<Empty>(pool)); break;
    case Timestamp: console.log('Timestamp found', myAny.unpack<Timestamp>(pool)); break;
    case MyMessage: console.log('MyMessage found', myAny.unpack<MyMessage>(pool)); break;
    default: console.log('No packed message inside');
  }
} catch (ex) {
  console.error('Something went wrong, e.g. packed message definition is not in the pool');
}

google.protobuf.Timestamp

The google.protobuf.Timestamp has additional methods to cast from / to Date and ISO string date representation.

Custom well-known types

For well-known types that are not part of the google.protobuf package, you can override the imports to use for specific packages.

This is especially useful if you are using a protobuf schema registry like Buf for sharing some common messages through different projects.

Example of a custom well-known type configuration:

module.exports = {
  customWellKnownTypes: {
    "company.internal.commons": "@company-internal/grpc-commons"
  }
}

This will change all import statements that reference a message of the package company.internal.commons to use @company-internal/grpc-commons, instead of the relative file path.

If the embedWellKnownTypes configuration is enabled, the customWellKnownTypes configuration will be ignored and the messages will be generated as usual.

Interceptors

You can add global interceptors to all gRPC calls like Angular's built-in HttpClient interceptors.

The important differences

• unlike HttpClient interceptors GrpcInterceptors need to work with event streams; there are no errors thrown. Instead you should listen to the GrpcStatusEvent and decide whether it is an error or not. Please keep this in mind
• the incoming data can be a message or a stream of messages (in case of client streaming)

As an example see GrpcLoggerInterceptor in the core package.

Logger

You can enable logging using GrpcLoggerInterceptor (provided by @ngx-grpc/core). Add to your AppModule the following import:

GrpcLoggerModule.forRoot(),

Then open the browser console and you should see all the requests and responses in a readable format.

Optionally, you can provide a more detailed configuration. Example:

GrpcLoggerModule.forRoot({ 
  settings: { 
     // enables logger in dev mode and still lets you see them in production when running `localStorage.setItem('logger', 'true') in the console`
    enabled: localStorage.getItem('logger') === 'true' || !environment.production,
     // protobuf json is more human-readable than the default toObject() mapping
     // please beware: if you use google.protobuf.Any you must pass the proper `messagePool` argument
    requestMapper: (msg: GrpcMessage) => msg.toProtobufJSON(),
    responseMapper: (msg: GrpcMessage) => msg.toProtobufJSON(),
  },
}),

Alternative client: @improbable-eng/grpc-web

The alternative grpc-web implementation from Improbable Engineering provides way more features than standard grpc-web from Google. It supports various transports including WebSocket-based and even Node (can be useful e.g. for SSR).

The only client that supports client / bidirectional streaming. This however also requires the server to be able to handle websocket transport. For this purpose improbable-eng team introduced grpc-web-proxy - a special facade for the normal grpc server that acts like envoy, but has also the ability to handle websocket transport.

Installation:

npm i -S @ngx-grpc/improbable-eng-grpc-web-client @improbable-eng/grpc-web

Then configuration is similar to the other clients, however there is a transport to configure:

import { grpc } from '@improbable-eng/grpc-web';
import { GrpcCoreModule } from '@ngx-grpc/core';
import { ImprobableEngGrpcWebClientModule } from '@ngx-grpc/improbable-eng-grpc-web-client';

const xhr = grpc.CrossBrowserHttpTransport({});
const ws = grpc.WebsocketTransport();

@NgModule({
  imports: [
    GrpcCoreModule.forRoot(),
    ImprobableEngGrpcWebClientModule.forChild({
      settings: {
        host: 'http://localhost:8080',
        // we might want to use different transports as recommended by improbable-eng team
        // because websocket transport acts a bit differently and is intended for client streaming only
        transport: {
          unary: xhr,
          serverStream: xhr,
          clientStream: ws,
          bidiStream: ws,
        },
        // or simply e.g.
        // transport: ws, // to configure all methods to use websockets
      },
    }),
  ],
})
export class AppModule {}

Web worker

Web worker allows to run gRPC clients, messages serialization and deserialization in a separate thread. It might give some performance benefits on large data sets; however the main reason of the worker is to avoid blocking the main thread. That means that rendering engine has more resources to work on rendering while the messages processing is done in parallel.

First, install additional packages:

npm i -S @ngx-grpc/worker @ngx-grpc/worker-client

Then configure the web worker. First you need to adapt the code generation settings (see above) to generate pbwsc files. These files will contain the worker service client definitions.

Now, generate the worker (angular cli), e.g. with the name grpc:

ng g web-worker grpc

or for Angular < 9

ng g worker grpc

You should see grpc.worker.ts close to your app.module.ts. Open this file and replace the contents with the following:

/// <reference lib="webworker" />

import { GrpcWorker } from '@ngx-grpc/worker';
import { GrpcWorkerEchoServiceClientDef } from '../proto/echo.pbwsc';

const worker = new GrpcWorker();

worker.register(
  // register here all the service clients definitions
  GrpcWorkerEchoServiceClientDef,
);

worker.start();

Finally use the following imports:

@NgModule({
  imports: [
    GrpcCoreModule.forRoot(),
    GrpcWorkerClientModule.forRoot({
      worker: new Worker('./grpc.worker', { type: 'module' }),
      settings: { host: 'http://localhost:8080' },
    }),
  ],
})
export class AppModule {
}

That's it. All your requests are served by worker.

Not implemented (yet)

• Proto 2 Extensions

Contributing

• to run tests on Apple m1 chips use npm ci --no-optional and brew install protoc-gen-grpc-web

License

MIT