API / transport protocol

[Deleted user]

We want a transport protocol that allows for 1) querying the schema of a plugin 2) running a step with a specified input 3) querying the result of a plugin run.

Requirements

• The protocol must function over standard input/output.
• The protocol must not assume that there will always be a party listening to output. The engine will reconnect when it is time for it to fetch the results, but may disconnect intermittently if the API becomes unavailable.
• The protocol must be strictly typed.
• The protocol must be able to transport mixed data types and convey null values.

Options

Protobuf

Protobuf is the underlying protocol of gRPC. However, there is very little documentation on how to create custom RPC handlers, which makes it difficult to use. It is also not self-delimiting, which will require us to add some sort of framing for messages.

CBOR

CBOR is less efficient than Protobuf, but has the advantage of being self-describing and self-delimiting. As a tradeoff, it is less well known and is also less efficient than Protobuf in terms of compactness.