Support for Apache Pulsar

[RobertIndie]

Support for Apache Pulsar

[blinktag]

Hi @RobertIndie, Pulsar is currently supported in Plumber. Are you having a specific issue using it?

$ plumber read pulsar -h
Usage: plumber read pulsar --dsn="pulsar://localhost:6650" --topic=STRING --subscription-name=STRING

Apache Pulsar

Flags:
  -h, --help                                   Show context-sensitive help.
  -d, --debug                                  Enable debug output
  -q, --quiet                                  Disable fancy output (like curl -s)
      --version                                Display plumber version

  -f, --continuous                             Follow/read continuously
      --convert-output=unset                   Convert output before it is printed to STDOUT (options: unset; base64; gzip)
      --verbose-output                         Display more verbose information during reads (varies by backend)
      --pretty                                 Pretty (colorized) output
      --json                                   Display all output as JSON (including tabular metadata)

      --dsn="pulsar://localhost:6650"          Full DSN to connect to Pulsar
      --connect-timeout-seconds=10             Connection timeout
      --tls-skip-verify                        Whether to verify server certificate
      --tls-client-cert=TLS-CLIENT-CERT,...    TLS client certificate file
      --tls-client-key=TLS-CLIENT-KEY,...      TLS client key file
      --topic=STRING                           Topic to read messages from
      --subscription-name=STRING               Subscription name
      --subscription-type=shared               Subscription type (options: shared exclusive failover keyshared)

sampling
  --sample-rate=100               How many events to fetch during a sample interval
  --sample-interval-seconds=60    Sample interval seconds

decode
  --decode-type=unset    Decode type (options: unset, protobuf, avro, thrift, flatbuffer)

protobuf
  --protobuf-root-message=STRING            Input message(s) should be encoded with this message envelope
  --protobuf-dirs=PROTOBUF-DIRS,...         One or more directories which contains protobuf schemas
  --protobuf-envelope-type=deep             Envelope type (options: deep, shallow)
  --shallow-envelope-message=STRING         For shallow envelope messages, the payload field should be encoded with this message name
  --shallow-envelope-field-number=INT-32    For shallow envelope messages, the field number of the root message that contains the shallow envelope payload

avro
  --avro-schema-file=STRING    If encode-type is set to avro, must specify avro schema file

[RobertIndie]

Thanks, @blinktag. Sorry, I didn't notice it before. It's good to see plumber supports pulsar.

[duc2h]

hi guys, i try to use plumber to read payload from nats-jetstream, my decode-type=protobuf but i dont know assign what value for --protobuf-root-message and protobuf-dirs. do you have an example for this one?

[blinktag]

Hi @edarha,

For --protobuf-root-message, you specify the name of the protobuf message type that you are trying to read and decode.
For --protobuf-dirs this is the directory containing your .proto definition files

For example this message:
https://github.com/batchcorp/collector-schemas/blob/main/protos/events/outbound.proto
You would specify --protobuf-root-message events.Outbound --protobuf-dirs ~/path/to/protos/events/
Then plumber would be able to decode the binary data from your message bus into the events.Outbound message type and display it to you