Proposal: Real-World Examples and Setup for Long-Lived Services

[mtmk]

Problem Statement:
Users coming from other messaging platforms (e.g. RabbitMQ, Masstransit) might find it challenging to register NATS constructs (e.g. consumers) and process messages in the current setup. Most examples and samples provided are overly simplistic, often showcasing a single consumer in a console application. There is a need for more realistic examples, particularly for long-lived services with multiple consumers.

Proposed Enhancements:

• Provide More Real-World Examples: Create examples that demonstrate the setup of multiple consumers within a long-lived service. Include scenarios that show how to handle faults and ensure the resilience of the service.
• Improve Documentation: Expand the documentation to cover dependency injection (and others e.g. OTEL?) in a .NET hosting context. Provide detailed guides and best practices for setting up and managing multiple consumers efficiently.
• Propose Extension: We have dependency injection extensions, for example. Maybe we can come up with other extensions to the NATS .NET ecosystem (managed by nats.io or other projects).

Benefits:

• Eases the transition for users familiar with other messaging systems to adopt and implement the current technology.
• Enhances the reliability and resilience of long-lived services using the provided examples.
• Improves overall user experience by reducing the complexity of setting up NATS in real-world applications.

[Kunal-Thakur]

I am facing a similar problem while running multiple NATS jetstream consumers as background operations in .net webapi. Feel like dependency injection operations would also be difficult in this. Comming from Masstransit RabbitMq world implementing these operations are quite easy over there.

[robertmircea]

Suggestions for samples:

• NATS and Jetstream samples
• show resilient consumers which survive network interruptions and recover in case of errors
• handle deserialization errors
• durable vs ephemeral consumers
• detect and handle deliver retries (idempotency in consumer)
• batch processing: I want to accumulate x messages at once (or after a timeout interval elapses) from a ConsumerAsync loop and send them all as a batch for processing and then acknowledge them.
• WorkQueue vs regular streams
• show differences between AckAsync, NackAsync (with or without delay) and AckTerminateAsync. Show dead letter queue pattern. Intercept/process from Jetstream advisories messages which were not acknowledge by consumer.
• NatsConnection vs NatsConnectionPool - when to use either one or the other
• Nats KV Store (e.g. samples for leader election, app configuration set, reload from KV object, use KV store to "register" apps and maintain a list of apps for service discovery - e.g. x instances active for the same app on nodes A,B,C etc and which keep a heartbeat)
• Nats object store real world examples.

[to11mtm]

I just came up with a fun/useful idea!

General thought, what if we could use Jetstream and/or KV (I'm thinking primarily KV here) for implementing a JWT jti token validator?

As a background, jti is an optional field in JWTs intended to prevent replay of a token; if included it is expected to be used only once across the auth context.

My general thought being, a service could take in JTIs along with an expiration time, and then return whether the JTI has been used or not. The assumption being, the consumer will be pre-checking expiration as part of normal JWT flow, so one could set an expiration time on the buicket of the ceiling for JWT lifetime.

I'm pretty sure it would be fairly easy to wire up, also could be expandable to something like 'use NATS in ASPNETCORE middleware' case.

(I'm willing to give this a crack but if someone else wants to beat me to it, all the better.)

[Jarema]

Those are some great examples to consider!
Some are very .NET specific, but most are general NATS challenges, that can be fact by using any client.
I would love if those ideas were somehow realized in a way, that provides a solution in each major client.

Maybe a section in https://natsbyexample.com that is focused on Real-World examples would be a good idea, maybe some other medium.