Add tooling for create sharded streams
AbstractiveNord opened this issue · 3 comments
Proposed change
Add a feature 'sharded' for command 'stream add'. Idea is all about modification user subject by adding partition number into subject and internally create a requested count of stream-shards, also with subject mapping configuration for nats servers. For example, nats stream add streamname --shards=100.
Use case
Everytime when you want to split load or utilize more servers for storing data in Jetstream streams.
Contribution
Yes.
It's a good idea but we'll deliver the broad idea in another way, will close now as its not really a CLI-only thing but would be much more cross client work
I think it's more for some type of controller on top of NATS. Since NATS runtime, NEX, is arrived, it's looks possible.
We are doing work where the data at rest will be unpartitions, consumers will dynamically partition data and support handling pinned clients, failover clients, scale out etc to do a proper server assisted feature set around this that isnt so static or altering the data on disk