understanding the deletion of datastream and recreating

[Chandru0211]

Need to understand what will happen when trying to delete the datastream and recreate the same datastream.

Does it try to reprocess all the messages which already been replicated?
Since the offset of the group was maintained in the brooklin Kafka... will it resume the replicate only remaining messages?

[thomaslaw]

Hey,

If you keep .["metadata"]["group.id"] as the same value in the new datastream's metadata as the old/deleted one, it will use the same underlying consumer group in Kafka when it begins processing.

This will pick up the existing checkpoints as long as Kafka has not cleaned up stale consumer groups (default 7 days).

Please let us know if you have any other questions.

Thanks,
Thomas
Brooklin SRE