Create GracefulShutdown utility
Closed this issue · 1 comments
https://github.com/fd4s/fs2-kafka/blob/series/1.x/docs/src/main/mdoc/consumers.md#graceful-shutdown has an example of graceful shutdown code, but it's quite a bit of code. I imagine a lot of users might not be aware of it and consider a KafkaConsumer
a Resource
that will be shutdown gracefully.
The docs state:
stopConsuming
is just a building block for making your own graceful shutdown, not a ready-made solution for all needs. This design is intentional, because different applications may need different graceful shutdown logic.
However we could extract the code in the example as a base for simple-shutdown logic, and develop it for more complex situations later.
I imagine the most difficult part will be the tests, and proving that code has been shutdown in the correct order.
Hi @agustafson. I understand your point and agree with it. Yes, the current solution is pretty complex and low level. And I explicitly mentioned this in the pull request:
About design decisions. This method is pretty low-level and not opinionated. It's a building block for making your own graceful shutdown, not a ready-made solution. At first, I tried to add some high-level graceful shutdown functionality. But I found that with the current library design it would be problematic in terms of API. All my experiments end with pretty complex and at the same time not very extensible implementations. So, I decided to provide only a low-level API with detailed documentation.
The main problem for a high-level graceful shutdown is that you need to mark the user stream as uncancelable
(point 7 in the docs). You just can't do this from the library API.
I imagine a lot of users might not be aware of it and consider a KafkaConsumer a Resource that will be shut down gracefully.
Not sure what you mean here. KafkaConsumer
is a Resource
and it follows its rules and guarantees. But consumer.stream
not. And graceful shutdown through stopConsuming
gives you a way to shut down both consumer and stream gracefully.
You can think about this functionality like about Resource#allocated
method. In most situations you don't need it, you could just call use
on your resource. But for some complex or custom scenarios, you will need some low-level handling of your resource. So, stopConsuming
for KafkaConsumer
is like allocated
for Resource
.
Anyway, If you will end up with some simple and extensible high-level API for graceful shutdown it would be very cool.