Ability for a Consumer to bypass retries and DLQ a terminally failing message
jcass8695 opened this issue · 3 comments
Is your feature request related to a problem? Please describe.
If I consume a message and fail during the processing of that message in a terminal way, I know that I don't want to retry it, I want to send it directly to the DLQ. Currently, if I wanted a persistent record of the message, I would either have to
- ACK the message and log it, write it to a database etc.
- NACK the message and retry it up to the configured max retries, when it would then land in the DLQ.
Describe the solution you'd like
I would like a consumer to be able to signal a message to be routed directly to the DLQ.
Describe alternatives you've considered
See options 1 & 2 above.
Additional context
Add any other context or screenshots about the feature request here.
I think we could add an interceptor in the DLQPolicy so that we can let users choose whether each message need to be retried. Does it make sense to you?
Before implementing this, we need to follow the PIP process and get approval. Are you willing to lead this discussion? Would you like to start by sending the discussion to the mailing list?
FYI, you can find the PIP process guide here: https://github.com/apache/pulsar/blob/master/pip/README.md
I'm happy to lead the discussion on this. I agree that an interceptor in DLQPolicy looks to be a good route forward. I should have time to dedicate to it by the end of the week.