This is a Go application that consumes messages from Kafka, which are produced by a Java application.
The Java application inserts data into a database and publishes these insert operations as messages to a Kafka topic. The Go application uses the github.com/segmentio/kafka-go library to read messages from the Kafka cluster. It has a retry mechanism for handling read and processing errors, using separate goroutines and queues (ReadRetryQueue and ProcessRetryQueue) for each operation.
When a message is successfully read from Kafka, it is sent to a ProcessRetryHandler implementation for processing. If a read error occurs, the message is added to the ReadRetryQueue, and a separate goroutine continually attempts to read it again until successful.
If a processing error occurs, the message is added to the ProcessRetryQueue, and another goroutine attempts to process it again until successful. Between each retry attempt, there is a backoff period determined by backoff.NextBackOff(). Retries continue until a maximum number of retries (options.MaxRetries) is reached.
The order of messages is not important in this application. If message ordering is crucial, multiple consumers should be used. Both read and processing errors allow the main topic to continue being read (thanks to goroutines).
In case of a read error, the message is sent to the ReadRetryQueue and retried until successful. When successful, it is sent for processing. In case of a processing error, the message is sent to the ProcessRetryQueue and retried until successful.
New goroutines are not spawned for each new message. Instead, existing goroutines process the queued elements when errors occur. The backoff period is observed between retries, but a successful retry is not guaranteed (for both read and processing errors).
Messages are retried for reading or processing until successful (only processing errors are simulated in the provided code). This application leverages Go's concurrency features (goroutines and channels) to provide a robust retry mechanism for handling Kafka messages produced by a Java application, ensuring fault tolerance and reliable message processing.