redis-field-engineering/redis-kafka-connect

No retry logic on poll()/put() for both source/sink

Closed this issue · 1 comments

DNewt commented

Currently if Redis isn't available or there is an intermittent network issue both the Sink and Source connector throw errors such as

org.apache.kafka.connect.errors.ConnectException: Could not read messages from stream at com.redis.kafka.connect.source.StreamSourceRecordReader.poll(StreamSourceRecordReader.java:109) at com.redis.kafka.connect.source.RedisSourceTask.poll(RedisSourceTask.java:113) .... Caused by: io.lettuce.core.RedisCommandTimeoutException: Command timed out after 1 minute

Which completely stops the connectors and only a restart of the task will fix it.

For those more experienced with this connector, would setting redis.timeout in the connector config potentially fix this issue? Are there other settings I may have missed?

Does this connector need some retry logic wrapping the poll() and put() function calls (Source/Sink)?

jruaux commented

Thanks for reporting this. I added some retry logic for both sink and source ee0ffd1