Yolean/kubernetes-kafka

Spring Kafka producers throwing TimeoutExceptions

Opened this issue · 1 comments

Oduig commented

Problem description

Spring Kafka SSL producers are throwing timeouts when producing to a Kafka cluster with 3 brokers. I set up the cluster using this repo, altering only the broker config and services to arrange outside access.

Some observations:

  • The SSL configuration seems correct, since messages are sent and received for large periods of time (e.g. hours) before showing intermittent problems.
  • Both the server and clients have plenty of resources left.
  • The network is stable and powerful enough (200 mbit, no downtime in the last year).
  • The errors do not appear in a PLAINTEXT setup.
  • Configurations for the three brokers are identical apart from their broker ID and listeners.
  • No errors appear in the broker log or Spring Kafka consumers.

2018-06-06 11:15:44.103 ERROR 1 --- [ad | producer-1] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='null' and payload='[...redacted...]': org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for topicname-0: 30001 ms has passed since last append

Detailed description

https://stackoverflow.com/questions/50725643/spring-kafka-producers-throwing-timeoutexceptions

Question

How can I determine if this is a problem on my end, or a bug somewhere in the toolchain?
Has anyone tried installing an SSL outside/PLAINTEXT inside Kafka cluster before?

solsson commented

The configuration given in the SO thread is the best example so far in this repo of a working SSL setup. Maybe given some time someone else will be able to try it.

Have you tried other clients? When I have client issues I tend to try kafkacat with -d broker. Don't know how it can be used with SSL though.