Fluent-bit failed to connecting to the Kafka brokers using Self Signed certs
rameshar16 opened this issue · 3 comments
Describe the bug
Fluent-bit failed to connecting to the Kafka brokers using Self Signed certs.
[2022/08/10 20:26:54] [error] [output:kafka:kafka.1] fluent-bit#producer-2: [thrd:ssl://kafka2-xxxxxxx:xxxx/bootstrap]: ssl://xxxxxxxxx:xxxxxx/bootstrap: SSL handshake failed: ../ssl/statem/statem_clnt.c:1914: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 24ms in state SSL_HANDSHAKE)
To Reproduce
Generate SSL certs and configured the below SSL configs.
[OUTPUT]
Name kafka
Match serverlog*
Brokers xxxxxxxxxxxxx
Topics logtopic
rdkafka.debug All
rdkafka.enable.ssl.certificate.verification true
rdkafka.ssl.certificate.location /certs/cert/fluent-bit.cert
rdkafka.ssl.key.location /certs/key/fluent-bit.key
rdkafka.ssl.ca.location /certs/ca-cert/fluent-bit-ca.cert
rdkafka.security.protocol ssl
rdkafka.request.required.acks 1
rdkafka.log.connection.close false
rdkafka.metadata.broker.list xxxxxxxxxxxxx
Expected behavior
Fluentbit should be able to connect to the Kafka brokers over SSL.
Your Environment
I am using the "cr.fluentbit.io/fluent/fluent-bit:latest" fluent-bit image.
Generate SSL certs and configured the below SSL configs.
[OUTPUT]
Name kafka
Match serverlog*
Brokers xxxxxxxxxxxxx
Topics logtopic
rdkafka.debug All
rdkafka.enable.ssl.certificate.verification true
rdkafka.ssl.certificate.location /certs/cert/fluent-bit.cert
rdkafka.ssl.key.location /certs/key/fluent-bit.key
rdkafka.ssl.ca.location /certs/ca-cert/fluent-bit-ca.cert
rdkafka.security.protocol ssl
rdkafka.request.required.acks 1
rdkafka.log.connection.close false
rdkafka.metadata.broker.list xxxxxxxxxxxxx
Your Configuration
I am using the "cr.fluentbit.io/fluent/fluent-bit:latest" fluent-bit image.
Your Error Log
[2022/08/10 20:26:54] [error] [output:kafka:kafka.1] fluent-bit#producer-2: [thrd:ssl://kafka2-xxxxxxx:xxxx/bootstrap]: ssl://xxxxxxxxx:xxxxxx/bootstrap: SSL handshake failed: ../ssl/statem/statem_clnt.c:1914: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 24ms in state SSL_HANDSHAKE)
Additional context
[2022/08/10 20:26:54] [error] [output:kafka:kafka.1] fluent-bit#producer-2: [thrd:ssl://kafka2-xxxxxxx:xxxx/bootstrap]: ssl://xxxxxxxxx:xxxxxx/bootstrap: SSL handshake failed: ../ssl/statem/statem_clnt.c:1914: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 24ms in state SSL_HANDSHAKE)
Hey @rameshar16 your issue will be better served in the fluentbit repository (https://github.com/fluent/fluent-bit/issues) as this repo is for fluentd.
Please forward your report to https://github.com/fluent/fluent-bit/issues