Upgrade to Confluent.Kafka 1.5.2
amotl opened this issue · 8 comments
Dear @TsuyoshiUshio, @fbeltrao and @ryancrawcour,
as I see you have upgraded to Confluent.Kafka 1.4.3 within #151 the other day, I would like to humbly point out confluentinc/librdkafka#3109 to you.
Without using the Azure Kafka bindings yet, we and others tripped into different issues when connecting to Event Hubs with librdkafka <1.5.0 through its Python binding confluent-kafka-python when running within a Kubernetes environment on Azure.
Now, as I believe the same things might also apply to the Azure Kafka bindings, I wanted to give you a heads up on this.
Without any offense to you or the networking infrastructure within Azure, citing others:
TL;DR: Azure has a nasty artificial limitation that results in being unable to use long-lived TCP connections that have >= 4 minutes of radio silence at any given point.
After some issues revolving around that had been popping up on the Confluent's issue trackers, @edenhill was able to add some mitigations to librdkafka just recently.
Apart from upgrading to the most recent version of librdkafka, I would like to emphasize that it is probably also crucial to apply the recommended configuration settings for Kafka on Azure as outlined within confluentinc/librdkafka#3109 (comment).
After applying these measures to our systems, things have improved tremendously.
With kind regards,
Andreas.
Thank you for sharing it, @amotl
Absolutely, I'd like to upgrade it once it comes. I'd like to upgrade the code to adopt the recommended configuration for Azure.
I appreciate your contribution. I also might ping him to heads up the intermediate certificate issue.
Dear Tsuyoshi,
librdkafka 1.5.2 GA has just been released, see [1]. For making progress on that, we will probably have to wait for the bump propagating to confluent-kafka-dotnet and being available on NuGet [2].
With kind regards,
Andreas.
[1] https://github.com/edenhill/librdkafka/releases/tag/v1.5.2
[2] https://www.nuget.org/packages/Confluent.Kafka/
Thanks. It's a good new. Once it is released. I'll release this library with it.
Once it is released, I'll release this library with it.
Thanks. @mhowlett is on it, see confluentinc/confluent-kafka-dotnet#1432.
Confluent.Kafka 1.5.2 has been released, see https://www.nuget.org/packages/Confluent.Kafka/. Thanks a bunch, @mhowlett and @edenhill!
It's a good news I'll include it
Sounds good? #186
Thanks a again for considering to upgrade to 1.5.2! Closing this now.