awslabs/amazon-kinesis-producer

Explicit dependency on libidn shared library?

Closed this issue · 4 comments

Hi,

In the process of updating our kpl version from 0.12.3 to 0.12.5, we noticed that some of our tests started failing. Upon a bit of digging, we saw that it was due to an inability to load the libidn library:

LogInputStreamReader - /tmp/amazon-kinesis-producer-native-binaries/kinesis_producer_d93825f806782576ef9f09eef67a2baeadfec35c: error while loading shared libraries: libidn.so.11: cannot open shared object file: No such file or directory None

I didn't see mention of this library in the README, and the AWS documentation on supported platforms says simply Linux distributions with kernel 2.6.18 (September 2006) and later. It's worth noting that our tests happen to run on a Docker image which is ultimately based on an Alpine Linux base. Moreover, upon manually installing libidn on that container, the tests then pass.

I see that PR #157 may be related, but I'm not clear on how. What I'm trying to understand is whether this dependency was, in fact, added somewhere between 0.12.3 and 0.12.5 (or whether we have some other issue in our environment), and if so, whether we need to be advising our customers (who may be installing on a variety of platforms) to install this library if they intend to use our Kinesis producer functionality (which ultimately uses kpl). Thanks for any insight!

Nevermind, just saw issue #86. Will close this one for now.

This isn't actually related to #86, instead it's a mistake on my part. I forgot to remove the libidn configuration. It's currently unused, and accidentally got included as a dynamically linked library. I'm currently working on a release of the KPL which will remove the libidn dependency, and will have the correct static version of libuuid.

I'm going to accelerate my work on that release.

To answer why #157 fixes it:
When I build the KPL I need to ensure that static versions of most all the dependencies exist, normally that is accomplished by the build scripts, but for some temporary dependencies I've been manually managing them. In this case the AWS C++ SDK was built to no longer require libidn, but the dependency wasn't correctly removed at that time. When the application was linked the build system happened to find the dynamic, and linked it.

This was fixed in 0.12.7, feel free to reopen if you have further questions, or issues.