awslabs/amazon-kinesis-producer

dyld: lazy symbol binding failed: Symbol not found: _clock_gettime

BenRomberg opened this issue · 6 comments

We're getting the following error from the binary when trying to add a record to a stream (via KinesisProducer.addUserRecord) using amazon-kinesis-producer 0.12.4 or 0.12.5:

15:53:06.730 [kpl-daemon-0003] WARN  c.a.s.k.p.LogInputStreamReader - dyld: lazy symbol binding failed: Symbol not found: _clock_gettime
15:53:06.730 [kpl-daemon-0003] WARN  c.a.s.k.p.LogInputStreamReader -   Referenced from: /tmp/bfce9c0d-2028-4815-bc6e-7aeb63582f0b/amazon-kinesis-producer-native-binaries/kinesis_producer_db53b066fadea1904ec9eea479f386f5868a9fda (which was built for Mac OS X 10.12)
15:53:06.730 [kpl-daemon-0003] WARN  c.a.s.k.p.LogInputStreamReader -   Expected in: /usr/lib/libSystem.B.dylib

This only happens on some systems:

  • It fails on MacOSX 10.11.5 and 10.11.6.
  • It works on MacOSX 10.12 and Ubuntu 16.10 with kernel 4.8.0

The producer subsequently fails with the following log messages. Due to the binding failure, it probably fails to read from the input channel, which leads to the updateCredentials task to be interrupted while the ExecutorService is being terminated:

13:31:58.131 [kpl-daemon-0007] WARN com.amazonaws.services.kinesis.producer.Daemon - Exception during updateCredentials
java.lang.InterruptedException: sleep interrupted
	at java.lang.Thread.sleep(Native Method)
	at com.amazonaws.services.kinesis.producer.Daemon$5.run(Daemon.java:321)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
13:31:58.131 [kpl-daemon-0005] ERROR com.amazonaws.services.kinesis.producer.KinesisProducer - Error in child process
java.lang.RuntimeException: EOF reached during read
	at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:521)
	at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:497)
	at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:493)
	at com.amazonaws.services.kinesis.producer.Daemon.readSome(Daemon.java:542)
	at com.amazonaws.services.kinesis.producer.Daemon.receiveMessage(Daemon.java:246)
	at com.amazonaws.services.kinesis.producer.Daemon.access$500(Daemon.java:63)
	at com.amazonaws.services.kinesis.producer.Daemon$3.run(Daemon.java:301)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Using version 0.12.3 we don't have this issue. Looking through the commits since 0.12.3, the issue was probably introduced with this commit (although I'm not a C++ expert): 766bac1

@pfifer any help would be greatly appreciated. Let me know if any more details are required. Thanks!

We resolved this ourselves by upgrading all build and developer machines to MacOS 10.12. It may make sense to document in the readme that this library needs at least MacOS 10.12 to run on a Mac.

I am experiencing this issue on OS X 10.11.6 with the 0.12.6 release of the Amazon Kinesis Producer Library as well. I have tried releases 0.12.6, 0.12.5, 0.12.4, and 0.12.3 and 0.12.3 is the only release that does not have this issue. Releases 0.12.6, 0.12.5, and 0.12.4 all fail with the _clock_gettime symbol not being found (as described above).

This is related to a change with that Apple made that added clock_gettime. The headers don't honor the setting of macosx-version-min for the method. This causes Boost to pick it up, and use it. There is a fix in Boost to deal with it, but it requires a newer version of Boost. I'm looking at upgrading all the dependencies to current versions, which would fix this.

This is fixed in #159, sorry for the delay.

I've verified that it on 10.11, and will be starting a release cycle soon

I can confirm that the 0.12.7 release resolves this issue on OS X 10.11.6. Thank you for fixing this.

Closing this, thanks for reporting the issue.