Comcast/sirius

AskTimeoutExceptions during startup

jsnikeris opened this issue · 0 comments

We sometimes see akka.pattern.AskTimeoutException during startup, and generally a restart fixes the problem or the timeouts will occur several times and then cease.
The timeouts occur while executing:

sirius.enqueuePut(event.getKey().fullKey(), data).get();

As you can see, we enqueuePut, and then immediately block until the future returns. Here is a thread dump I took while waiting for the future.

The main thread is waiting for the future, but there don't seem to be any other threads doing anything sirius related. I had expected to see our main thread waiting on a sirius implementation thread to finish its work, but I don't see any threads like that.

This node's sirius cluster config only contained /user/sirius.

This is happening at startup, but after com.comcast.xfinity.sirius.api.Sirius.isOnline() returns true. Sometime around then, we see this in the logs:

2017-07-14 19:46:24,297 WARN s=localListingInfoWebService-root_out env="ape" [sirius-system-akka.actor.default-dispatcher-12] Sirius(akka://sirius-system): SiriusSupervisor Actor received unrecognized message IsInitializedResponse(true)

Perhaps this warning is relevant to the issue.