sky-uk/kafka-message-scheduler

Hanging whilst loading state on startup

lacarvalho91 opened this issue · 1 comments

{"@timestamp":"2019-01-11T06:19:47.137+00:00","@version":"1","message":"Finished loading data from map.schedules-3","logger_name":"com.sky.kafka.topicloader.TopicLoader$","thread_name":"kafka-message-scheduler-akka.actor.default-dispatcher-56","level":"INFO","level_value":20000}
{"@timestamp":"2019-01-11T06:19:47.528+00:00","@version":"1","message":"Finished loading data from map.schedules-8","logger_name":"com.sky.kafka.topicloader.TopicLoader$","thread_name":"kafka-message-scheduler-akka.actor.default-dispatcher-47","level":"INFO","level_value":20000}
{"@timestamp":"2019-01-11T06:19:50.652+00:00","@version":"1","message":"Finished loading data from map.schedules-9","logger_name":"com.sky.kafka.topicloader.TopicLoader$","thread_name":"kafka-message-scheduler-akka.actor.default-dispatcher-32","level":"INFO","level_value":20000}
{"@timestamp":"2019-01-11T06:19:52.351+00:00","@version":"1","message":"Finished loading data from map.schedules-4","logger_name":"com.sky.kafka.topicloader.TopicLoader$","thread_name":"kafka-message-scheduler-akka.actor.default-dispatcher-24","level":"INFO","level_value":20000}
{"@timestamp":"2019-01-11T06:19:58.265+00:00","@version":"1","message":"Finished loading data from map.schedules-6","logger_name":"com.sky.kafka.topicloader.TopicLoader$","thread_name":"kafka-message-scheduler-akka.actor.default-dispatcher-56","level":"INFO","level_value":20000}
{"@timestamp":"2019-01-11T06:20:03.340+00:00","@version":"1","message":"Finished loading data from map.schedules-2","logger_name":"com.sky.kafka.topicloader.TopicLoader$","thread_name":"kafka-message-scheduler-akka.actor.default-dispatcher-53","level":"INFO","level_value":20000}
{"@timestamp":"2019-01-11T06:20:03.983+00:00","@version":"1","message":"Finished loading data from map.schedules-7","logger_name":"com.sky.kafka.topicloader.TopicLoader$","thread_name":"kafka-message-scheduler-akka.actor.default-dispatcher-57","level":"INFO","level_value":20000}
{"@timestamp":"2019-01-11T06:24:08.288+00:00","@version":"1","message":"Cluster ID: ox_4ySQFTaKqyqxn9mLsMQ","logger_name":"org.apache.kafka.clients.Metadata","thread_name":"kafka-producer-network-thread | producer-1","level":"INFO","level_value":20000}

Looks like this happens when current offset lag is negative...which is something that should not happen but we have seen it happen when Kafka is in a bad state