bwlogsender error when using kafka
Closed this issue · 3 comments
We are using kafka in our setup, and all of the BS servers are sending fine, except for a single pair of AS, which had the security hardening script run on them (our other AS do not have it and work fine). If we send the files via the sender/receiver, the traffic is transferred fine, but whenever we enable kafka we get the below error, thousands of times within minutes. I turned on sender debug and restarted but I didn't get any other logs which could point me in the right direction. The Sender/Receiver setup would work temporarily, but we know we need kafka once the server starts handling production traffic or else we will drop traffic.
2019-10-02_10:55:55.564 [Producer Thread #0] INFO c.b.kafka.common.BaseKafkaProducer - Generic Exception Publishing data to Kafka
org.apache.kafka.common.KafkaException: java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy
at org.apache.kafka.common.record.CompressionType$3.wrapForOutput(CompressionType.java:75)
at org.apache.kafka.common.record.MemoryRecordsBuilder.(MemoryRecordsBuilder.java:122)
at org.apache.kafka.common.record.MemoryRecordsBuilder.(MemoryRecordsBuilder.java:158)
at org.apache.kafka.common.record.MemoryRecords.builder(MemoryRecords.java:474)
at org.apache.kafka.common.record.MemoryRecords.builder(MemoryRecords.java:458)
at org.apache.kafka.common.record.MemoryRecords.builder(MemoryRecords.java:406)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.recordsBuilder(RecordAccumulator.java:244)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.append(RecordAccumulator.java:220)
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:806)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:760)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:648)
at com.broadsoft.kafka.common.BaseKafkaProducer.publishData(BaseKafkaProducer.java:144)
at com.broadsoft.kafka.common.BaseKafkaProducer.write(BaseKafkaProducer.java:109)
at com.broadsoft.kafka.common.BaseKafkaProducer.run(BaseKafkaProducer.java:81)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy
at org.xerial.snappy.SnappyOutputStream.(SnappyOutputStream.java:97)
at org.xerial.snappy.SnappyOutputStream.(SnappyOutputStream.java:89)
at org.xerial.snappy.SnappyOutputStream.(SnappyOutputStream.java:79)
at org.apache.kafka.common.record.CompressionType$3.wrapForOutput(CompressionType.java:73)
... 14 common frames omitted
just found a network server that was not hardened, but has the same error, so it must be something else.
If you look at your mounts - is there a noexec option on one of the tmp mounts? Specifically see if there is a diff between a server that is working and one that isn't.
That was it, that security script is a real pain.