Release-207 intergration test:kafka systemctl status failed and workaround
Closed this issue · 3 comments
yaoyao12138 commented
kafka systemctl status failed:
root@yao-instana-vm-32-instana-db:~# systemctl status kafka
● kafka.service - Kafka Server
Loaded: loaded (/lib/systemd/system/kafka.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sat 2021-10-16 20:27:08 PDT; 11h ago
Main PID: 119367 (code=exited, status=127)
Oct 16 20:27:08 yao-instana-vm-32-instana-db.fyre.ibm.com systemd[1]: kafka.service: Scheduled restart job, restart counter is at 5.
Oct 16 20:27:08 yao-instana-vm-32-instana-db.fyre.ibm.com systemd[1]: Stopped Kafka Server.
Oct 16 20:27:08 yao-instana-vm-32-instana-db.fyre.ibm.com systemd[1]: kafka.service: Start request repeated too quickly.
Oct 16 20:27:08 yao-instana-vm-32-instana-db.fyre.ibm.com systemd[1]: kafka.service: Failed with result 'exit-code'.
Oct 16 20:27:08 yao-instana-vm-32-instana-db.fyre.ibm.com systemd[1]: Failed to start Kafka Server.
vi /var/log/syslog
...
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: [2021-10-17 09:11:49,272] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: java.nio.file.NoSuchFileException: /etc/kafka/server.properties
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at java.base/java.nio.file.Files.newByteChannel(Files.java:421)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at java.base/java.nio.file.Files.newInputStream(Files.java:155)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:627)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at kafka.Kafka$.getPropsFromArgs(Kafka.scala:51)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at kafka.Kafka$.main(Kafka.scala:67)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db kafka[127338]: #011at kafka.Kafka.main(Kafka.scala)
Oct 17 09:11:49 yao-instana-vm-32-2-instana-db systemd[1]: kafka.service: Main process exited, code=exited, status=1/FAILUR
...
Workaround:
cp /opt/kafka/config/server.properties /etc/kafka/server.properties
systemctl stop kafka
systemctl daemon-reload
systemctl start kafka
root@yao-instana-vm-32-instana-db:/etc/kafka# systemctl status kafka
● kafka.service - Kafka Server
Loaded: loaded (/lib/systemd/system/kafka.service; disabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kafka.service.d
└─override.conf
Active: active (running) since Sun 2021-10-17 20:30:21 PDT; 3s ago
Main PID: 213379 (java)
Tasks: 25 (limit: 38431)
Memory: 92.2M
CGroup: /system.slice/kafka.service
└─213379 /opt/instana/jre11/bin/java -Xmx256M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:MaxInlineLevel=15 -Djava.awt>
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:99)
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:86)
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at scala.collection.mutable.ArraySeq.map(ArraySeq.scala:38)
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at kafka.log.LogManager.<init>(LogManager.scala:106)
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at kafka.log.LogManager$.apply(LogManager.scala:1212)
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at kafka.server.KafkaServer.startup(KafkaServer.scala:290)
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at kafka.Kafka$.main(Kafka.scala:82)
Oct 17 20:30:23 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: at kafka.Kafka.main(Kafka.scala)
Oct 17 20:30:24 yao-instana-vm-32-instana-db.fyre.ibm.com kafka[213379]: [2021-10-17 20:30:24,401] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
yaoyao12138 commented
cc @BonnieKris
BonnieKris commented
@yaoyao12138 could you help verify it? Thanks.
yaoyao12138 commented
@BonnieKris Of course, could you share me the new kafka deb package for test? Thank you.