systemd services for Confluent Platform
Available for:
- Confluent Kafka
- Confluent Schema Registry
- Confluent Kafka Connect
- Confluent ZooKeeper
- Confluent Control Center
Prerequisites
Users, groups and directories
- Create the users and group:
- kafka
- zookeeper
- confluent-cc
- Create directories:
- /var/log/kafka
- /var/log/zookeeper
- /var/log/confluent-control-center
- Set appropriate permissions
Kafka Connect Log4j settings
Edit /etc/kafka/connect-log4j.properties, add/change the following lines
log4j.rootLogger=INFO, kafkaConnectAppender
log4j.appender.kafkaConnectAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.kafkaConnectAppender.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.kafkaConnectAppender.File=${kafka.logs.dir}/connect.log
log4j.appender.kafkaConnectAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.kafkaConnectAppender.layout.ConversionPattern=[%d] %p %m (%c)%n
Confluent Control Center
For logging into files, rename log4j-rolling.properties to log4j.properties.
By default Control Center will log to /tmp. To change, set the paths in log4j.properties after renaming.
Installation
Put the unit file into the location: /etc/systemd/system/
Reload systemd:
systemctl daemon-reload
For auto restart of these services use:
systemctl enable servicename.service
Additional settings
Some Apache Kafka settings are set via JVM parameters and environment variables. For example security settings or log paths.
To add environment parameters for systemd services "Environment" entries can be added to the "Service" section of a service. As an example the following line can be added to specify the path to the jaas configuration file needed to enable Kerberos for authentication.
Environment="KAFKA_OPTS=-Djava.security.auth.login.config=/etc/kafka/kafka-jaas.conf"
More information about Apache Kafka settings and environment variables can be found in the official documentation.
Summary
JMX is enabled by default. To disable JXM remove the "Environment=" line.
Confluent Kafka
Kafka properties: /etc/kafka/server.properties
Logs: /var/log/kafka
JMX Port: 10030
Confluent Schema Registry
Schema Registry properties: /etc/schema-registry/schema-registry.properties
JMX Port: 10050
Confluent Kafka Connect
Kafka Connect properties: /etc/kafka/connect-distributed.properties
Log: /var/log/kafka/connect.log
JMX Port: 10040
Confluent ZooKeeper
ZooKeeper properties: /etc/kafka/zookeeper.properties
Logs: /var/log/zookeeper
JMX Port: 10020