This project provides implementations of Kafka's ConfigProvider interface. These implementations make it possible to configure Kafka/Kafka Connect using various configuration sources.
The library can be downloaded from Maven central using com.redhat.insights.kafka:config-providers:${version} coordinates.
This implementation is similar to the built-in FileConfigProvider. Unlike FileConfigProvider, PlainFileConfigProvider does not require the configuration file to be a properties file. Instead, the entire file content is read and used as a configuration value for the given key.
This is useful when working with Kubernetes Secrets that are mounted to containers.
Consider the following Kubernetes secret:
- apiVersion: v1
kind: Secret
metadata:
name: database
stringData:
db.host: database.database.svc.cluster.local
db.port: 5432
db.name: inventory
db.user: inventory
db.password: secretWhen the secret is mounted to a container's filesystem, a file structure is created, such as:
/opt/kafka/external-configuration/database
├── db.host
├── db.name
├── db.password
├── db.port
└── db.userFileConfigProvider then allows for these individual files to be referenced from Kafka configuration.
First, registed FileConfigProvider with Kafka or Kafka Connect:
config.providers: file
config.providers.file.class: com.redhat.insights.kafka.config.providers.PlainFileConfigProviderThen, reference the secret files in Kafka or Kafka Connect configuration:
{
"config": {
"database.hostname": "${file:/opt/kafka/external-configuration/database/db.host}",
"database.port": "${file:/opt/kafka/external-configuration/database/db.port}",
"database.user": "${file:/opt/kafka/external-configuration/database/db.user}",
"database.password": "${file:/opt/kafka/external-configuration/database/db.password}",
"database.dbname": "${file:/opt/kafka/external-configuration/database/db.name}"
// the rest of the connector configuration left out for brevity
}
}