A salt formula to deploy Apache NiFi
Configure Apache NiFi in 3 different configurations:
- Standalone
- Cluster with embedded Zookeeper
- Cluster with external Zookeeper
Configures systemd and sysctl limits automatically. See the pillar.example for detailed list of pillars.
Table of Contents
See the full SaltStack Formulas installation and usage instructions.
If you are interested in writing or contributing to formulas, please pay attention to the Writing Formula Section.
If you want to use this formula, please pay attention to the FORMULA file and/or git tag,
which contains the currently released version. This formula is versioned according to Semantic Versioning.
See Formula Versioning Section for more details.
If you need (non-default) configuration, please pay attention to the pillar.example file and/or Special notes section.
Commit message formatting is significant!!
Please see How to contribute for more details.
- CentOS, Ubuntu, Debian
- Cluster configurations require hosts setup with FQDN and DNS.
By Default all of the settings will configure a standalone NiFi Node
nifi:
pkg:
name: nifi
downloadurl: https://mirror.csclub.uwaterloo.ca/apache/nifi/1.12.1/nifi-1.12.1-bin.tar.gz
version: 1.12.1
installdir: /opt
# JDK Package to install. Leave empty to not install.
javajdk: java-1.8.0-openjdk
nifi:
cluster.is.node: 'false'
systemdconfig:
user: root
group: root
limitnofile: 50000
limitnproc: 10000
nifi:
pkg:
name: nifi
downloadurl: https://mirror.csclub.uwaterloo.ca/apache/nifi/1.12.1/nifi-1.12.1-bin.tar.gz
version: 1.12.1
installdir: /opt
# JDK Package to install. Leave empty to not install.
javajdk: java-1.8.0-openjdk
systemdconfig:
user: root
group: root
limitnofile: 50000
limitnproc: 10000
nifi:
# cluster node properties (only configure for cluster nodes) #
cluster.is.node: 'true'
cluster.node.address: {{ grains['fqdn'] }}
cluster.node.protocol.port: '1111'
cluster.flow.election.max.wait.time: '1 mins'
zookeeper.connect.string: 'nifi-1.localdomain:2181,nifi-2.localdomain:2181,nifi-3.localdomain:2181'
state.management.embedded.zookeeper.start: 'true'
web.http.host: {{ grains['fqdn'] }}
zookeeperproperties:
# To configure Zookeeper.properties set 'state.management.embedded.zookeeper.start' to 'true' above, and then define your Embedded Zookeeper servers here.
customservers:
Node1:
hostname: nifi-1.localdomain
zookeeper_myid: 1
zookeeper_clientPort: 2181
zookeeper_peerPorts: '2888:3888'
Node2:
hostname: nifi-2.localdomain
zookeeper_myid: 2
zookeeper_clientPort: 2181
zookeeper_peerPorts: '2888:3888'
Node3:
hostname: nifi-3.localdomain
zookeeper_myid: 3
zookeeper_clientPort: 2181
zookeeper_peerPorts: '2888:3888'
nifi:
pkg:
name: nifi
downloadurl: https://mirror.csclub.uwaterloo.ca/apache/nifi/1.12.1/nifi-1.12.1-bin.tar.gz
version: 1.12.1
installdir: /opt
# JDK Package to install. Leave empty to not install.
javajdk: java-1.8.0-openjdk
systemdconfig:
user: root
group: root
limitnofile: 50000
limitnproc: 10000
nifi:
# cluster node properties (only configure for cluster nodes) #
cluster.is.node: 'true'
cluster.node.address: {{ grains['fqdn'] }}
cluster.node.protocol.port: '1111'
cluster.flow.election.max.wait.time: '1 mins'
zookeeper.connect.string: 'nifi-1.localdomain:2181,nifi-2.localdomain:2181,nifi-3.localdomain:2181'
state.management.embedded.zookeeper.start: 'true'
web.http.host: {{ grains['fqdn'] }}
Meta-state (This is a state that includes other states).
This installs the nifi package, manages the nifi configuration file and then starts the associated nifi service.
This state will install the nifi package only. This downloads the tar.gz file from the downloadurl and deploys it to servers.
This state will configure the nifi service and has a dependency on nifi.install
via include list.
This state will start the nifi service and has a dependency on nifi.config
via include list.
Linux testing is done with kitchen-salt.
- Ruby
- Docker
$ gem install bundler
$ bundle install
$ bin/kitchen test [platform]Where [platform] is the platform name defined in kitchen.yml,
e.g. debian-9-2019-2-py3.
Creates the docker instance and runs the nifi main state, ready for testing.
Runs the inspec tests on the actual instance.
Removes the docker instance.
Runs all of the stages above in one go: i.e. destroy + converge + verify + destroy.
Gives you SSH access to the instance for manual testing.