/helm-nifi

Helm Chart for Apache Nifi

Primary LanguageHTMLApache License 2.0Apache-2.0

Helm Chart for Apache Nifi

CircleCI License version

Introduction

This Helm chart installs nifi in a Kubernetes cluster.

Prerequisites

  • Kubernetes cluster 1.10+
  • Helm 3.0.0+
  • PV provisioner support in the underlying infrastructure.

Installation

Add Helm repository

helm repo add cetic https://cetic.github.io/helm-charts
helm repo update

Configure the chart

The following items can be set via --set flag during installation or configured by editing the values.yaml directly (need to download the chart first).

Configure the way how to expose nifi service:

  • Ingress: The ingress controller must be installed in the Kubernetes cluster.
  • ClusterIP: Exposes the service on a cluster-internal IP. Choosing this value makes the service only reachable from within the cluster.
  • NodePort: Exposes the service on each Node’s IP at a static port (the NodePort). You’ll be able to contact the NodePort service, from outside the cluster, by requesting NodeIP:NodePort.
  • LoadBalancer: Exposes the service externally using a cloud provider’s load balancer.

Configure the way how to persistent data:

  • Disable: The data does not survive the termination of a pod.
  • Persistent Volume Claim(default): A default StorageClass is needed in the Kubernetes cluster to dynamic provision the volumes. Specify another StorageClass in the storageClass or set existingClaim if you have already existing persistent volumes to use.

Configure authentication:

  • You first need a secure cluster which can be accomplished by enabling the built-in CA nifi-toolkit container (ca.enabled to true). By default, a secure nifi cluster uses certificate based authentication but you can optionally enable ldap or oidc. See the configuration section for more details.

⚠️ This feature is quite new. Please open an issue if you encounter a problem. We are currently working on the ldap authentication. Also, any help is welcome to add other authentication methods.

Install the chart

Install the nifi helm chart with a release name my-release:

helm install --name my-release cetic/nifi

Install from local clone

git clone https://github.com/cetic/helm-nifi.git nifi
cd nifi
helm repo add incubator https://kubernetes-charts-incubator.storage.googleapis.com
helm repo update
helm dep up
helm install --name nifi .

Uninstallation

To uninstall/delete the my-release deployment:

helm delete --purge my-release

Configuration

The following table lists the configurable parameters of the nifi chart and the default values.

Parameter Description Default
ReplicaCount
replicaCount Number of nifi nodes 1
Image
image.repository nifi Image name apache/nifi
image.tag nifi Image tag 1.11.4
image.pullPolicy nifi Image pull policy IfNotPresent
image.pullSecret nifi Image pull secret nil
SecurityContext
securityContext.runAsUser nifi Docker User 1000
securityContext.fsGroup nifi Docker Group 1000
sts
sts.podManagementPolicy Parallel podManagementPolicy Parallel
sts.AntiAffinity Affinity for pod assignment soft
sts.pod.annotations Pod template annotations security.alpha.kubernetes.io/sysctls: net.ipv4.ip_local_port_range=10000 65000
secrets
secrets Pass any secrets to the nifi pods. The secret can also be mounted to a specific path if required. nil
configmaps
configmaps Pass any configmaps to the nifi pods. The configmap can also be mounted to a specific path if required. nil
nifi properties
properties.externalSecure externalSecure for when inbound SSL false
properties.isNode cluster node properties (only configure for cluster nodes) true
properties.httpPort web properties HTTP port 8080
properties.httpsPort web properties HTTPS port null
properties.clusterPort cluster node port 6007
properties.clusterSecure cluster nodes secure mode false
properties.needClientAuth nifi security client auth false
properties.provenanceStorage nifi provenance repository max storage size 8 GB
properties.siteToSite.secure Site to Site properties Secure mode false
properties.siteToSite.port Site to Site properties Secure port 10000
properties.siteToSite.authorizer managed-authorizer
properties.safetyValve Map of explicit 'property: value' pairs that overwrite other configuration nil
nifi user authentication
auth.admin Default admin identity CN=admin, OU=NIFI
auth.ldap.enabled Enable User auth via ldap false
auth.ldap.host ldap hostname ldap://<hostname>:<port>
auth.ldap.searchBase ldap searchBase CN=Users,DC=example,DC=com
auth.ldap.searchFilter ldap searchFilter CN=john
auth.oidc.enabled Enable User auth via oidc false
auth.oidc.discoveryUrl oidc discover url https://<provider>/.well-known/openid-configuration
auth.oidc.clientId oidc clientId nil
auth.oidc.clientSecret oidc clientSecret nil
auth.oidc.claimIdentifyingUser oidc claimIdentifyingUser email
postStart
postStart Include additional libraries in the Nifi containers by using the postStart handler nil
Headless Service
headless.type Type of the headless service for nifi ClusterIP
headless.annotations Headless Service annotations service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
UI Service
service.type Type of the UI service for nifi NodePort
service.httpPort Port to expose service 8080
service.httpsPort Port to expose service in tls 443
service.annotations Service annotations {}
service.loadBalancerIP LoadBalancerIP if service type is LoadBalancer nil
service.loadBalancerSourceRanges Address that are allowed when svc is LoadBalancer []
service.processors.enabled Enables additional port/ports to nifi service for internal processors false
service.processors.ports Specify "name/port/targetPort/nodePort" for processors sockets []
Ingress
ingress.enabled Enables Ingress false
ingress.annotations Ingress annotations {}
ingress.path Path to access frontend (See issue #22) /
ingress.hosts Ingress hosts []
ingress.tls Ingress TLS configuration []
Persistence
persistence.enabled Use persistent volume to store data false
persistence.storageClass Storage class name of PVCs (use the default type if unset) nil
persistence.accessMode ReadWriteOnce or ReadOnly [ReadWriteOnce]
persistence.configStorage.size Size of persistent volume claim 100Mi
persistence.authconfStorage.size Size of persistent volume claim 100Mi
persistence.dataStorage.size Size of persistent volume claim 1Gi
persistence.flowfileRepoStorage.size Size of persistent volume claim 10Gi
persistence.contentRepoStorage.size Size of persistent volume claim 10Gi
persistence.provenanceRepoStorage.size Size of persistent volume claim 10Gi
persistence.logStorage.size Size of persistent volume claim 5Gi
persistence.existingClaim Use an existing PVC to persist data nil
jvmMemory
jvmMemory bootstrap jvm size 2g
SideCar
sidecar.image Separate image for tailing each log separately and checking zookeeper connectivity busybox
sidecar.tag Image tag 1.32.0
Resources
resources Pod resource requests and limits for logs {}
logResources
logresources. Pod resource requests and limits {}
nodeSelector
nodeSelector Node labels for pod assignment {}
terminationGracePeriodSeconds
terminationGracePeriodSeconds Number of seconds the pod needs to terminate gracefully. For clean scale down of the nifi-cluster the default is set to 60, opposed to k8s-default 30. 60
tolerations
tolerations Tolerations for pod assignment []
initContainers
initContainers Container definition that will be added to the pod as initContainers []
extraVolumes
extraVolumes Additional Volumes available within the pod (see spec for format) []
extraVolumeMounts
extraVolumeMounts VolumeMounts for the nifi-server container (see spec for details) []
env
env Additional environment variables for the nifi-container (see spec for details) []
extraContainers
extraContainers Additional container-specifications that should run within the pod (see spec for details) []
zookeeper
zookeeper.enabled If true, deploy Zookeeper true
zookeeper.url If the Zookeeper Chart is disabled a URL and port are required to connect nil
zookeeper.port If the Zookeeper Chart is disabled a URL and port are required to connect 2181
registry
registry.enabled If true, deploy Nifi Registry true
registry.url If the Nifi Registry Chart is disabled a URL and port are required to connect nil
registry.port If the Nifi Registry Chart is disabled a URL and port are required to connect 80
ca
ca.enabled If true, deploy Nifi Toolkit as CA false
ca.server CA server dns name nil
ca.port CA server port number 9090
ca.token The token to use to prevent MITM 80
ca.admin.cn CN for admin certificate admin

Credits

Initially inspired from https://github.com/YolandaMDavis/apache-nifi.

TLS work/inspiration from https://github.com/sushilkm/nifi-chart.git.

Contributing

Feel free to contribute by making a pull request.

Please read the official Contribution Guide from Helm for more information on how you can contribute to this Chart.

License

Apache License 2.0