/elk-kubed

ELK configuration to work on Kubernetes (tested on Azure Kubernetes Service)

Primary LanguageShell

EKL Kubed

Dependencies:

kube metrics kubernetes plugin

kubectl apply -f 0_dependencies/kube-state-metrics

General configuration (namespace, storage)

kubectl apply -f 1_general/namespace.yaml Kubectl apply -f 1_general

Elastic search

kubectl apply -f 2_elastic

Kibana

kubectl apply -f 3_kibana

Beat general configurations

kubectl apply -f 4_beats

Beat workers

kubectl apply -f 5_beats

Logstash

kubectl apply -f 6_logstash

Fluentd

kubectl create -f 7_fluentd

You can run all these commands manually or use the bash script contained on utility folder.

Reduce or Increase elastic nodes

Data nodes

In the example have 1 node of data declared in elastic-data.yaml, for increase only need add more replicas.

spec:
  serviceName: elasticsearch-data
  replicas: 1

Master Nodes

In the case of master elastic-master.yaml nodes you need update the replica param and the enviroment cluster.initial_master_nodes need add or remove the name of master nodes candidates.

spec:
  serviceName: elasticsearch-master
  replicas: 2
  
...
          env:
            - name: discovery.seed_hosts
              value: "elasticsearch-master.elk.svc.cluster.local"
            - name: cluster.initial_master_nodes
              value: "elasticsearch-master-0,elasticsearch-master-1" # necessarily have to match the number of replicas

Fluentd

You can change the elastisearch index pattern:

            - name: FLUENT_ELASTICSEARCH_LOGSTASH_INDEX_NAME
              value: "fluentd"
            - name: FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX
              value: "fluentd"

Author

Ismael Leiva

License

Copyright © 2017, Ismael Leiva. Released under the MIT License.

Credits

Based on: