kiwigrid/helm-charts

Fluentd elasticsearch - ILM Policy

Closed this issue · 7 comments

Deleted user commented

Is this a request for help?: yes

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT

Version of Helm and Kubernetes:
not of interest

Which chart in which version:
v7.9.0 - https://github.com/elastic/helm-charts/tree/master/elasticsearch
v3.0.4 - https://github.com/kiwigrid/helm-charts/tree/master/charts/fluentd-elasticsearch

What happened:
i have tried to use ilm and im getting an error that ilm_policy and ilm_policies dont work together.
2020-08-20T09:52:13.21790686Z 2020-08-20 09:52:13 +0000 [error]: config error file="/etc/fluent/fluent.conf" error_class=Fluent::ConfigError error="You can't specify ilm_policy and ilm_policies at the same tim │ │ e"

What you expected to happen:
When i leave one of the both empty it should not be honored in the daemonset

How to reproduce it (as minimally and precisely as possible):

elasticsearch:
  hosts: ["<yourServiceNameHere>"]
  indexName: "fluentd"
  logstash:
    enabled: false
  ilm:
    enabled: true
    policy_id: fluentd-30d-delete-policy
    policy:
      phases:
        delete:
          min_age: 30d
          actions:
            delete:
              delete_searchable_snapshot: true
    policy_overwrite: true

Anything else we need to know:
I can fix that problem when i comment out the lines in the daemonSet:

#        - name: ILM_POLICIES
#          value: map[]

So this is a simple fix in the helm chart (if not empty) :)

Deleted user commented

It works also if i overwrite policies to:
policies: instead of policies: {} maybe that is an even easier fix. To make it as default.

  • 👍1
Deleted user commented

In general the policy generation dont work. There are no errors but in elasticsearch the policy is missing.

monotek commented

The ilm stuff was a third party contribution in #380

Maybe ask the original contributor @niklaskhf for help.

niklaskhf commented

@monotek I remember that it was necessary to add a template file as ConfigMap. Since I didn’t have time to implement this in a configurable way, I forked this repo and added a default fluentd.json ConfigMap and the required environment variables https://github.com/niklaskhf/helm-charts/tree/master/charts/fluentd-elasticsearch/templates

monotek commented

@niklaskhf So the ilm part of this chart does not work at all?

stale commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.