kiwigrid/helm-charts

[fluentd] Apache log parsing with extraConfigMaps

Closed this issue · 4 comments

amitaymolko commented

Is this a request for help?: Yes

Version of Helm and Kubernetes: Helm: v2.14.1 K8s: 1.14

Which chart in which version: fluentd-elasticsearch 2.7.0

How to reproduce it:
I'm trying to parse apache logs with the following conf:

extraConfigMaps:
  site.conf: |-
    <filter kubernetes.**>
      @id apache_filter
      @type parser
      key_name message
      reserve_data true
      reserve_time true
      remove_key_name_field true
      <parse>
        @type apache2
      </parse>
    </filter>
monotek commented

Sorry, no apache users here...

  • 👎1
amitaymolko commented

Ok, do you have an example for any custom parsing?

monotek commented

no

  • 👎1
amitaymolko commented

For future reference I got it works like this:

extraConfigMaps:
  containers.site.conf: |-
    <filter kubernetes.**>
      @id apache_parser
      @type parser
      key_name message
      reserve_data true
      # remove_key_name_field true
      <parse>
        @type multi_format
        <pattern>
          format apache2
        </pattern>
        <pattern>
          format json
        </pattern>
        <pattern>
          format none
        </pattern>
      </parse>
    </filter>

The important thing to note is the conf file name, its crutial to the order of the fluentd conf.

Edit: previous code caused fluentd parsing errors

  • 🎉2