[fluentd-elasticsearch] stack trace logs doesn't concatenate

[adelforge]

Is this a request for help?: YES

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Version of Helm and Kubernetes: helm version : Client: &version.Version{SemVer:"v2.16.3"
kubectl version : Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0"

Which chart in which version: 9.2.0

What happened: When my container logs is stack trace java, it's put in my elasticsearch like several line (eg : { "log": "java.sql.SQLException: Network Error ... }
{ "log": " at net.sourceforge.jtds.jdbc.ConnectionJDBC2..... } )

What you expected to happen: I want all my stack trace java in one log event

How to reproduce it (as minimally and precisely as possible): here is my values file :

values:
    fullnameOverride: fluentd

    elasticsearch:
      auth:
        enabled: true
        user: "***"
        password: "***"
      hosts: ["***"]
      logstash:
        enabled: true
        prefix: ${tag}
      scheme: "https"

    env:
      OUTPUT_HOSTS: "***"
      OUTPUT_SCHEME: "https"
      OUTPUT_USER: "***"
      OUTPUT_PASSWORD: "***"
      LOGSTASH_PREFIX: ${tag}
    tolerations:
    - key: node-role.kubernetes.io/master
      effect: NoSchedule

    extraConfigMaps:
      parser.conf: |-
          <filter kubernetes.**>
          @type record_transformer
          enable_ruby true
            <record>
            kubernetes_namespace_name ${record["kubernetes"]["namespace_name"]}
            </record>
          </filter>
          
          # retag based on the namespace and container name of the log message
          <match kubernetes.**>
            @type rewrite_tag_filter
            # Update the tag have a structure of kube.<namespace>
            <rule>
              key kubernetes_namespace_name
              pattern ^(.+)$
              tag kube.$1
            </rule>
          </match>

          # Parse logs in the xxx namespace using the kubernetes formatter.
          <filter kube.xxx*>
            @type parser	
            key_name log
            reserve_data true
            reserve_time true
            <parse>
              @type multiline
                format_firstline /\[\d{4}-\d{1,2}-\d{1,2}/
                format1 /^\[(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2},\d{1,3})\] (?<level>[^\s]+) \((?<logger>.*)\) \[(?<thread>.*)\] \((?<ID>.*)\) (?<message>.*)/
            </parse>
          </filter>
          
    extraVolumes:
      - name: parser-conf
        configMap:
          name: fluentd
          items:
            - key: parser.conf
              path: parser.conf

    extraVolumeMounts:
      - name: parser-conf
        mountPath: /fluentd/etc/conf.d/
        readOnly: true

Anything else we need to know:
Nothing else to say thanks by this wonderful k8s-app

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mkuziemko]

@adelforge did you find a solution for this problem? I noticed the same bug in my env and I am wondering about a possible fix.