"kiwigrid/fluentd-elasticsearch": Containers single event log is splitting into multiple parts.

[Amar69]

Is this a request for help?:
Please help me to get container's single event log as one entry in kibana dashboard

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

It can be a kind of bug.

Version of Helm and Kubernetes:

Helm: version.BuildInfo{Version:"v3.2.0", GitCommit:"e11b7ce3b12db2941e90399e874513fbd24bcb71", GitTreeState:"clean", GoVersion:"go1.13.10"}

Kubernetesl: GitVersion:"v1.18.2"
Which chart in which version:
kiwigrid/fluentd-elasticsearch
version: latest

What happened:
Container's single event log is splitting into multiple parts based on every parameter and timestamp.
When I am seeing the containers logs in fluentd, There are multiple records of a single event log. It is very difficult to read log or understand.

What you expected to happen:
Container's single event log should appear as one entry, not in multiple parts.
I am expecting this kind of log

{"statusCode":404,"status":"Not Found","message":"Cannot GET /branches/geolocation/lat/41.12/long/"}

How to reproduce it (as minimally and precisely as possible):
helm install my-release kiwigrid/fluentd-elasticsearch --debug
you can check any container's running log under that Kubernetes cluster

My mock API container log :
{"log":" "timestampUTC": "2020-06-01 17:34:11.300 +05:30",\n","stream":"stdout","time":"2020-06-01T12:04:11.301078376Z"}
{"log":" "url": "//branches/10",\n","stream":"stdout","time":"2020-06-01T12:04:11.301082072Z"}
{"log":" "method": "GET",\n","stream":"stdout","time":"2020-06-01T12:04:11.301085543Z"}
{"log":" "statusCode": 404,\n","stream":"stdout","time":"2020-06-01T12:04:11.301088985Z"}
{"log":" "status": "Not Found",\n","stream":"stdout","time":"2020-06-01T12:04:11.301092393Z"}
{"log":" "responseMessage": "Cannot GET //branches/10",\n","stream":"stdout","time":"2020-06-01T12:04:11.30109595Z"}
{"log":" "httpVersion": "1.1",\n","stream":"stdout","time":"2020-06-01T12:04:11.301099602Z"}
{"log":" "ip": "::ffff:192.168.159.64",\n","stream":"stdout","time":"2020-06-01T12:04:11.30110313Z"}
{"log":" "userAgent": "PostmanRuntime/7.25.0",\n","stream":"stdout","time":"2020-06-01T12:04:11.301106602Z"}
{"log":" "responseTimeInMS": 1,\n","stream":"stdout","time":"2020-06-01T12:04:11.301110158Z"}
{"log":" "reqBody": {}\n","stream":"stdout","time":"2020-06-01T12:04:11.301113625Z"}
{"log":"}\n","stream":"stdout","time":"2020-06-01T12:04:11.301117396Z"}
{"log":"{\n","stream":"stdout","time":"2020-06-01T12:05:42.411409663Z"}
{"log":" "service": "MASTER_DEMIOS_MICROSERVICE",\n","stream":"stdout","time":"2020-06-01T12:05:42.411442686Z"}
{"log":" "type": "REQ_RES_LOG",\n","stream":"stdout","time":"2020-06-01T12:05:42.411447872Z"}
{"log":" "message": "[8vLXZ7Fsz3kdsjf0pcnUU, 542b4a1b-e755-4594-93bf-3c9296faf4df] GET /branches/geolocation/lat/41.12/long/ | 404 Not Found | Cannot GET /branches/geolocation/lat/41.12/long/ | 2ms",\n","stream":"stdout","time":"2020-06-01T12:05:42.411451872Z"}
Anything else we need to know:

I am using AWS ES and EKS(Version=1.16) and default config of kiwigrid/fluentd-elasticsearch helm chart.

Expected Log: {"statusCode":404,"status":"Not Found","message":"Cannot GET /branches/geolocation/lat/41.12/long/"}

[Amar69]

Guys, if you are facing the issue then, you have to go to kibana dashboard and the add available fields.