aws/amazon-cloudwatch-logs-for-fluent-bit

Default config and error message invalid time format %d/%b/%Y:%H:%M:%S %z

caevv opened this issue · 5 comments

caevv commented

I'm getting invalid time format error message, but I'm not asking to convert any time, and I don't know where this is coming from. Where is this config coming from? Which field is it trying to parse? How can I change? Thanks a lot.

I'm using the default config from https://github.com/aws/aws-for-fluent-bit/blob/master/configs/parse-json.conf by doing:

        "options":{
           "config-file-type": "file",
           "config-file-value": "/fluent-bit/configs/parse-json.conf"
        }

Error message:

[2020/09/11 10:33:56] [ warn] [parser:json] invalid time format %d/%b/%Y:%H:%M:%S %z for '2020-09-11T10:33:56Z'

Service container definition:

  {
    "name": "${name}",
    "image": "${image}",
    "essential": true,
    "portMappings": [
      {
        "containerPort": ${port},
        "hostPort": ${port}
      }
    ],
    "logConfiguration": {
      "logDriver": "awsfirelens",
      "options": {
          "AWS_Region": "${region}",
          "AWS_Auth": "On",
          "Name": "es",
          "Host": "${es_logs_host}",
          "Port": "443",
          "tls": "On",
          "Index": "logs",
          "Logstash_Format": "On",
          "Logstash_Prefix": "logs"
      }
    }
  }

Firelens container definition:

  {
    "name": "log_router",
    "image": "docker.io/amazon/aws-for-fluent-bit:latest",
    "essential": true,
    "firelensConfiguration": {
        "type": "fluentbit",
        "options":{
           "config-file-type": "file",
           "config-file-value": "/fluent-bit/configs/parse-json.conf"
        }
    },
    "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
            "awslogs-create-group": "true",
            "awslogs-group": "${name}_log_router",
            "awslogs-region": "${region}",
            "awslogs-stream-prefix": "ecs"
        }
    },
    "memoryReservation": 50
  }
caevv commented

Answering my own question, seems like it's coming from default fluentbit, which is quite annoying:
https://github.com/fluent/fluent-bit/blob/b10fa5c96971f393ed0c9eb7b6cfc3acfbe886b3/conf/parsers.conf#L38-L39

Closing, re-open if you require more assistance

Hey @caevv, how did you go about solving this?

caevv commented

Hey @caevv, how did you go about solving this?

We had to change the parsers config on our own Docker image.
https://github.com/fluent/fluent-bit/blob/b10fa5c96971f393ed0c9eb7b6cfc3acfbe886b3/conf/parsers.conf#L38-L39

I figured as much. Thanks for confirming!