[Question] [Help] asctime not recognized as time field in Kibana

[babaMar]

I'm struggling with getting a time field in Kibana, using the awsfirelens plugin.

Here's my ContainerDefinitions:

      ContainerDefinitions:
        - Essential: true
          Image: amazon/aws-for-fluent-bit:latest
          Name: !Join [ '-', [ 'LogRouter', 'energy', !Ref DeployEnvironment] ]
          FirelensConfiguration:
            Type: fluentbit
            Options:
              config-file-type: 'file'
              config-file-value: '/fluent-bit/configs/parse-json.conf'
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: firelens-container
              awslogs-region: !Ref 'AWS::Region'
              awslogs-create-group: 'true'
              awslogs-stream-prefix: firelens
          MemoryReservation: 50

        - Environment:
          ...
          Essential: true
          Image: !Sub '${RepositoryURL}:${CommitHash}'
          LogConfiguration:
            LogDriver: awsfirelens
            Options:
              Name: firehose
              region: !Ref 'AWS::Region'
              delivery_stream: !FindInMap [ EnvMap, !Ref DeployEnvironment, LogDeliveryStream ]
              data_keys: 'asctime,name,module,lineno,funcName,levelname,message'
              time_key: 'asctime'
              time_key_format: '%Y-%m-%dT%H:%M:%S%L'

I can see asctime in Kibana fields, but only as a string.

Do I need an extra config for that?

[PettitWesley]

I am not certain... this seems like its probably a Kibana thing? The Fluent Bit firehose plugin and the firehose API have no concept of a timestamp. Just of bytes. So Fluent Bit is sending this timestamp as a string, and I think Kibana has to parse it. Unfortunately I do not know Kibana. But I am not aware of anyway that fluent bit can do anything beyond sending a serialized string to Firehose.

[babaMar]

Yeah, it turned out I had to set the date type field on my index via API on Kibana to make it working.