fluent/fluent-plugin-mongo

Dynamic collection name with date from record field error

clouddebug opened this issue · 1 comments

json log file

{"message_id":10,"device_id":"xx","device_token":""},"level":500,"level_name":"CRITICAL","channel":"message","datetime":"2023-05-08T21:41:13.345528+08:00","extra":{}}

config file

...
<filter notification_task>
  @type record_transformer
  enable_ruby
  <record>
    log_suffix ${Time.strptime(record['datetime'], '%Y-%m-%dT%H:%M:%S.%L%z').strftime('%Y%m%d')}
  </record>
</filter>

<match notification_message>
  @type mongo
  database test
  # collection ${tag}_%Y%m%d # it can be ok
  collection ${tag}_${record['log_suffix']}  # error
...
</match>
...

problem

When i use collection ${tag}_${record['log_suffix']} , run error:

2023-05-09 03:21:25 +0000 [debug]: #0 fluent/log.rb:339:debug: taking back chunk for errors. chunk="5fb3a3e80cb34407c5f63cf9fec49c34"
2023-05-09 03:21:25 +0000 [warn]: #0 fluent/log.rb:381:warn: failed to flush the buffer. retry_times=4 next_retry_time=2023-05-09 03:21:43 +0000 chunk="5fb3a3e80cb34407c5f63cf9fec49c34" error_class=Mongo::Error::OperationFailure error="[73:InvalidNamespace]: Invalid collection name: notification_message_${record['log_suffix']} (on 1.x.x.x:27017)"

I have refer to https://github.com/fluent/fluent-plugin-mongo/issues/85
I want crete mongo collection like 'log_20230508'

I have resolved it.