Dynamic collection name with date from record field error
clouddebug opened this issue · 1 comments
clouddebug commented
json log file
{"message_id":10,"device_id":"xx","device_token":""},"level":500,"level_name":"CRITICAL","channel":"message","datetime":"2023-05-08T21:41:13.345528+08:00","extra":{}}
config file
...
<filter notification_task>
@type record_transformer
enable_ruby
<record>
log_suffix ${Time.strptime(record['datetime'], '%Y-%m-%dT%H:%M:%S.%L%z').strftime('%Y%m%d')}
</record>
</filter>
<match notification_message>
@type mongo
database test
# collection ${tag}_%Y%m%d # it can be ok
collection ${tag}_${record['log_suffix']} # error
...
</match>
...
problem
When i use collection ${tag}_${record['log_suffix']}
, run error:
2023-05-09 03:21:25 +0000 [debug]: #0 fluent/log.rb:339:debug: taking back chunk for errors. chunk="5fb3a3e80cb34407c5f63cf9fec49c34"
2023-05-09 03:21:25 +0000 [warn]: #0 fluent/log.rb:381:warn: failed to flush the buffer. retry_times=4 next_retry_time=2023-05-09 03:21:43 +0000 chunk="5fb3a3e80cb34407c5f63cf9fec49c34" error_class=Mongo::Error::OperationFailure error="[73:InvalidNamespace]: Invalid collection name: notification_message_${record['log_suffix']} (on 1.x.x.x:27017)"
I have refer to https://github.com/fluent/fluent-plugin-mongo/issues/85
I want crete mongo collection like 'log_20230508'
clouddebug commented
I have resolved it.