Parser type filter not working with systemd logs
vipulbehalpande08 opened this issue · 0 comments
vipulbehalpande08 commented
Hi Team,
I have an inline JSON string in message field of systemd logs that I want to parse but when I add filter @type parser it stops capturing the logs in cloudwatch. It works with other log sources. Also, there are no error logs in td-agent.log
{
"syslog_identifier": "vault-audit",
"message": "{\"field_1\": \"value1\", \"field_2\": \"value_2\"}"
}
I am trying to parse above message in JSON format like below
{
"syslog_identifier":"vault-audit",
"message":{
"field_1":"value1",
"field_2":"value_2"
}
}
Config:
<source>
@type systemd
tag journal
path /var/log/journal
read_from_head true
<storage>
@type local
path /var/log/td-agent/fluentd-journald-cursor.json
</storage>
<entry>
fields_strip_underscores true
fields_lowercase true
</entry>
</source>
<filter vault-audit>
@type parser
key_name message
reserve_data true
hash_value_field app_message
emit_invalid_record_to_error false
<parse>
@type json
</parse>
</filter>
<match journal>
@type rewrite_tag_filter
<rule>
key syslog_identifier
pattern /^vault-audit$/
tag vault-audit
</rule>
</match>
<match vault-audit>
@type cloudwatch_logs
region {{region}}
log_group_name /ec2/vault/audit
log_stream_name {{instance_id}}
auto_create_stream true
retention_in_days 7
</match>