How to populate timestamp column
bschooly opened this issue · 2 comments
bschooly commented
What is the proper/recommended way to populate the timestamp?
If have:
<source>
@type tail
path /tmp/example.log
pos_file /var/log/td-agent/example.log.pos
tag example.*
<parse>
@type json
keep_time_key true
</parse>
</source>
...
<match example.**>
@type sql
host localhost
port 3306
database example
adapter mysql2
username example
password XXXXXX
socket /var/lib/mysql/mysql.sock
remove_tag_prefix my.rdb # optional, dual of tag_prefix in in_sql
<table>
table logs
column_mapping 'time,host,size,method'
# This is the default table because it has no "pattern" argument in <table>
# The logic is such that if all non-default <table> blocks
# do not match, the default one is chosen.
# The default table is required.
</table>
</match>
The above works but I couldn't get the time value reformatted. How do I do this? Also how do I use the implicit timestamp generated by fluentd? Ie. the timestamp that is not part of the json payload.
Here is a input record:
{"time":"1362020444","host":"localhost","size":777,"method":"PUT"}
This works. I get exacly the data shown above in the db but I want to know if I can reformat the time and how to populate the timestamp if it is not part of the json data.
Thanks in advance for a reply...
Bill
husamrahman commented
Any luck on the issue? Facing the same thing at the moment
untsakas commented
Maybe you can use filter
<filter app.*>
@type record_transformer
enable_ruby true
<record>
@timestamp ${time.strftime('%Y-%m-%dT%H:%M:%S.%L')}
</record>
</filter>