Decoding nested JSON objects
luckyb56 opened this issue · 4 comments
luckyb56 commented
When using Packetbeat to send messages directly to Graylog2, the nested JSON objects won't be decoded and would be seen as '[object Object],[object Object]' under search UI. Example input JSON message:
{
"_index" : "graylog_5",
"_type" : "message",
"_id" : "572b9193-16df-11e6-8a3b-000c2942c251",
"_version" : 1,
"found" : true,
"_source" : {
"packetbeat_bytes_in" : 32,
"packetbeat_method" : "QUERY",
"packetbeat_type" : "dns",
"packetbeat_responsetime" : 140,
"packetbeat_query" : "class IN, type A, conn.skype.com",
"gl2_remote_ip" : "172.16.220.1",
"packetbeat_dns_question_name" : "conn.skype.com",
"gl2_remote_port" : 65532,
"packetbeat_dns_additionals_count" : 0,
"packetbeat_dns_answers_count" : 2,
"source" : "abs-MacBook-Pro.local",
"type" : "dns",
"gl2_source_input" : "572a39d0cdf3830902a406df",
"packetbeat_dns_response_code" : "NOERROR",
"packetbeat_direction" : "out",
"packetbeat_client_ip" : "192.168.0.3",
"packetbeat_dns_flags_recursion_allowed" : true,
"packetbeat_dns_flags_truncated_response" : false,
"packetbeat_dns_question_class" : "IN",
"gl2_source_node" : "b6d4add1-2cfc-4fd1-b18d-0ad0478e00a8",
"packetbeat_dns_flags_authoritative" : false,
"packetbeat_status" : "OK",
"packetbeat_client_port" : 60426,
"timestamp" : "2016-05-10 18:45:16.558",
"packetbeat_ip" : "192.168.0.1",
"packetbeat_dns_op_code" : "QUERY",
"packetbeat_bytes_out" : 83,
"packetbeat_dns_flags_recursion_desired" : true,
"packetbeat_transport" : "udp",
"packetbeat_dns_authorities_count" : 0,
"packetbeat_resource" : "conn.skype.com",
"streams" : [ "572ae5c9cdf3830902a4bb7f" ],
"packetbeat_dns_answers" : [ {
"class" : "IN",
"data" : "conn.skype.akadns.net",
"name" : "conn.skype.com",
"ttl" : 464,
"type" : "CNAME"
}, {
"class" : "IN",
"data" : "91.190.216.81",
"name" : "conn.skype.akadns.net",
"ttl" : 300,
"type" : "A"
} ],
"message" : "-",
"packetbeat_dns_question_type" : "A",
"packetbeat_count" : 1,
"name" : "MacBook-Pro.local",
"packetbeat_dns_id" : 62527,
"facility" : "packetbeat",
"packetbeat_port" : 53
}
}
packetbeat_dns_answers structure won't be decoded in this example.
pramodanarase commented
@luckyb56 what you want ? decode in single message or multiple message?
if decode into multiple message on nested field(in your case packetbeat_dns_answers) i think we need to implement feature like logstash split filter using "MultiMessageCodec".
luckyb56 commented
@pramodanarase A single message would be great as it will keep the context and do basic level of stats.
rompic commented
+1
kroepke commented
This has been partially addressed in Graylog2/graylog2-server#3106 and #29