Cisco FTD IE 231 & IE 232 (and maybe others)

[farces]

Please ignore, original title referenced IE 231 & 232 which were fixed in 3.13.2 (my plugin update hadn't installed that version). Second part is still applicable but looks to be Cisco-specific in that some fields which should represent absolute byte values are underflowing and showing (2^64)-value in some edge cases.

logstash_1       | [2018-05-07T23:57:04,042][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"elastiflow-2018.05.07", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x2435b0a4>], :response=>{"index"=>{"_index"=>"elastiflow-2018.05.07", "_type"=>"doc", "_id"=>"SzMJPWMB7cSyCdlKhFSb", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [flow.bytes]", "caused_by"=>{"type"=>"json_parse_exception", "reason"=>"Numeric value (18446744073709551677) out of range of long (-9223372036854775808 - 9223372036854775807)\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@4da95dc1; line: 1, column: 633]"}}}}}

(the log entry is for netflow.bytes which elastiflow constructs from netflow.rev_flow_delta_bytes and netflow.fwd_flow_delta_bytes). The numeric value there is larger than 2^64 as it's a sum of the the two above fields, where one is close to 2^64.

In checking the raw data from a pcap the source looks like it contains those values as-is so I think we can safely say that the data is being read from the flow correctly at the least:

Flow 3
    Flow Id: 5631978
    SrcAddr: 13.88.181.66
    SrcPort: 443
    InputInt: 5
    DstAddr: 203.1.1.1
    DstPort: 63765
    OutputInt: 13
    Protocol: TCP (6)
    IPv4 ICMP Type: 0
    IPv4 ICMP Code: 0
    Post NAT Source IPv4 Address: 13.88.181.66
    Post NAT Destination IPv4 Address: 192.168.1.1
    Post NAPT Source Transport Port: 443
    Post NAPT Destination Transport Port: 63765
    Firewall Event: Unknown (5)
    Extended firewall event code: Unknown (2022)
    Observation Time Milliseconds: May  7, 2018 15:43:40.299000000 AUS Eastern Standard Time
    Initiator Octets: 6741
    Responder Octets: 18446744073709544936
    Initiator Packets: 5
    Responder Packets: 1
    StartTime: May  7, 2018 15:43:36.379000000 AUS Eastern Standard Time

Has anyone else experienced this? Should these fields be post-processed and not just taken as a raw bytes counter? I can't find the relevant RFC or definitions to confirm.

We have a case open with Cisco TAC to see if the cause can be identified as an issue on their side.

[robcowart]

This issue will be addressed once the following PRs are merged and released for the...

Logstash UDP Input: logstash-plugins/logstash-input-udp#46
Logstash Netflow Codec: #187