logstash-plugins/logstash-codec-netflow

Cisco ASA 9.8 netflow v9 - template is read and data parsed successfully, but after some time template not recognized anymore - WARN - can't (yet) decode flowset id...

rol-ubiqube opened this issue · 2 comments

Hi,

We are facing a strange issue while trying to integrate cisco asa (9.8) netflow pcap into logstash 5.6.4 with netflow codec 3.7.0.

Template (previously cached) is read successfully by netflow codec when logstash is started, data flows parsed successfully and exported into ES.
But, after a minute or so, template is not recognized anymore, and WARN messages are seen in the logs for all new attempts of data processing, e.g.
[WARN ][logstash.codecs.netflow  ] Can't (yet) decode flowset id 256 from source id 0, because no template to decode it with has been received. This message will usually go away after 1 minute.

Was this issue seen before?


  • Version: Logstash 5.6.4 + netflow codec 3.7.0
  • Operating System: CentOs release 6.8 (Final)
  • Config File (if you have sensitive info, please remove it):
    netflow input configuration:

input {
udp {
type => "netflow"
port => 9995
codec => netflow {
versions => [5,9]
cache_save_path => "/tmp"
cache_ttl => 100
}
}
}

Allow me to provide additional information from further tests, that show an interesting conclusion.
Having logstash receiving a few data netflow packets, in log (with trace activated) is possible to see that:

  1. when logstash is started recognizes correctly the template (cached previously)
  2. after some time, template is not recognized anymore (message error indicated in previous comment)
  3. when template is received again, starts processing traffic correctly again

Didn't anyone see this issue before? I believe now might be related with template caching use issue or wrong configuration.

Thanks

Further troubleshooting showed that issue was related with cache_ttl parameter, and the fact that this is not in minutes as described in documentation (see below) but in smaller unit (seconds I believe).

https://www.elastic.co/guide/en/logstash/5.5/plugins-codecs-netflow.html#plugins-codecs-netflow-cache_ttl

cache_ttl
Value type is number
Default value is 4000
Netflow v9/v10 template cache TTL (minutes)