logstash-plugins/logstash-codec-netflow

Add support for MPLS labels

JasonLZJ opened this issue · 20 comments

JasonLZJ commented

I send netflow v9 packets to logstash
The pacp file is this
Can you help me?

thanks

jorritfolmer commented

Thanks for the pcap, I'll look into it later.
Are you running the latest version of the codec?
What network device do you use to send Netflow9?

JasonLZJ commented

hi @jorritfolmer ：
my device is huawei router
my netflow codec is logstash-codec-netflow (3.14.1)

JasonLZJ commented

hi @jorritfolmer I update it to 4.1.1
The problem is not solved.
thx

jorritfolmer commented

Sorry I haven't looked into it yet

JasonLZJ commented

hi @jorritfolmer
Can you give priority to it? We rely on this service more. Thank you very much.

JasonLZJ commented

hi @jorritfolmer
Can you give priority to it? We rely on this service more. Thank you very much.

hi @jorritfolmer

JasonLZJ commented

hi @jorritfolmer I find the log
Unsupported field in template 1317 {:type=>70, :length=>3}

@robcowart
@janniten

thanks

robcowart commented

Your PCAP includes templates 1315, 1317, 1319 & 1320. However it only include flow records for 1315. Can you provide a PCAP that has flow records for all four flowsets?

janniten commented

Hi,
In my case templates are 256,257,258 and the flows in my pcap are from flow 256.
Regards
Ana

robcowart commented

@janniten your's is a different issue related to IXIA IPFIX records. This issue is for Huawei.

janniten commented
robcowart commented

@JasonLZJ are you able to provide a more complete PCAP as I requested above. I have this fix largely completed, but I need the PCAP to confirm all is as intended.

jorritfolmer commented

So this appears to be first time in years that we finally encounter fields 70-79 MPLS labels in Netflow v9! And we almost have a complete pcap!

jorritfolmer commented

There is a closed PR #5 from 2 years ago that we can re-use

robcowart commented

#5 looks like it would handle the unpacking of the MPLS labels. However, the other issue with the Huawei Netflow records is that they use a uint32 for interface index values (similar to IPFIX), where Netflow (v5 or v9) specifies a uint16 for these values.

From the template ...

Field (9/25): INPUT_SNMP
    Type: INPUT_SNMP (10)
    Length: 4
Field (10/25): OUTPUT_SNMP
    Type: OUTPUT_SNMP (14)
    Length: 4
jorritfolmer commented

If I'm remembering correctly we already ignore the YAML for uint* fields and prefer the length as defined in the template.

(EDIT: https://github.com/logstash-plugins/logstash-codec-netflow/blob/master/lib/logstash/codecs/netflow.rb#L447-L468)

JasonLZJ commented

I caught more packages for the community. mpls

@robcowart

thanks

jorritfolmer commented

Your pcap still only contains data packets for template 1315.
The MPLS ones are 1317 and 1319.

Traffic with template 1315 is getting decoded properly.
All your traffic uses template 1315.

I don't think you are actually missing/losing data.
You're seeing an error message, but it doesn't affect your netflow data.

JasonLZJ commented

My application only needs to parse 1315 packets.

Do you mean that I can parse 1315 packets without upgrading the version of plugin ? My version is 3.14.1
@jorritfolmer
@robcowart
@janniten
thanks

jorritfolmer commented

Yes Huawei flows with template 1315 have been supported since 3.11.0.