CESNET/ipfixcol2

Json output plugin and nat events

korbind opened this issue · 4 comments

korbind commented

Hi, I am trying to test ipfixcol2 in my environment: ubuntu host with nat (iptables) and ipt_NETFLOW to generate NetflowV9 NAT events.
Example output with json output plugin:
{
"@type": "ipfix.entry",
"en4294967294:id323": ...,
"iana:sourceIPv4Address": "100.79.62.203",
"iana:destinationIPv4Address": "87.255.2.39",
"en4294967294:id225": ...,
"en4294967294:id226": ...,
"iana:sourceTransportPort": 51413,
"iana:destinationTransportPort": 4865,
"en4294967294:id227": 51413,
"en4294967294:id228": 4865,
"iana:protocolIdentifier": "UDP",
"en4294967294:id230": 1
}
How can I configure ipfixcol2 to show fields decoded as en4294967294:id225,id226,id323 in human format? For example, these fields are presented in system/elements/iana.xml in libfds in the correct format.

Lukas955 commented

Hi,
first of all, identifiers "en4294967294" and "en4294967295" are "internal" scope identifications of NetFlow fields, which cannot be directly converted to appropriate IPFIX fields as RFC7012 (IPFIX data model) guarantees compatibility only for NetFlow fields with ID 0-127. Therefore, the collector is using these specific Private Enterprise Numbers (PEN) for NetFlow fields with ID > 127.

When I finished support for NetFlow v5/v9, I forgot to add configuration files for these NetFlow fields. I will have to go through NetFlow fields (not just Cisco but also from other vendors) and prepare configuration files. It should be done on Monday. So if you can wait, I will let you know.

On the other hand, there is also fast but very naive and temporary solution if you don't want to wait.
You can create copy of iana.xml file and set following <scope> parameters: <pen>4294967294</pen> and <name>netflowLow</name>. It should work in your case...

  • 👍1
ekocbiyik commented

Hi @Lukas955 this is not an issue, just need your advice for debugging ipfixcol2 plugins. we are trying to extend a new plugin (CSV) from JSON output plugin. But does not get caught in breakpoints during debug. We use CLion editor. Could you give us some trick? Thank you!

Lukas955 commented

The configuration files for NetFlow fields (netflow.xml and netflowExtra.xml) are now available in the devel branch of libfds.

I just tried to debug the collector in CLion and these are my steps:

  • Build ipfixcol2 and dsv-output (i.e. your plugin name) targets in CLion (top right corner - for both targets select the target and then press "hammer" button )
  • Run > Edit configurations... > select "ipfixcol2" in the left column and in the right column add -c /your/path/to/startup.xml to "Program arguments" > OK.
  • Add breakpoint, for example, within ipx_plugin_init() function of your plugin.
  • Run > Debug... > ipfixcol2

It is possible that if you haven't build the plugin, the collector tried to load plugin from the system. If the problem remains, you can also try to force the collector to load plugin file from a specified location by adding -p path/to/libdsv-output.so to "Program arguments". Usually you can find .so file in build directory - in your case, it should be something like ipfixco2/cmake-build-debug/src/plugins/output/dsv/libdsv-output.so.

By the way, if you need help with writing the plugin, feel free to ask...

  • 👍1
korbind commented

Thanks for the update, now everything works for NAT events.