Nfdump 1.7.5 incompatibilities
Opened this issue · 7 comments
Recent install, building a new server with nfsen-ng instead of nfsen.
All flows are collecting properly, graphs / rrd's updating. When I issue any query, i get
something went wrong. firstSeen,duration,proto,srcAddr,srcPort,dstAddr,dstPort,packets,bytes,flows
and in some queries i get that as well as this, which is what looks like because $output is empty (the above issue? unsure)
Warning: Undefined array key 0 in /opt/nfsen-ng/nfsen-ng-0.3.1/backend/processor/Nfdump.php on line 175
I was running the HEAD/master version, and then switched to the last release 0.3.1 and get the same issue. I've turned on debugging which gives the nfdump command in the apache web logs - i can run it no probem as the www-data (apache) user, so permissions are ok. the values after "something went wrong" re the .csv headers - they change depending on the query, but i never see results.
Guessing this is something wrong compatibility wise - my system is I followed the INSTALL.md instructions (which were pretty great)
- Ubuntu 20.4.x
- PHP 8.1
- nfdump 1.7.5 (which, btw, is only two weeks old)
Here's sample data output from cli, in case the output from nfdump is somehow the problem
firstSeen,duration,proto,srcAddr,srcPort,dstAddr,dstPort,packets,bytes,flows
2024-11-08 12:00:22.272,0.000,6,10.150.47.2,45708,34.199.107.131,443,4096,212992,1
2024-11-08 12:00:03.328,31.744,6,10.150.47.2,443,69.129.204.33,59649,12288,4648960,1
2024-11-08 12:01:55.456,10.240,6,10.150.47.2,443,69.129.204.33,59649,8192,425984,1
2024-11-08 12:03:14.560,0.000,6,10.150.47.2,443,69.129.204.33,59649,4096,1167360,1
2024-11-08 12:03:51.424,0.000,6,10.150.47.2,443,69.129.204.33,59649,4096,1818624,1
2024-11-08 12:04:30.848,102.912,6,10.150.47.2,443,69.129.204.33,59649,49152,54398976,1
2024-11-08 12:07:03.680,141.568,6,10.150.47.2,443,69.129.204.33,59649,61440,57442304,1
2024-11-08 12:09:57.248,145.664,6,10.150.47.2,443,69.129.204.33,59649,69632,54509568,1
2024-11-08 12:09:57.248,275.200,6,10.150.47.2,443,69.129.204.33,59649,53248,37867520,1
Did a little more investigation here - i believe there are many issues, and perhaps the main one is the way nfsen 1.7.x outputs data.
Eyeballing processor/Nfdump.php does a lot of things with the output such as "remove the 3 summary lines at the end of the csv output" - these three lines aren't there in 1.7.5, but are in 1.6.x. haven't tested any other versions, will do more looking around next week but it seems to simply not like the output.
Strangely though, the "if last element contains a colon, it's not a csv" portion of code doesn't even work - it seems that that returns a non csv as a bunch of text, but still errors.
if I choose to download a csv file i get a nice error:
<br />
<b>Fatal error</b>: Uncaught TypeError: fputcsv(): Argument #2 ($fields) must be of type array, string given in /opt/nfsen-ng/gitmaster/backend/api/Api.php:103
Stack trace:
#0 /opt/nfsen-ng/gitmaster/backend/api/Api.php(103): fputcsv()
#1 /opt/nfsen-ng/gitmaster/backend/index.php(12): mbolli\nfsen_ng\api\Api->__construct()
#2 {main}
thrown in <b>/opt/nfsen-ng/gitmaster/backend/api/Api.php</b> on line <b>103</b><br />
In recent nfdump versions there has been mentions of a json output. Maybe this would be better suited...
I went from NFdump 1.7.5 -> 1.7.4 and things actually work now, so there's some incompatibility w/ 1.7.5. And indeed it says json output is supported since 1.6.17.
However, I also get "Unknown Extension ##" in the nfsen command output. This also comes through if i run the command from CLI as well. Guessing this is an Nfcapd/Nfdump issue, but perhaps they could be filtered out on the Nfsen-ng side?
Additional note - the graphs for "ports" wasnt working either - and now it is.
I went from NFdump 1.7.5 -> 1.7.4 and things actually work now, so there's some incompatibility w/ 1.7.5. And indeed it says json output is supported since 1.6.17.
However, I also get "Unknown Extension ##" in the nfsen command output. This also comes through if i run the command from CLI as well. Guessing this is an Nfcapd/Nfdump issue, but perhaps they could be filtered out on the Nfsen-ng side?
You have incompatible version of the collector nfcapd and the cli tool nfdump. The collector is newer.
Indeed as i had just downgraded. 1.7.4 -> 1.7.5
After waiting a day and doing a few tests on data since, downgrading, i am not getting this error.
So perhaps this ticket should be renamed to "nfsen-ng incompatible with nfdump 1.7.5 - want me to open a new issue for that and close this?