[iptables]: not properly parsing VyOS logs
Opened this issue · 2 comments
Integration Name
Iptables [iptables]
Dataset Name
No response
Integration Version
1.16.1
Agent Version
8.15.0
Agent Output Type
elasticsearch
Elasticsearch Version
8.15.0
OS Version and Architecture
Ubuntu 22.04.4 LTS
Software/API Version
No response
Error Message
No response
Event Original
[7598791.530254] [RULE_IN-2001-A] IN=eth0 OUT=eth1 MAC=00:1b:21:bc:c7:ba:52:54:00:4a:b3:d0:08:00 SRC=10.10.10.10 DST=10.11.11.11 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=5751 DF PROTO=UDP SPT=49704 DPT=53 LEN=64
What did you do?
I forward IPTables logs from a VyOS firewall to an agent where I use the Custom UDP input, with Syslog parsing enabled.
Then in the custom ingestion pipeline for that dataset, I tell it to call the logs-iptables.log-1.16.1 pipeline.
What did you see?
some of these logs, at least some DROP logs, seem to be parsed fine with the UBIQUITY_LABEL, but others, at least what I saw now, ACCEPT or REJECT logs, have a "space" between the [UBIQUITY_LABEL] and the following IN=.... but the 4th grok pattern in the ingest pipeline Grok "Extracts values from "message" that match a grok pattern", looks alike:
%{GREEDYDATA}\[%{UBIQUITI_LABEL}\]%{IPTABLES}%{SPACE}
due to the missing space, it didn't matched, and moved on with the more general pattern below. So I only ended up with event.action: drop events, and the others don't have that field set.
What did you expect to see?
Since VyOS more or less the same naming convention for its rules, the IPTables integration should parse them same way as Ubiquity IPTables logs. So that event.action will properly be set to accept, drop, reject.
Anything else?
updating above mentioned grok pattern to:
%{GREEDYDATA}\[%{UBIQUITI_LABEL}\]%{SPACE}?%{IPTABLES}%{SPACE}
did the trick for me. now in discover, I can see "drop" and "accept", for some reason, when there are reject rules I only see "r", haven't yet found where the event.action is expanded from the letters to the long names.
Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)
my proposed solution seem to have caused a redundant nested repeat operator * error.
Seems SPACE is implemented as \s* and with that, the trailing ? caused errors, update to:
%{GREEDYDATA}\[%{UBIQUITI_LABEL}\]%{SPACE}%{IPTABLES}%{SPACE}