Suricata Stats
MyCodeRocks opened this issue · 9 comments
Hi @robcowart
Just a question I am sending the suricata eve file via filebeats to my ELK server. I see there is a stats log file as well but when specifying this log in the filebeats on the pfsense server they not getting ingested into the ELK server. The normal eve files works really well and thank you so much!
Any advise on what to check for?
Hi,
The Pfsense logfile does not work good with eof. So filebeat cannot read it well.
Try syslog instead. Been there.
Greetz
Thierry
@MyCodeRocks in your suricata.yml file you have to enable - stats: being added to the eve.json file by enable it under the - eve-log: output section.
This is my output config...
# Configure the type of alert (and other) logging you would like.
outputs:
# Extensible Event Format (nicknamed EVE) event log in JSON format
- eve-log:
enabled: yes
filetype: regular
filename: eve.json
rotate-interval: day
metadata: yes
community-id: true
community-id-seed: 0
xff:
enabled: yes
mode: extra-data
deployment: reverse
header: X-Forwarded-For
types:
- alert:
metadata: yes
tagged-packets: yes
- anomaly:
enabled: yes
types:
decode: no
stream: no
applayer: yes
packethdr: no
- http:
extended: yes
custom: [ accept, accept_charset, accept_datetime, accept_encoding, accept_language, accept_range, age,
allow, authorization, cache_control, connection, content_encoding, content_language, content_length,
content_location, content_md5, content_range, content_type, cookie, date, dnt, etag, expires, from,
last_modified, link, location, max_forwards, org_src_ip, origin, pragma, proxy_authenticate,
proxy_authorization, range, referrer, refresh, retry_after, server, set_cookie, te, trailer,
transfer_encoding, true_client_ip, upgrade, vary, via, warning, www_authenticate, x_authenticated_user,
x_bluecoat_via, x_flash_version, x_forwarded_proto, x_requested_with ]
dump-all-headers: [both]
- dns:
enabled: yes
version: 1
requests: yes
responses: yes
#formats: [detailed, grouped]
#types: [a, aaaa, cname, mx, ns, ptr, txt]
- tls:
extended: yes
session-resumption: yes
custom: [ subject, issuer, session_resumed, serial, fingerprint, sni, version, not_before, not_after,
certificate, chain, ja3 ]
- files:
force-magic: no
# force logging of checksums, available hash functions are md5, sha1 and sha256
force-hash: [md5]
- drop:
alerts: yes
flows: all
- smtp:
extended: yes
custom: [ bcc, content-md5, date, importance, in-reply-to, message-id, organization, priority, received,
references, relays, reply-to, sensitivity, subject, user-agent, x-mailer, x-originating-ip ]
# you need to set app-layer.protocols.smtp.mime.body-md5 to yes
md5: [ body, subject ]
- dnp3
- ftp
- rdp
- nfs
- smb
- tftp
- ikev2
- krb5
#- snmp
- sip
- dhcp:
enabled: yes
extended: yes
- ssh
- stats:
totals: yes
threads: no
deltas: no
# bi-directional flows
- flow
# uni-directional flows
#- netflow
# Metadata event type.
- metadata
Thank you @ipworkx
@robcowart - thank you - so I understand correctly
- That is your filebeats config or your suricata config file on the firewall / suricata box?
Trying to understand if its changing suricata's config or if its a config in: synesis_lite_suricata
Thank you both once again
That is part of my Suricata config. I am running Suricata standalone, not as part of a firewall appliance, but as long as you can edit the Suricata config it shouldn't make a difference.
Thank you looking at the Suricata config now. I see stats is enabled but I think its creating a sperate stats file and not in the eve.json
I see it creates a stats.log file and doesn't add it to the eve.json.
/var/log/suricata/suricata_<interface_name>/stats.log
Are the logs suppose to be part of the eve.json file?
You have to enable stats collection. Then you also enable whether it will be part of particular log (for those logs types that support it). If you enable it for the - eve-log: output section, as I did above, stats will be included in the eve.json file.
@robcowart thank you so I have the following now:
`
-
eve-log:
enabled: yes
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: eve.json
metadata: yes
community-id: true
community-id-seed: 0- stats: totals: yes threads: no deltas: no- ssh
`
- ssh
Then checking the log:
{"timestamp":"2020-06-08T12:25:02.437203+0200","flow_id":2081667870986982,"in_iface":"cccc.4090","event_type":"fileinfo","src_ip":"y.y.y.y","src_port":80,"dest_ip":"x.x.x.x","dest_port":9260,"proto":"TCP","http":{"hostname":"xxxxxx.synology.com","url":"\/onedrive_proxy\/check_new_notification.php?major=1&minor=1&subscription_id=WLS_SubscriptionId","http_content_type":"application\/json","http_method":"GET","protocol":"HTTP\/1.1","status":200,"length":54},"app_proto":"http","fileinfo":{"filename":"\/onedrive_proxy\/check_new_notification.php","sid":[],"gaps":false,"state":"CLOSED","sha256":"xxxxxxxxxxxxxxxxxx","stored":false,"size":45,"tx_id":0}}
Just checking above I did enable it? Checking your config file
We can close this thread. Got it to work (with the eve.json).
In Pfsense (I have the appliance) you can't edit the raw config file as it causes some unknown issue in the background and is not persistent through reboots.
I went and looked through all the settings and deep down in one of the settings there is check boxes for the stats and extended settings you had in your config file above. After enabling this and restarting the Suricata service I now have stats etc streaming into ELK.
Thank you again @robcowart