Wireless Analysis With Kismet and ELK

A project for analyzing Kismet data in the ELK stack.

Presented at Bsides Idaho 2020 (https://www.youtube.com/watch?v=MU3hu3WuRA0). Slides in repo (Wireless Analysis with Kismet and ELK (WAWKELK).pdf).

Zero to Hero

Download the Bitnami VM (https://bitnami.com/stack/elk/virtual-machine)
Configure the network interface as appropriate, then start the Bitnami VM
Note the IP address and Kibana username and password
Log in and change the default Linux password (bitnami/bitnami)
(If you want to use SSH) Remove the SSH blocker: rm /etc/ssh/ssh_to_not_be_run
(If you want to use SSH) Start the SSH server: sudo systemctl enable ssh && systemctl start ssh
Copy the two Logstash pipeline files into /opt/bitnami/logstash/pipeline/
Change the URL and username/passwords in each file to match your own settings
Launch Kibana in web browser
Navigate to Management > Dev Tools
Using the values in mappings.md, create the kismet_alerts and kismet_device_activity mappings
Restart Logstash sudo /opt/bitnami/ctlscript.sh restart logstash
Navigate to Management > Stack Management > Index Patterns
Wait until data has been populated (based on the timers in the Logstash pipeline files)
Click Create Index Pattern
Create an index pattern for each index
Navigate to Discover, select an index pattern, and browse the data
Create visualizations for the data
Create dashboards of the visualizations
Use the kismetdevices_to_elk.py to push already collected kismetdb files to Elasticsearch

Bitnami ELK Stack VM Notes

/home/bitnami/bitnami_credentials < File containing the username/password for Kibana/ELK/etc

/opt/bitnami/logstash/pipeline/ < Place pipeline conf files in here then restart Logstash

sudo /opt/bitnami/ctlscript.sh status < Check status of services

sudo /opt/bitnami/ctlscript.sh restart logstash < Restart logstash (or other services)

Elasticsearch

There are two provided Elasticsearch Index/Mappings in mappings.md. Use the Dev Console in Kibana for passing commands to Elasticsearch.

To delete an index with funky characters, you'll need to replace the special characters
- % → %25
- { → %7B
- } → %7D

Logstash

There are two provided Logstash pipeline files.

kismet_alerts.conf - For pulling Kismet Alerts
kismet_device_activity.conf - For pulling all Kismet devices observed in the last X minutes

Logstash notes

Test pipeline config changes via: sudo /opt/bitnami/logstash/bin/logstash -f /opt/bitnami/logstash/pipeline --config.test_and_exit
If using the .ekjson end point, set codec => "json_lines"
If using the .json end point, set codec => "json"
Content type is important; some issue with Manticore defaulting to an encoded type which broke Kismet -- headers => { "Accept" => "application/json" "Content-Type" => "application/x-www-form-urlencoded" }
Set the time to be an appropriate value (pulling too often results in more data captured than necessary)
document_id is used by Elasticsearch to deconflict documents (Primary key of sorts); if you want to do temporal analysis, add a timestamp value to the document ID; if you want to only have one document per device, then use the base_key or MAC address as the document ID
Add "stdout { codec => rubydebug }" in the output section when debugging
Logstash combines all the pipeline files so if you use multiple pipeline conf files, ensure your filters sections are looking at the index first, otherwise they will attempt to apply to all pipelines
Triple check syntax; look for missing "}" values
Monitor the log file tail -f /opt/bitnami/logstash/log/logstash.log

Kibana

Maps

Great resource for WMS options: https://www.elastic.co/blog/custom-basemaps-for-region-and-coordinate-maps-in-kibana
If you use Maptiler, here are the settings for the Kibana Coordinates Visualization:
- Options > Base layer settings
  - Enable WMS map server
  - WMS url: https://api.maptiler.com/maps/streets/{z}/{x}/{y}.png?key=
  - WMS layers: 1
  - WMS version: 1.3.0
  - WMS format: image/jpg
  - WMS attribution: "© OpenMapTiles|© OpenStreetMap contributors"
  - WMS styles: default

Dev Tools Console Helpful Commands

To delete an index

DELETE kismet_devices

To Reindex...

Push the current documents to a temp index

PUT temp_new_mapping_index
{
  "settings": {
    "index.mapping.ignore_malformed": true 
  }
}
POST _reindex
{
  "source": {
    "index": "original_index"
  },
  "dest": {
    "index": "temp_new_mapping_index"
  }
}

Erase the original index

DELETE original_index

Recreate the original index with the intended mappings

PUT original_index
{
  "settings": {
    "index.mapping.ignore_malformed": true 
  }
}

Now reindex the original data into the newly creating index with updated mappings

POST _reindex
{
  "source": {
    "index": "temp_new_mapping_index"
  },
  "dest": {
    "index": "original_index"
  }
}

Delete the temp index

DELETE temp_new_mapping_index

cwilliams001/wawkelk