🎼 Partoche
Terminal based pew pew map that feeds on Melody logs
Table of Contents
Usage
usage: partoche.py [-h|--help] (-D|--demo | -d|--data DATA) [-k|--kind {melody,elasticsearch}]
Quickstart
- Check the Requirements section to make sure that everything is setup correctly
- [OPTIONAL] Install virtual environment
virtualenv ./venv --python=$(which python3)
- or
make venv
- Install Python requirements
./venv/bin/pip install -r requirements.txt
- or
make reqs
- Note that the Elasticsearch package version is fixed to 7.10.0 in the
requirements.txt
file. You need to install the right version if your instance uses another one. Use./venv/bin/pip install elasticsearch==X.X.X
to install the right package, e.g../venv/bin/pip install elasticsearch==7.9.0
for Elasticsearch 7.9.0.
- Copy the default config file
cp config/config.sample.yml config/config.yml
- Install
asciiworld
(see the "Requirements" section) - Pull the GeoIP dataset (see the "Requirements" section)
- (OPTIONAL) Pull and consolidate the reputation dataset using the
listbot
script (see the "Requirements" section) - Run the program using the demo dataset
./venv/bin/python partoche.py --demo
- or
make demo
Requirements
A few things are needed before starting Partoche for the first time.
Asciiworld
Partoche uses asciiworld
to generate the ascii map. It needs to be installed and in path before running the program. Refer to these instructions to install it.
GeoIP
Partoche uses the GeoLite2-ASN
and GeoLite2-City
MaxMind databases to get coordinates for each hit. You need to pull them in the data/geoip folder before starting.
Refer to MaxMind's documentation to obtain them (it's free).
IP Reputation file
A local IP reputation dataset can be build locally using multiple sources and used to enrich the logs data. IP reputation data adds emojis next to hit headers.
Log type
The -k|--kind
flag is required to specify the kind of logs you're feeding Partoche with (default: melody). This is needed since the program can work using either an Elasticsearch live feed or flat melody log files.
Melody
The program should work out of the box with Melody flat log files.
However, using an Elasticsearch live feed expects enriched logs through a Logstash setup that will add IP reputation data using the listbot
database on the fly.
The enrich.py
script available at the project's root can be used to automatically add this data to the flat log files by updating each row based on an IP reputation database. This database should be updated regularly (ideally before each enrich.py
run) by running the data/listbost/listbot.sh
script, which will pull and build the database at data/listbot/live/iprep.yaml
.
That the script must be started in the data/listbot
directory. We recommend to use make iprep
to build/update the database.
Note that the updated version is saved on another file by default; you can use the -i|--in-place
to update the log file directly.
enrich.py
example
- Create an updated copy of the
melody.ndjson
file to./parsed.ndjson
:./venv/bin/python enrich.py --data melody.ndjson
- Create an updated copy of the
melody.ndjson
file to./custom_output.ndjson
:./venv/bin/python enrich.py --data melody.ndjson --out custom_output.ndjson
- Save the updated rows directly to
melody.ndjson
:./venv/bin/python enrich.py --in-place --data melody.ndjson
Elasticsearch
The program can also feed from an Elasticsearch instance to which live Melody logs are sent. In this scenario, the interval
specified in the config/config.yml
file will be used to query the corresponding timerange. The collected data will be automatically refreshed for a continuous display.
Note that to use this mode, you must create an API access with access to the indexes handling Melody data and update the config/config.yml
file accordingly.
Limitations
Partoche has been developed and tested with Elasticsearch 7.x and Python 3.8.
Reputation emojis
These can be customized in the config/config.yml
file.
Default:
reputation_emoji:
"👹":
- known attacker
- bad reputation
"🥸":
- anonymizer
"🤖":
- bot
- crawler
"🔎":
- mass scanner
"â‚¿":
- bitcoin node
"📨":
- spam
- form spammer
"🧅":
- tor exit node
"🧟":
- compromised
matches_emoji:
"âš¡":
- cve
profile_emoji:
"💣":
- dropper
action_emoji:
"🔑":
- login