Tracking Tor Exit Nodes

These set of scripts fetch data on Tor exit nodes from the only - as of now - currently known valid and authoritative source: https://check.torproject.org/exit-addresses

Upon downloading the data, it gets inserted into a DB (directory: 'db/') There is a web interface written in python+flask in 'www/'

Directory Contents

www/		-> website stuff
db/			DB structure and initial data import mechanisms
data/		data directory

Prerequisites

wget
postgresql 9.3 or higher
python
flask
webserver such as nginx

See the requirements.txt file in www/tor

Initial setup

$ cd db
$ sudo su
# su - postgresql
$ createuser -s userename
$ psql template1 < db.sql

Testing if it works

Activate any virtualenv or conda environment in case you use that to install the prerequisites.
Test if fetching the data works:

First, make sure that the newly created user tordb may access the tables and the DB and add it to the postgresql pg_hba.conf file.

$ ./fetch-tor-list.sh 
psql -U tordb tordb_simple
select count(*) from node

you should see a non-zero result.

If it works, you can continue to run this automatically...

The execution of fetch-tor-list.sh is expected to output a lot of error messages like this:

ERROR:  duplicate key value violates unique constraint "idx_node_combined"
DETAIL:  Key (node_id, ip, exit_address_ts, id_nodetype)=(0011BD2485AD45D984EC4159C88FC066E5E3300E, 162.247.74.201, 2019-08-08 09:12:18+02, 1) already exists.

They can/should be ignored.

How to get this to run automatically?

$ crontab -l
(...)
# fetch the list once a day at 1:05 A.M.
# m h  dom mon dow   command
5 01   *   *   *     ( cd /home/your_user/torexitnodes_simple; source venv/bin/activate ; ./fetch-tor-list.sh  >/dev/null 2>&1 )

(Note that this assumes you installed the prerequisites via virtual-env).

Deploying the web interface properly

The built-in webserver of flask is a no-no for production environments. Hence, please follow the great documentation on production setups. The instructions vary depending on which web server you use.