/domain-crawler

A domain crawler that crawls whois information of domains

Primary LanguagePythonApache License 2.0Apache-2.0

Domain CrawlerBuild Status

Introduction

This is a domain crawler that crawls whois information of domains.

How to run

First, launch a Tor container,

docker run -d --name tor -p 9050:9050 -p 9051:9051 -p 8118:8118 soulmachine/tor

Second, launch a MongoDb container,

docker volume create mongodb-data
docker run --name mongodb -p 27017:27017 -v mongodb-data:/data/db -d mongo --serviceExecutor adaptive

At last, start the crawler,

python3 whois_crawler.py google-10000-english-usa-no-swears-short.txt ai
python3 whois_crawler.py google-10000-english-usa-no-swears-short.txt coin prefix com