About Darksearch
Darksearch allows you to query cached onion sites, irc chatrooms, various pdfs, game chats, blackhat forums etc...
Technologies
- Tor and Scrapy for web scraping
- Apache Kafka for streaming messages
- Apache Tika for text translation
- Postgres for the database
- Elasticsearch as an index
- Flask/flask-api/Gunicorn for the server
- Nginx for reverse proxy
The Darksearch index is growing as more scrapers get built...