Repository is inactive. It has been moved to a private repo.
Crawler computer specs
- Python 2.7
- Pip
- Scrapy 0.24.4
- Py2neo 1.6.4
- Neo4j 2.1.4
- pymongo 2.7.2
- TorCtl
- Tor
Comming soon ...
Install Tor and set it up on port 9050.
Install
Clone the repo using git clone https://github.com/Frenesius/CrawlerProject56.git
Get in to the folder using cd Make Start.sh executable(run as root) chmod +x Start.sh
Make sure Neo4j database is up and running. User Neo4j-octo-love to create base database.
Execute ./Start.sh
and have fun
see Licence.md