Python Simple Crawler A simple base crawler write in Python Requirements Python 3.x Installation I recommend using virtualenv for environment building $ git clone https://github.com/volneyrock/SimpleCrawler $ cd SimpleCrawler $ pip install -r requirements.txt #Running $ python crawler.py