Simple and powerful crawler. It scraps content and collects links from websites using request or phantomjs. The whole magic and simplicity is behind configuration.
Work currently in progress and project in beta version but tested locally with big data and works fine.
Inspired by https://github.com/sylvinus/node-crawler/ and python http://scrapy.org/