/crawler

Crawl a single website

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

Web crawler using asyncio coroutines

Install

$> git clone https://github.com/KKfo/crawler.git

Requires python 3.5 >=

$> pyvenv env
$> source env/bin/activate
$> pip install -r requirements/production.txt