SiteCrawler - A Scrapy spider
Primary LanguagePython
scrapy runspider SiteCrawler.py --nolog -a url=<url>