alecxe/broken-links-checker

Usage

Configure urls to parse in config.py. Then, run:

scrapy runspider broken_links_spider.py -o output.json

Then check 404 items in the output.json file.