/scrapy-deltafetch

Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls

Primary LanguagePython

No issues in this repository yet.