/web-crawlers

web scraping

Primary LanguagePython

Scrapy Examples

Scrapy is an application framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival.

To run the spider using the runspider command.

scrapy runspider <file_name>.py -o <file_name>.json Scrapy Documention https://docs.scrapy.org/en/latest/index.html