Web Scraping using Scrapy

Install scrapy

conda install -c conda-forge scrapy pip install Scrapy

Create a Scrapy Project

scrapy startproject demo_project

Start the crawl

scrapy crawl goodreads(spider name)

Save the results as a csv, json

scrapy crawl goodreads -o data.json