This repository contains codes which I wrote during learning Scrapy by building a spider to scrap data from Books To Scrap Website.
Wesbite: Books to Scrape
- Clone the repository
git clone https://github.com/scienmanas/Scraper_bookstoscrape.com.git
- Install the requirements
pip install -r requirements.txt
- Run the spider
scrapy crawl bookspider
- Create a new Scrapy Cloud project
- Deploy the project
shub deploy
- Run the spiderpider
shub crawl books
The purpose of this repository is to learn Scrapy and to help others who are learning Scrapy.
The webiste scraped is open to scrapping to individuals to learn and test we scrapping
- The Project uses ScrapeOps APIs for proxies, handling headers and user agents .
- The Website can be accessed by the link : https://scrapeops.io/