/Scraper_bookstoscrape.com

This repository contains codes which I wrote during learning Scrapy

Primary LanguagePythonMIT LicenseMIT

Scrapy Spider bookstoscrape.com

This repository contains codes which I wrote during learning Scrapy by building a spider to scrap data from Books To Scrap Website.

Scrapped Wesbite

Wesbite: Books to Scrape

How to run

Running locally

  1. Clone the repository
git clone https://github.com/scienmanas/Scraper_bookstoscrape.com.git
  1. Install the requirements
pip install -r requirements.txt
  1. Run the spider
scrapy crawl bookspider

Running on Scrapy Cloud

  1. Create a new Scrapy Cloud project
  2. Deploy the project
shub deploy
  1. Run the spiderpider
shub crawl books

Purpose

The purpose of this repository is to learn Scrapy and to help others who are learning Scrapy.

The webiste scraped is open to scrapping to individuals to learn and test we scrapping

Technologies Used

  1. The Project uses ScrapeOps APIs for proxies, handling headers and user agents .
  2. The Website can be accessed by the link : https://scrapeops.io/