nirala96/Web_Scraping_scripts

Some Web Scraping Scripts using scrapy written in python that scrapes the data from a website

Web Scraping Scripts

Overview

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

Check the Scrapy homepage at https://scrapy.org for more information, including a list of features.

My Script contains

• IMDB in multiple domains
  1. Most popular movies.
  2. Lowest rated movies.
  3. Most popular TV shows.
  4. Top rated TV shows.
  5. Top rated movies.
• All restaurants in gandhinagar (Gujarat) from zomato.
• Basic scraping of quotes.scrape.com

Technologies used

- Scrapy

Languages used

• Python
• SQL

Requirements

• Python 3.6+
• Works on Linux, Windows, macOS, BSD

Install

The quick way::

pip install scrapy

See the install section in the documentation at https://docs.scrapy.org/en/latest/intro/install.html for more details.

Documentation

Documentation is available online at https://docs.scrapy.org/ and in the docs directory.

Releases

You can check https://docs.scrapy.org/en/latest/news.html for the release notes.