/himpub-web-scraper

A web scraper that scrape all the book data from the website and stores in a sqlite3 database.

Primary LanguagePython

himpub-web-scraper

A web scraper that scrape all the book data from the himpub and stores in a MYSQL database. The scraper use concurrency features for faster scraping. It currently scrapes around 3K records in 4 minutes.

Instructions

Clone the repository Install necessary modules: pip install requests pip install beautifulsoup4

run the scraperToDb.py file and the scraper will scrape and store the data in MYSQL database.