This was just for Fun-type project to scrap data from various web site. Here we used Python and it’s some powerful packages like selenium, beautiful soup , scrap , request etc.
Selected Sites for data scraping were :
Amazon
- Scrap best seller book list
- Scrap most valuable comment
- Describing info for a book
IMDB
- Download all top rated movie poster
- Scrap top rated movies
Linked In
- Scrap information of target individual
- Load n times pages at a time
Udemy
- Scrap list of best selling courses
Downlaod Python 3x from the official website.
pip install selenium
pip install BeautifulSoup
pip install requestsor just download anaconda distribution for get all the necessary pacakges simultaneously.
Download Chrome WebDriver or Download PhantomJS
git clone https://github.com/iphton/Data-Scraping
cd Web-Scraping
jupyter notebook
Go do the desired notebook, files that end with '.ipynb'. To run the model, go to the menu then click on Cell > Run all
Regards
World of Void.