Amazon Books Web Scraper using Python and Pandas
This program will scrape books from a list of Amazon links (spreadsheet). It will generate a csv file with book data such as:
- ASIN
- Name
- Author
- Stars
- ISBN10 and ISBN13 number
- Item category, subcategory and specific category
- Item availability
- Original price and sale price
Usage Example
Input links:
Usage:
Repeated links: The program can detect if there are repeated books based on its ASIN number, these repeated books won't be scraped.
Output data:
Getting Started
- Clone the repo
git clone git@github.com:guidosantillan01/amazon-books-web-scraper.git- Run
modules/main.pyfile:
PYTHON_PATH/python.exe "FOLDER_PATH/amazon-books-web-scraper/modules/main.py"Example:
C:/ProgramData/Anaconda3/python.exe f:/Downloads/amazon-books-web-scraper/modules/main.pyPrerequisites
- You will need Python. The Anaconda Distribution is recommended.
- Install the VSCode Python extension.
- Install these python libraries:
pip3 install pandas certifi urllib3- pandas 0.23.4 or greater is required, if you have an older version of pandas upgrade it with:
pip3 install --upgrade pandasImportant Considerations
- Check out variables.py file to modify the desired behavior of the program such as:
- Number of desired books to scrape.
- Change the name of the input and output files and its directory
- REGEX filtering queries
- Useful web scraping resources
- XPATHS for web scraping Amazon
Built With
- Python 3.7
- Pandas 0.23.4
- certifi 2018.8.24
- urllib3 1.23
Contributing
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
Authors
- Guido Santillan Arias - guidosantillan01@gmail.com - www.guidosantillan.com
Licence
This project is licensed under the MIT license - see the LICENSE file for details.