ScraperBot
A bot to get product description, product sizes, product price
Installation
- run
virtualenv -p python3.6 venvto create a virtual environment - activate the virtual environment using
source venv/bin/activate - run
pip install -r requirements.txtto install all dependencies
Usage
-
run the program using
python scraperbot.py -
after running the program you will see a data.csv has been generated it contains the following data in the respective order
data.csv contents:-
- product title (string)
- product description (string)
- product price (string)
- product colors (array)
- product image url (string)
- product sizes (array)
Other Requirements
- To enable the proxy feature you need to ensure that you are running tor on you local computer, to download tor browser visit https://www.torproject.org/download/, once downloaded start the browser and set proxy=True while calling bot.start()