This project is an advanced tool specifically designed for Amazon traders aiming to optimize their process of searching and comparing products on the platform. The tool employs web scraping techniques to gather detailed product information from Idealo, a reference platform for finding deals and competitive prices on various products.
It provides an intuitive graphical interface that enables users to collect product URLs from Idealo. This is accomplished by selecting the number of "workers," streamlining the link retrieval process, and maximizing the efficiency of scraping.
The product extractor is the central part of the tool. Once the product URLs are collected from Idealo, it uses advanced scraping techniques (Selenium and BeautifulSoup) to compare prices and details of products on Amazon. Users can configure specific filters, such as a minimum profit percentage and a minimum euro profit value. The extractor filters products that meet these criteria and stores relevant details, such as product name, prices, profit percentage, and links, in an Excel file. This function enables traders to identify efficiently and organizedly products that could offer more profitable business opportunities on Amazon.
- Create a virtual environment:
- Linux/Mac:
python3 -m venv venv - Windows:
python -m venv venv
- Linux/Mac:
- Activate the virtual environment:
- Linux/Mac:
source venv/bin/activate - Windows:
venv\Scripts\activate
- Linux/Mac:
- Install the libraries:
- Run
pip install -r requirements.txt
- Run
- Execute
index.pyto update the list of URLs with all products (products.txt):python index.py
- Once the list of products is obtained, run
get_products.pyto start analyzing all products:python get_products.py
- Let the magic happen; you can observe the Excel file
products.xlsxupdating in real-time.