From a terminal
- Install Requirements
pip3 install -r requirements.txt
- Add URLS to urls.txt
- Set selector in selectors.yml
- Run
python3 single_page.py
- Get data from output.jsonl
This scraper only scrapes product from the first page of search results
- Add URLS to search_results_urls.txt
- Set selector in search_results.yml
- Run
python3 searchresults.py
- Get data from search_results_output.jsonl
Each result would look similar
{
"title": "Dell Latitude E6430 Laptop WEBCAM - HDMI - Intel Core i5 2.6ghz - 8GB DDR3-128GB SSD - DVD - Windows 10 Pro 64bit - (Renewed)",
"asin": "B01M293O5P"
}