This app was written during the cars2click recruitment process.
Its purpose is to scrape the car dealership data from this website. The output is a JSON file containing a list of data points with the following format:
{
"brand": "Car Brand",
"dealer": "Car Dealership",
"address": "The address",
"tel": "0123465789"
}You need Python 3.6+ and the Scrapy Python library. I recommend using a virtual environment.
To create a virtual environment run python3 -m venv venv. Then you need to activate it by running source venv/bin/activate.
Install all the required components (in the activated virtual environment) by running python -m pip install -r requirements.txt from the root of the project.
For more information check the following links:
To run the scraper you have to run the following command from the concession directory:
scrapy crawl concession -O results.jsonThis command will create a JSON file called results.json in the concession directory which contains the scraped data. From my testing the crawler manages to scrape 4882 data points.