Script build to download data from a flaky API.
This is a take-home interview for HomeVision that focuses primarily on writing clean code that accomplishes a very practical task.
Given a simple pagianted API hosted at http://app-homevision-staging.herokuapp.com/api_project/houses?page=1 that returns a list of houses along with some metadata. The task is to write a script that accomplishes the following tasks:
- Requests the first 10 pages of results from the API
- Parses the JSON returned by the API
- Downloads the photo for each house and saves it in a file with the format:
id-[NNN]-[address].[ext]
There are a few gotchas to watch out for:
- This is a flaky API! That means that it will likely fail with a non-200 response code. The code must handle these errors correctly so that all photos are downloaded
- Downloading photos is slow so please think a bit about how you would optimize your downloads, making use of concurrency
On the project root
docker-compose up
You'll need to create and activate a new python3.6 virtual enviroment and configure the respective Mongo database connection on the config.yml file.
pip install -r requirements.txt
python3.6 start.py