Building an 2-way sync against a REST API in Python for a coding challenge. It uses MongoDB and MongoEngine as ORM.
The full instructions are available here.
To use this package, you need to run Python 3.7 or newer.
Start by installing the required packages from the requirements.txt file:
pip install -r requirements.txtThen, make sure to add the API endpoint in the .env file:
API_URL=https://SECRET.mockapi.io/api/v1Last, start your local MongoDB instance:
mongodYou can run Dosync with the following command:
python3 start.pyIf you need to force the syncronization again, you can remove the attributes in the metadata collection on your MongoDB instance.
To run the tests, run the following command from the root directory:
pytestThe decision was made to nest adsets, keywords and ads inside a unique campaigns collection in MongoDB.
An alternative architecture would be to create separate adsets, keywords and ads collections, mainly based on the use case and performance needs.
We only update campaigns that have been recently updated to avoid making unnecessary API calls to the remote API. To do so, we store and update the latest sync timestamps in a metadata collection in our database.
To avoid rate limiting from MockAPI.io, a 1-second delay was added before any API request.
We're facing an issue where MockAPI.io seems to be converting all integers to strings, so the updated data is never identical to the original one.
- Batch update in database to reduce number of write operations
- Add more unit tests for improved coverage, more particularly for the upstream sync as we have no test API endpoint for now
Nicolas Spehler – @NSpehler