The aim of this project is to create a small application from scratch and apply some good devops principles to the project. The project is about learning and training devops, not about the backend/frontend aspects of it, but they are included as DevOps is by nature a discipline that touches all aspects of development
For the first project we won't focus too much on security.
- Create a simple python+flask application that has 4 endpoints:
- Healthcheck endpoint: This returns a 200 when the application is healthy
- An endpoint to save your name and surname (assume names and surnames are unique)
- An endpoint that returns all the names saved
- An endpoint to query your surname based on the name you give
- The application should use postgresql as the backend database.
- The postgresql database must be run in docker
- Dockerize the service and add everything to a docker-compose file
- Create another, different, python service asynchronously writes phones numbers linked to names+surnames
- When adding a name+username, add an additional field to the 1st service to add numbers.
- Communication between services should happen with a broker based system. I suggest using rabbitmq.
- Modify the first service to also fetch the associated number when queried with the name.
- The rabbitmq broker should be deployed in docker
- Add everything to the docker-compose. The 2 python services should be 2 separate containers running.
- Add Grafana and Prometheus to the docker-compose and get them talking.
- Use prometheus to monitor you healthcheck endpoints
- Install prometheus-flask-exporter to get specific flask based metrics
- Monitor rabbitmq with prometheus
- Make some cool dashboards in grafana to visualize all this.
- Add some basic info logging to you apps ("added JOHN DOE to the database")
- Add loki to you docker-compose
- You promtail to scrape the logs from you services and write them to loki
- View the logs in grafana and add them to the dashboard
- Use promtail to scrape rabbitmq logs and add them to the dashboard
- Use github-actions to automatically build and push your container images to a container-registry (dockerhub, ghcr, etc.)
- Write a basic unit test for the services using any framework you like
- Run the unit tests in the CI in a place that makes sense. If the tests fail, the containers should NOT push.
- Create an AWS account
- Using free-tier, create a EC2 server and ssh key that goes with it
- Using github-actions, deploy the docker-compose to the EC2 server
If you have any questions, feel free to contact me at wernich.bekker@gmail.com or create an issue on the repo.