This project is a production grade MVP of Dukaan(DIY platform to create your own E-Commerce store).
In this project I've used Django for the backend, PostgreSQL for production database, Introduced caching(Redis) mechanism
for increasing the throughput of the server and used event+time driven caching invalidation mechanism for invalidating cache.
Used Gunicorn for the production server, Nginx for revrese proxy and for serving static files.
Then I have Dockerize my project into 4 containers. i.e Web(Django+Gunicorn), db(PostgreSQL), redis_db(Redis), nginx(Nginx).
And finally I've used Docker Compose for running multiple containers as a single service.
I've deployed this project in AWS EC2 instance(ubuntu server, 1 gig ram)
Go to http://ec2-13-233-115-87.ap-south-1.compute.amazonaws.com/ and login via below credential(I've already populated the DB for the demo)
username:
demo
password:
demo1234
(Note: While creating the project my aim was to focus enterily in the Backend part and to implement state of the art Backend Infra.
So, I've created a very simple frontend by using just html and css. I'm able to do this by leveraging DTL and using my engineering jugadu mind)
The web app is having minimal functionality of Dukaan. i.e Vendors can add their products and manage them using a dashboard.
Then there is a unique link for each vendor which they can send to their customers. By using this unique link, customers can place the order
by entering the neccesary details. Each placed order then are displayed in the order's dashboard of vendor.
Below are the snapshot of the web app.
Signup Page (http://127.0.0.1:8000 #eg url for showing route
) 👇
Login Page ( http://127.0.0.1:8000/login/ #eg url for showing route
) 👇
Vendor's "Your Product Dashboard" (http://127.0.0.1:8000/your_product/ #eg url for showing route
) 👇
Vendor's Add NeW Product Form ( http://127.0.0.1:8000/new_product/ #eg url for showing route
) 👇
Unique Link To Buy Product From vendors (http://127.0.0.1:8000/order/demo #eg url for showing route
) 👇
Vendor's Your Orders Dashboard (http://127.0.0.1:8000/your_order/ #eg url for showing route
) 👇
Suppose there's a sale in a particular day in a vendor's dukaan shop. In this day the website is going to face high amount of traffic and this is gonna cause heavy load in the backend.
When customers visits the vendor's unique link for buying products.
Then for every visit, django is creating same response again and again. Because of this the throughput of the
server is gonna be decrease drastically.
Let's test throughput speed by bursting 100 requests in the vendor's unique link(I have already populated the db with dummy data)
For completing 100 requesting it's taking around 3.38 sec
. Pretty slow right?
For copinng with this problem I've implemented caching mechanism(redis). Which create a view level cache for each vendors
order
view(this view is mapped to vendor's unique link).
Now, let's again test throughput speed by bursting 100 requests in the vendor's unique link but this time having cache enabled .
Insane, this time it took only 0.95 sec
to process 100 requests. Which is roughly 72%
increase in throughput 🔥 🔥 🔥
But what will happen when the vendor adds a new product or make the product unavailable? The customers will still get the cached page.
For this problem we've to invalidate cache. Cache Invalidation is a crucial part in any caching mechanism system.
I've implemented two cache Invalidation technique in this project.
- Event based cahce invalidation.
- Time Based cache Invalidation.
Let's see it in action and monitor what is happening underhood using redis-cli monitor
command.
When the customer first visits the unique url http://127.0.0.1:8000/order/demo #eg unique url
then django will accept the request do some query in the Product
model and then generate the order
page
simultaneously it cahced the response in the Redis Memory and then send the response back to the customer.
After that any subsequent request to the unique url will not hit the db rather it'll get the requested page from the cache which is stored
in the Redis memory.
- When the customer first the unique url
http://127.0.0.1:8000/order/demo
then this request hit the db do the costly query and generate theorder
page.
and this page is then stored in the Redis Memory.
You can see the$SETX
redis command is executed for storing the cache. 👇
- When we reload the page you can see that this time django is not hitting the db and hence not doing the costly query
but rather than it's getting the requested page from Redis Memory.
You can see the$GET
redis command is executed for sending the requested page. 👇
- Whenever the vendor is doing any modification in the
Product
model we're inavalidating the stored cache.
i) Event Based Cache Invalidation: Whenever the vendor is adding "New Product" or making the available product
available/unvaliable by using toogle "Yes" or "No" buttons, we're Invalidating the cache.
ii) Time Based Cache Invalidation: All the stored cached will automatically gets invalidated after15 mins
. When we're making one of the product unavailable the cache are getting Invalidated. You can see the$DEL
redis command is executed for Invalidating the cache. 👇
To run this project you need to install Docker and Docker Compose.
Checkout the documentation if you don't have
How to install Docker
How to install Docker Compose
After that open terminal and clone the repo
$ git clone https://github.com/abhinavsp0730/Dukaan-Prototype
$ cd Dukaan-Prototype
Update the file permissions for making entrypoint.sh
executable
$ chmod +x app/entrypoint.sh
Then simply run
# running the docker compose script to buid & spin up docker compose service.
$ docker-compose -f docker-compose.prod.yml up -d --build
# for doing the django migration
$ docker-compose -f docker-compose.prod.yml exec web python manage.py migrate --noinput
# copying the static files to right dir so that Nginx can serve them
$ docker-compose -f docker-compose.prod.yml exec web python manage.py collectstatic --no-input --clear
Now visit,
http://localhost:1337/
Yay, your production grade dukaan-protoproject has succefully up and is running 🎉 🎉 🎉
You can verify if it's working properly by runining and then looking the logs
$ docker-compose -f docker-compose.prod.yml logs -f
You can spin down the docker compose service by runing
$ docker-compose -f docker-compose.prod.yml down -v
While I was creating the project I didn't find the example of the exact tech stack in the internet.
Also, there are lots of other thing happening in this project which I didn't cover in the readme.md
intentionally.
So, if you want a tutorial blog for this project then send me mail at abhinavsp0730@gmail.com
or say a hello to me in twitter @NeurlAP
I made this project for getting an internship at Dukaan in the position of django backend intern.
I'll update it below wheter I'm selected or not
Result
I got the internship and then got promoted to
full time Backend Engineer at Dukaan.