Video To Audio Conversion Microservices

A microservices-based video-to-audio conversion system that utilizes Kubernetes to orchestrate five containerized services.

Note: The design image above depicts how the services communicate with each other, and not the direction of data flow.

Gateway service

Handles requests from users. It makes requests to the auth service to create new users (signup), obtain json web token via the login endpoint and validates users for the upload and download requests.

When authenticated users uploads new video files, it stores the video files to MongoDB (using GridFS) and publishes the file metadata into the default exchange of the RabbitMQ broker with video routing key (broker puts the metadata in video queue)

It is exposed to the public via Kubernetes Ingress with a custom host name (added to /etc/host).

Auth service

It uses JWT to authenticate users. It receives requests from the gateway service to signup, login and validate users. Uses MySQL database for user management. It is not exposed to the public, that is, only accessible within the kubernetes cluster.

RabbitMQ Broker

Has 2 queues video and audio.

Video queue: Publisher is the gateway service and consumer is the converter app.
Audio queue: Publisher is the converter application and consumer is the notification application.

Converter

Picks video metadata from the video queue in rabbitmq, converts the video to audio file, stores the audio file in MongoDB (using GridFS) and publishes the audio file metadata to the audio queue in rabbitmq.

Notification app

Picks audio metadata from the audio queue in rabbitmq and notifies the owner via email

Usage

Prequisites

Docker
Kubernetes
Minikube
MongoDB
MySQL

How to setup locally

Ensure you have all the prequisites are installed and running

Run minikube start to create an active Kubernetes cluster
Run minikube addons enable ingress to enable Kubernetes Ingress API
Add the following in your /etc/hosts file:
```
# Added for video2audio microservices system
127.0.0.1 video2audio.service
127.0.0.1 rabbitmq-manager.service
```
This gateway ingress uses video2audio.service as host and the rabbitmq ingress uses the rabbitmq-manager.service as host. You can change the hostname (after updating the hostname in the two ingress files)
Run minikube tunnel to start a tunnel process that create a network route with the host machine to allow external traffic reach services exposed by ingress rules (gateway and rabbitmq service)
Update secret.yaml in notification manifests folder with your smtp details. (If you do not have an smtp host, You can create a test gmail address and use google smtp configurations)
Run the following from the project directory:
- kubectl apply -f ./auth/manifests/
- kubectl apply -f ./gateway/manifests/
- kubectl apply -f ./rabbitmq/manifests/
- kubectl apply -f ./converter/manifests/
- kubectl apply -f ./notification/manifests/

Further Considerations:

Add simple frontend
Use AWS S3 for file management instead of MongoDB
Automate local setup with bash script
Include setup for production environment

tolopsy/video-2-audio