-
Nodejs
-
Typescript
-
Express.js
-
MongoDB
-
Redis
-
Docker
-
FFMEPEG
-
Serverless ( for working with AWS lambdas Seamlessly )
-
AWS Related Services
S3 ( Simple Storage Service )
Lambda
ECS ( ECS with Fargate )
SQS
Event Bridge
- PORT
- MY_AWS_REGION
- MY_AWS_ACCESS_ID
- MY_AWS_SECRET_ACCESS_KEY
- MONGOOSE_DB_URL
- JWT_AUTH_SECRET_KEY
- REDIS_CLOUD_PASSWORD
- REDIS_CLOUD_HOST
- REDIS_CLOUD_PORT
- SQS_QUEUE_URL (AWS SQS Queue URL so you can enqueue the messages)
- SQS_ARN ( it is an ARN value of SQS service )
-
Lambda Functions
All lambda functions are located in the "
src/functions/
" path within the project structure. These functions are used by theserverless.yml
file.To push these lambda functions to AWS, run the following command:
serverless deploy
But before pushing it to AWS, make sure you have logged in using your AWS AccessID and Secret Access Key, and have also configured these credentials to the Serverless framework. You can do this by following the commands below:
// For aws login aws configure //For Configuring AWS Credentials to the Serverless serverless config credentials --provider aws --key "AWS AccessId" --secret "Secret Access Key" -o // Remember dont use quotes when specifying AWS AccessId and Secret Access Key
You can also create the environment to test this functions locally using this command
serverless offline
you can also invoke the function individually using this command
// JobConsumer is function name which is mentioned in the serverless.yml serverless invoke local -f JobConsumer
-
ECS ( for Video Transcoding service )
The Video Transcoding service is a node application located in "
src/VideoEncodingService/
". We create a Docker image from this service and push it to ECR.The Video Transcoding service has its own index file,
package.json
, andpackage-lock.json
file. If you are unsure how to push the image to ECR, you can go to the AWS console and visit the AWS ECR service. Create a Repository and click on the "View Pushing Commands" button at the top of the Repository for help with the commands.If you are a Windows user like me (slowly realizing that being a Windows user really sucks 😥😅), you may encounter errors while pushing the image to the ECR Repository. I found a fix for this by exporting the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If you are a Windows user, use
set
to export the environment variable. If you are a Linux user, useexport
.Before pushing the image, make sure to build it using Docker. Also, ensure that the Docker Compose App is running. After building the image, refer to the push commands provided by the AWS ECR service management console.
In the Docker File, you will see the following code:
COPY src/VideoEncodingService/package*.json . COPY tsconfig.json tsconfig.json RUN npm ci --maxsockets 1
Why do we use
RUN npm ci —maxsockets 1
? Let's find out.When I used
RUN npm i
, I encountered some errors related to I was unsure of what was happening, and it took me 3 days to resolve this issue (yes, embarrassing for a software developer 😂😅😥😥). Finally, I found the solution by using the commandRUN npm ci —maxsockets 1
. After successfully building this image, I never touched the Dockerfile again. It was a case of "If it Works, Don't Touch it".Before building the image, please ensure that the
package-lock.json
file is present. If it is not, the image building process will fail at the point ofRUN npm ci —maxsockets 1
. This is becausenpm ci
depends on thepackage-lock.json
file, not onpackage.json
.
If You want High Level Design of this project like How Video gets processing Then You Can Refer to Architecture of this Project Which is Mentioned at the Top of the Readme
You Can Use this using in the postman by importing the API using this URl : https://api.postman.com/collections/21349319-813be182-2835-42ef-ab8d-19a11e634bca?access_key=PMAT-01HKJ3ZGPDQD0X4TQ511N2A4XN
URl : {{base_url}}/video/signup
{
"userName":"guest",
"email":"guest@gmail.com",
"password":"123456"
}
Login
URl : {{base_url}}/video/login
{
"email":"guest@gmail.com",
"password":"123456"
}
Get Video Upload URL
Here you will send the name of the video you want to name it and video extension in URL path
this API will generate the Pre Signed URL of S3 Put Object and you will get that URL in response using that URL you will upload the video to S3 Bucket using this API called Upload Video URL which is mentioned Below
Remember you don’t need to add the video into body of this API this API will not handle the Video Uploading it is Handled By the Below API called Upload Video
We are doing this to avoid the load on our server
URL : {{base_url}}/video/upload_video_url/{video_name}/{video_extension}
Upload Video
Here we are Uploading the Video to the S3 Bucket using URL generated by API called Get Video Upload URL.
After Uploading the Video if you get 200 Response code then video will sent for the Processing ( for Transcoding video to Multiple Resolutions )
//Here you will send request to the link which you will get from this API called "Get Video Upload URL" as a Response
Get Videos
URL : {{base_url}}/video/get_videos
Using this you can get the videos which are uploaded by you and you can see the status of every video of every resolution transcoding status like Pending, Starting, Done