/AWS_S3_Gin_Mockup

Complete test Driven Development of authed S3 API using Go Gin , Postgres , Docker , Redis workers and Cronjobs with suitable unit testing automated workflows

Primary LanguageGo

Complete Test Driven Development of Backend using Go Lang and AWS S3

Features and Stacks

- Used Gin framework for Backend

  • Complete test driven development using testify

  • Migrations based DB setup using golang migrate

  • Redis with background processors for mail handling

  • PostgreSQL and Redis Dockerized at Alpine lightweight image

  • DB Mock being used to isolate dependencies [Mockgen]

  • SQLC generation

  • AWS

  • Cron Job for asset scheduled deletion [Hourly]

  • Graceful shutdown to prevent crashes

    image

and many more...

Feel free to checkout the code.

Technical Implementation

Pointer wise explanation :- Requirements:

  1. User Authentication & Authorization:

  • Implemented an advanced safe and secured Paseto token based auth with tests in token package . Helps offer clear distinction between private and public tokens

    (https://permify.co/post/jwt-paseto/)

  1. File Upload & Management:
  • Used AWS SDK for Go to upload objects and retrieve location and metadatas
  • Localised aws config for safe access key auth
  1. File Retrieval & Sharing:
  • Location is stored in postgresql for further retrieval
  1. File Search:
  • Has search function based on Coalace and sqlc args for a flexible search of file based on need
  1. Database Interaction:
  • Used Postgresql and sqlc with mocks for file details handling and mapping
  • Tests to keep DB operations in sync image
  1. Background Job for File Deletion:
  • Utilized cron job to run deleter function @hourley
  1. Testing: Complete app is test driven which is pretty evident
  • Makefile has every commands u need including test to facilitate all unit testing on multiple layers image

image

  1. Redis:
  • Redis setup with Task processor and Distributor using Asynq for background workers

Running and Installation Instructions in Local

Clone the repo to your local

Start your Docker Engine

Install Docker dependency Images for Postgres and Redis Alpine Images [ If not installed already ]

make docker
make redis

Configure AWS IAM config details to access AWS Services .

  • Install aws cli
  • Create a S3 Bucket in AWS Console
  • Create a IAM User role and attach policy to access S3 Buckets with the IAM role user
  • Download and record the Access key ID and Secret Access key
  • Now enter in the prompt when asked after
aws config

Launch the images

make launch

Start the server

make server