This project demonstrates the use of Terraform to automate the deployment of an AWS infrastructure that notifies users of file uploads to an S3 bucket. The infrastructure includes an S3 bucket, an SNS topic for notifications, an SQS queue, and a Lambda function that processes S3 events. Dependencies for the Lambda function are managed using a Lambda layer.
To implement the project step by step using the AWS Management Console, you can follow this project design narrative in my blog website: https://adityadubey.cloud/s3-file-upload-notification-system.
The architecture consists of the following components:
- S3 Bucket: The designated storage location for uploaded files.
- Lambda Function: The heart of the system, automatically triggered by S3 events (file uploads), it processes the uploaded file and initiates notifications.
- Amazon SNS (Simple Notification Service): Used to send notifications to subscribers via various channels (email, SMS, etc.).
- Amazon CloudWatch: Provides logging and monitoring of the Lambda function's execution, ensuring observability and troubleshooting capabilities.
- Amazon SQS (Simple Queue Service): Used for decoupling and scaling the processing of uploaded files.
- AWS Account: An AWS account with appropriate permissions to create the required resources.
- Terraform: Terraform installed and configured on your development machine. You can download it from https://developer.hashicorp.com/terraform/install.
- AWS CLI (Optional): The AWS CLI can be helpful for interacting with AWS services directly if needed.
- Python 3.8+: The Lambda function is written in Python.
git clone https://github.com/adityawdubey/S3-File-Upload-Notification-using-Terraform.git
Ensure your AWS CLI is configured with the necessary permissions.
The easiest way to install Terraform on macOS is with Homebrew.
brew tap hashicorp/tap
brew install hashicorp/tap/
Initialize Terraform to download the necessary providers:
terraform init
Before applying the Terraform configuration, it's good practice to review the changes that will be made. Run the following command to generate an execution plan:
terraform plan
This will create a plan file named tfplan which you can review to understand what changes will be applied.
Apply the Terraform configuration to create the resources:
terraform apply tfplan
Any file uploaded to the specified S3 bucket will trigger the Lambda function. The Lambda function processes the file and sends a notification via SNS.
To delete all resources created by this project, run:
terraform destroy