Der Kellner

About

This project creates a serverless application based on AWS lambda to process CSV files sent to a S3 bucket and send the processed data to SQS queue.

AWS lambda was chosen for this project in order to be simple as possible to implement and deploy.

The deployment is made wit Terraform by using just the AWS terraform provider, also chosen due to it's simplicity and extensive support.

Schemas

CSV schema

It's expected that the data is distributed among 3 types of CSV files. The files are named $TYPE_$DATE.csv, where TYPE can be customers, orders or items. Their schema can be found in the example-csv-files folder.

SQS Schema

After the processing, a message will be sent to SQS with the following schema:

{
"type": "customer_message",
"customer_reference": "ade3-11ed"
"number_of_orders": 2,
"total_amount_spent": 190.0
}

How to deploy

You need to have AWS credentials in order to deploy it to your AWS account.

export AWS_ACCESS_KEY_ID="anaccesskey"
export AWS_SECRET_ACCESS_KEY="asecretkey"
export AWS_REGION="us-west-2"

Access the terraform folder, fill the relevant variables in variables.tf file and run terraform:

cd terraform
terraform init
terraform apply

How to clean up

If you want to tear down all resources, run terraform destroy command from the terraform folder.