Agencies SFTP Server
We're helping a global financial institution with operations spanning six AWS regions as our customer.
Our customer works with 21 marketing agencies and companies all around the world and now as part of data lake project, we want to create a solution to those agencies to upload data to our S3 buckets. Being not very tech savvy, they ask for SFTP connection for daily or weekly uploads.
Setup
Follow this instruction, To deploy agency required resources.
Prerequisites
You will need terraform and aws-cli installed and configured.
Clone repository
git clone https://github.com/m-zare/tmp-tf-repository.git
Customization
Open agencies_sftp\sample.tfvars
and modify it to address your needs.
In the following sample user1
and user2
are users to be created. Please note that you can set public SSH key or path to the SSH public key file for each user. username
variable could contain as many users as agency needs.
You can set region
to the desired one.
senderName
and senderEmail
are holding notification sender name and email. recipientEmail
is the one who should be notified. And the scheduleExpression
is a cron pattern for sending out emails.
region = "eu-west-1"
production = false
username = {
user1 = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpyq9KM6h+bGuZNHktEmkwLL2BZ5xPifAZJ7k2GjS9ukpulh1AKuWUEIF74Lx1I1jQVKoGh9HkzFLmM7LEDl2viZ65hvKxIOw52X8GCJSLSJYujCyhXRA57cFJ1BDsDYf8/HiaXZ4tB11KpN3mB9a/xV9aYOpVRwUMuYFx42gYBjmIQkqM7vNOXwf6UxqLenFJHTsQkNhoI1kUsMQ9o357FCgar+05KtBNXDaSnAPazFrJWstskJS4FX+9aT1OPXmzcuCFDkPxwk/pG9axKDbUIScdC58MChd527HJZEWX0U7E6YJCqYxe/gXPpWeCrwiVNw4kJdVNDso8pdaEMckt"
user2 = "~/sftp.pub"
}
senderName = "AWS SFTP Notification System"
senderEmail = "mostafazare@gmail.com"
recipientEmail = "mostafazare@gmail.com"
scheduleExpression = "cron(0 8 * * ? *)"
Deploy AWS s3 backend
Initialize modules:
cd tmp-tf-repository/aws_infra
terraform init
Apply configuration:
terraform apply
Deploy SFTP server and users
Initialize modules:
cd ../agencies_sftp/
terraform init
Apply configuration:
terraform apply -var-file sample.tfvars
Add/ remove/ update user
To add, remove or update user, edit agencies_sftp\sample.tfvars
as described in the above section. Then run terraform apply
.
cd tmp-tf-repository/agencies_sftp/
terraform apply -var-file sample.tfvars
Destroy
To destroy resources run terraform destroy
in both agancies_sftp
and aws_infra
directories and answer yes
to the questions.
cd tmp-tf-repository/agencies_sftp/
terraform destroy
cd ../aws_infra
terraform destroy
Note
For security reason you are not able to destroy a non-empty s3 bucket in AWS.
In the agancies_sftp
you can set production
to false
to ignore this constraint.
To be on safe side, destroying s3 bucket is not allowed in aws_infra
(even with production=false
). To destroy it you have to manually Empty
the sftp-terraform-state-bucket
bucket, then running terraform destroy
.