After https://gist.github.com/pat/7b61376981b40cfdbb1166734b8d184f
This Terraform configuration allows running of a set of SQL commands on a new AWS RDS database instance that's operating within an AWS VPC.
The commands are executed via AWS Lambda functions - the first (rds_creation) operates outside the VPC and connects to the AWS API to determine credential information for the new database (endpoint, port, username, database). It then sends these details via SNS to another function operating within the VPC (rds_setup), which connects to the Postgres database and executes the SQL commands.
The initial notification comes via SNS from the RDS events (and there is the configuration within the Terraform file here to set up that subscription).
Please note:
- There are variables with defaults defined - everything else should be pretty self-contained.automatic (though you should definitely read through all of the code to ensure you understand it before using it).
- The internal Lambda function is expecting a Postgres database. This will need changing if you're going to use MySQL/Aurora/etc.
- The internal Lambda function connects via SSL, and so requires a local copy of the AWS SSL Certificate file (which I have saved as
rds-cert.pemwithin therds_setupdirectory).
curl https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem -o ./rds_setup/rds-cert.pem
- Build the lambdas
Each lambda function is stored within their respective folders (rds_creation, rds_setup), with their package.json for NPM to build.
Building:
npm install
These folders are automatically archived/zipped by Terraform.
This template uses:
- Cloudwatch
- S3
- Lambda
- SNS
- IAM
- aws-sdk
- SPS
- node
Set tags to allow tf to find VPC, Subnets and Security groups. Expects a Subnet called RDS to exist.
| Name | Version |
|---|---|
| terraform | >= 0.13 |
| archive | 2.0.0 |
| aws | 3.24.1 |
| template | 2.2.0 |
| Name | Version |
|---|---|
| archive | 2.0.0 |
| aws | 3.24.1 |
No modules.
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| common_tags | To implement the common tags scheme | map(any) |
{} |
no |
| db_password_path | The path in the parameter store where your encrypted db password can be retrieved | string |
"/rds/postgres/database/password" |
no |
| region | AWS region | string |
n/a | yes |
| runtime | n/a | string |
"nodejs12.x" |
no |
| sg_tag | The tag to find your security group for your RDS access, you will most likely need to change/supply this value | set(string) |
[ |
no |
| sse_algorithm | (optional) describe your variable | string |
"aws:kms" |
no |
| subnet_tag | A Name tag to find your private subnets, you will most likely need to change/supply this value | string |
"*private*" |
no |
| table_name | The name of the database to create all objects in, probably want to/change supply this | string |
"data" |
no |
| vpc_tag | A Name tag to find your VPC, you will need to supply this value | string |
"*Default*" |
no |
No outputs.
This is the policy required to build this project:
The Terraform resource required is:
resource "aws_iam_policy" "terraform_pike" {
name_prefix = "terraform_pike"
path = "/"
description = "Pike Autogenerated policy from IAC"
policy = jsonencode({
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"SNS:CreateTopic",
"SNS:DeleteTopic",
"SNS:GetTopicAttributes",
"SNS:ListTagsForResource",
"SNS:SetTopicAttributes",
"SNS:TagResource",
"SNS:UnTagResource"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"ec2:DescribeAccountAttributes",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs"
],
"Resource": "*"
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"iam:AttachRolePolicy",
"iam:CreateRole",
"iam:DeleteRole",
"iam:DeleteRolePolicy",
"iam:DetachRolePolicy",
"iam:GetRole",
"iam:GetRolePolicy",
"iam:ListAttachedRolePolicies",
"iam:ListInstanceProfilesForRole",
"iam:ListRolePolicies",
"iam:PassRole",
"iam:PutRolePolicy",
"iam:TagRole"
],
"Resource": "*"
},
{
"Sid": "VisualEditor3",
"Effect": "Allow",
"Action": [
"lambda:AddPermission",
"lambda:CreateFunction",
"lambda:DeleteFunction",
"lambda:GetFunction",
"lambda:GetFunctionCodeSigningConfig",
"lambda:GetPolicy",
"lambda:ListVersionsByFunction",
"lambda:RemovePermission",
"lambda:TagResource",
"lambda:UntagResource"
],
"Resource": "*"
},
{
"Sid": "VisualEditor4",
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:DeleteBucket",
"s3:DeleteObject",
"s3:GetAccelerateConfiguration",
"s3:GetBucketAcl",
"s3:GetBucketCORS",
"s3:GetBucketLogging",
"s3:GetBucketObjectLockConfiguration",
"s3:GetBucketPolicy",
"s3:GetBucketPublicAccessBlock",
"s3:GetBucketRequestPayment",
"s3:GetBucketTagging",
"s3:GetBucketVersioning",
"s3:GetBucketWebsite",
"s3:GetEncryptionConfiguration",
"s3:GetLifecycleConfiguration",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:GetObjectTagging",
"s3:GetReplicationConfiguration",
"s3:ListBucket",
"s3:PutBucketPublicAccessBlock",
"s3:PutObject"
],
"Resource": "*"
}
]
})
}
Replace invoke.sql with your own database script.
Got a question?
File a GitHub issue.
Please use the issue tracker to report any bugs or file feature requests.
Copyright 2021-2022 James Woolfenden
See LICENSE for full details.
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
