monstar-lab-consulting/terraform-aws-base

Initially set base configuration created by Terraform.

AWS Base Terraform

OverView

When building infrastructure in AWS, there are always things to consider for any project. For example, security, IAM, cost, log storage and notification, etc... It is quite difficult to build a separate Terraform for each project for all the things that must be considered.
In this repository, we use Terraform to build the baseline configuration.

INDEX

• Base
  • Required
• Functions
  • Security
    • CloudTrail
    • Config
    • GuardDuty
    • Security Hub
  • Other
    • Budgets
    • Compute Optimizer
    • IAM group policy
    • IAM User and Group
    • Resource Groups
    • Trusted Advisor
• Settings
  • Initial setting
• Logs
  • S3 bucket list

Required

• Terraform
  https://www.terraform.io/
• Slack
  For notifications, you will need Slack, OAuthToken, and ChannelID.
  https://slack.com/
  https://slack.dev/node-slack-sdk/getting-started

CloudTrail

AWS CloudTrail is a service for governance, compliance, operational and risk auditing of AWS accounts.CloudTrail enables you to log, continuously monitor and retain account activity across your AWS infrastructure.

After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, Slack notifications will be sent.
You will be notified with a message similar to the following.

Config

AWS Config is a service that allows you to evaluate, audit, and review the configuration of AWS resources. Config continuously monitors and records the configuration of AWS resources and automatically evaluates the recorded configuration against the desired settings. Config allows you to review configuration and association changes between AWS resources, examine detailed resource configuration history, and verify overall compliance with settings specified in company guidelines. This simplifies compliance audits, security analysis, change management, and operational troubleshooting.

After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, Slack notifications will be sent.
You will be notified with a message similar to the following.

GuardDuty

Amazon GuardDuty is a threat detection service that continuously monitors for malicious or unauthorized activity in order to protect AWS accounts, workloads, and data stored in Amazon S3.

After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, Slack notifications will be sent.
You will be notified with a message similar to the following.

Security Hub

The AWS Security Hub provides a comprehensive view of security alerts and security status across all your AWS accounts. A wide range of sophisticated security tools are at your disposal, from firewall and endpoint protection to vulnerability and compliance scanners.

The three security standards provided in the Security Hub are addressed to the best of our ability.

• AWS Foundational Security Best Practices
• CIS AWS Foundations Benchmark
• PCI DSS v3.2.1

The following is the security score when only this Terraform is applied.
You should be aware that the score will not be accurate until you re-evaluate it after building.

Budgets

AWS Budgets provides the ability to set up custom budgets and be alerted when costs or usage exceed (or are expected to exceed) the budgeted amount or amounts.

After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, you will receive Slack notifications at the specified time (default is 18:00 JST daily). An email will also be sent if the specified cost limit is exceeded.

Compute Optimizer

AWS Compute Optimizer recommends optimal AWS resources for your workloads to reduce costs and improve performance by using machine learning to analyze historical utilization metrics. Over-provisioning resources can lead to unnecessary infrastructure cost, and under-provisioning resources can lead to poor application performance. Compute Optimizer helps you choose optimal configurations for three types of AWS resources: Amazon EC2 instances, Amazon EBS volumes, and AWS Lambda functions, based on your utilization data.

IAM group policy

You can set the policy to assign to IAM groups. You can also make the virtual MFA setting mandatory as a base policy.
You can also configure the IAM Switch Role.

IAM User and Group

You can create an IAM User and Group.

Resource Groups

Overall, all resources created in Terraform will have the same TAG, and Resource Groups will be filtered by that TAG.

Trusted Advisor

AWS Trusted Advisor is a fully managed service that provides guidance on how to follow AWS best practices. Improve security and performance, reduce costs, and monitor service limitations.

After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, you will be able to receive Slack notifications at the specified time (default is 9:00 JST daily).
However, Trusted Advisor requires the support plan to be signed up for the Business or Enterprise plan. The default setting is false.

Initial setting

• Remove the access key from the root account
  Since this is a security issue, let's remove the access key from the root account from the management console.

• Manual creation of IAM user and IAM group to run Terraform
  Create an IAM user and an IAM group from the management console in order to run Terraform. Create an IAM group (pseudonym: deploy). Attach AdministratorAccess as the policy. Create an IAM user (pseudonym: terraform), giving it only Programmatic access for Access Type, and add it to the IAM group (pseudonym: deploy).

• Create an S3 to store the Terraform State
  Create an S3 from the management console to manage the Terraform State. However, if you have an environment where you can run the aws command and profile already configured, you can create an S3 by running the following command.
  https://github.com/y-miyazaki/cloud-commands/blob/master/cmd/awstfinitstate

# awstfinitstate -h

This command creates a S3 Bucket for Terraform State.
You can also add random hash to bucket name suffix.

Usage:
    awstfinitstate -r {region} -b {bucket name} -p {profile}[<options>]
    awstfinitstate -r ap-northeast-1 -b terraform-state
    awstfinitstate -r ap-northeast-1 -b terraform-state -p default -s

Options:
    -b {bucket name}          S3 bucket name
    -p {aws profile name}     Name of AWS profile
    -r {region}               S3 region
    -s                        If set, a random hash will suffix bucket name.
    -h                        Usage awstfinitstate

# awstfinitstate -r ap-northeast-1 -b terraform-state -p default -s
~
~
~
~
~
~
~
~
~
~
~
~
~
--------------------------------------------------------------
bucket_name: terraform-state-xxxxxxxxxx
region: ap-northeast-1
--------------------------------------------------------------

• terraform.{environment}.tfvars file to configure for each environment
  You need to rename the linked file terraform.example.tfvars and change each variable for your environment. The variables that need to be changed are marked with TODO comments; search for them in TODO.

• main_provider.tf file to set for each environment
  Rename the linked file main_provider.tf.example to main_provider.tf. After that, you need to change each parameter. The variables that need to be changed are marked with TODO comments, search for them in TODO.

#--------------------------------------------------------------
# Terraform Provider
#--------------------------------------------------------------
terraform {
  required_version = ">=0.13"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">=3.29.1"
    }
  }
  backend "s3" {
    # TODO: need to change bucket for terraform state.
    bucket = "xxxxxxxxxxxxxxxx"
    # TODO: need to change bucket key for terraform state.
    key = "xxxxxxxxxx"
    # TODO: need to change profile for terraform state.
    profile = "default"
    # TODO: need to change region for terraform state.
    region = "ap-northeast-1"
  }
}

#--------------------------------------------------------------
# AWS Provider
# access key and secret key should not use.
#--------------------------------------------------------------
provider "aws" {
  # TODO: need to change profile.
  profile = "default"
  # TODO: need to change region.
  region = "ap-northeast-1"
}

• Running Terraform
  Run the terraform command: terraform init followed by terraform apply. You may find that terraform apply fails due to conflicts or other problems, so run it again and it will succeed.

bash-5.1# terraform init
There are some problems with the CLI configuration:

Error: The specified plugin cache dir /root/.terraform.d/plugin-cache cannot be opened: stat /root/.terraform.d/plugin-cache: no such file or directory


As a result of the above problems, Terraform may not behave as intended.


Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/template from the dependency lock file
- Installing hashicorp/aws v3.29.1...
- Installed hashicorp/aws v3.29.1 (signed by HashiCorp)
- Installing hashicorp/random v3.1.0...
- Installed hashicorp/random v3.1.0 (signed by HashiCorp)
- Installing hashicorp/template v2.2.0...
- Installed hashicorp/template v2.2.0 (signed by HashiCorp)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

bash-5.1# terraform apply --auto-approve -var-file=terraform.example.tfvars
module.aws_recipes_s3_bucket_log_log.random_id.this: Creating...
random_id.this: Creating...
module.aws_recipes_s3_bucket_log_logdom_id.this: Creation complete after 0s [id=wiatHg]
random_id.this: Creation complete after 0s [id=uqe0bU7J]
module.aws_recipes_security_default_vpc.aws_default_subnet.this[1]: Creating...

...
...
...

Apply complete! resources: x added, x changed, 0 destroyed.

S3 bucket list

This is a description of the S3 bucket that will be created and the data in the bucket.

Category	bucket	Directory	Description	Note
AWS Config	aws-log-common	/AWSLogs/{accountID}/Config/{region}/yyyy/m/d/ConfigHistory/	AWS Config Compliance History Timeline for Resources.	https://docs.aws.amazon.com/config/latest/developerguide/view-compliance-history.html
AWS Config	aws-log-common	/AWSLogs/{accountID}/Config/{region}/yyyy/m/d/ConfigSnapshot/	AWS Config snapshot.	https://docs.aws.amazon.com/config/latest/developerguide/deliver-snapshot-cli.html
AWS Config	aws-log-common	/AWSLogs/{accountID}/Config/ConfigWritabilityCheckFile/yyyy/m/d	This is a test file to confirm that Config can be written to the S3 bucket normally.
AWS Config	aws-log-common	/AWSLogs/{accountID}/CloudTrail/AccessLog	This is the access log of the CloudTrail bucket.
AWS CloudTrail	aws-log-cloudtrail	/AWSLogs/{accountID}/CloudTrail-Digest/{region}/yyyy/mm/dd	Each digest file contains the names of the log files that were delivered to your Amazon S3 bucket during the last hour, the hash values for those log files, and the digital signature of the previous digest file. The signature for the current digest file is stored in the metadata properties of the digest file object. The digital signatures and hashes are used for validating the integrity of the log files and of the digest file itself.	https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-validation-digest-file-structure.html
AWS CloudTrail	aws-log-cloudtrail	/AWSLogs/{accountID}/CloudTrail-Insight/{region}/yyyy/mm/dd	CloudTrail Insights can help you detect unusual API activity in your AWS account by raising Insights events. CloudTrail Insights measures your normal patterns of API call volume, also called the baseline, and generates Insights events when the volume is outside normal patterns. Insights events are generated for write management APIs.	https://docs.aws.amazon.com/awscloudtrail/latest/userguide/log-insights-events-with-cloudtrail.html
AWS CloudTrail	aws-log-cloudtrail	/AWSLogs/{accountID}/CloudTrail/{region}/yyyy/mm/dd	It is recorded as an event in CloudTrail. Events include actions taken in the AWS Management Console, AWS Command Line Interface.	https://docs.aws.amazon.com/awscloudtrail/latest/userguide/get-and-view-cloudtrail-log-files.html
AWS Log	aws-log-application	/Logs	Application log from CloudWatch Logs.

Author Information

Author: Yoshiaki Miyazaki
Contact: https://github.com/y-miyazaki