AWS Storage Gateway Terraform module

This repository contains Terraform code which creates resources required to run Storage Gateway (https://aws.amazon.com/storagegateway/) in AWS and on premises.

AWS Storage Gateway is available in 4 types :

Amazon S3 File Gateway (FILE_S3)
Amazon FSx File Gateway (FILE_FSX_SMB)
Tape Gateway (VTL)
Volume Gateway (CACHED, STORED)

The module requires a Gateway type to be declared. The default is configured to FILE_S3 as an example. For more details regarding the Storage Gateway types and their respective arguments can be found here.

Usage with VMware S3 File Gateway module

Link to the S3 SMB Storage Gateway example for VMware: s3filegateway-vmware

Prerequisists

The VMware module requires the vsphere provider to be setup with a service account user name and password that has the necessary permissions in Vcenter to create a VM. This is found in the settings.tf file.

provider "vsphere" {
  allow_unverified_ssl = var.allow_unverified_ssl
  vsphere_server       = var.vsphere_server
  user                 = var.vsphere_user
  password             = var.vsphere_password
}

Note that var.allow_unverified_ssl is a boolean that can be set to true to disable SSL certificate verification. This should be used with care as it could allow an attacker to intercept your authentication token. The default value is set to false but can be changed to true for testing purposes only.

The module also requires connectivity to your vCenter server. Therefore it needs to be deployed from a virtual machine that can reach the vCenter APIs. You may also Terraform Cloud Agents if you use already use Terrform Cloud. This allows the modules to be deployed remotely.

vSphere Module

module "vsphere" {
  source     = "aws-ia/storagegateway/aws//modules/vmware-sgw"
  datastore  = var.datastore
  datacenter = var.datacenter
  network    = var.network
  cluster    = var.cluster
  host       = var.host
  name       = "my-s3fgw"
}

The virtual machine IP address needs to be passed to next module as the gateway IP address. In addition, the module also requires domain user name and passwords for the storage gateway to join the domain.

Note that in order to protect sensitive data such as domain credentials etc., certain variables are marked as sensitive. It is general best practice to never store credentials and secrets in git repositories. For more information about protecting sensitive variables refer to this documentation. Also as a best practice consider the use of services such as AWS Secrets Manager, Hashicorp Vault or Terraform Cloud to dynamically inject your secrets.

Also note that the domain password despite being a sensitive variable can be still found in the Terraform state file. Follow this guidance to protect state file from unauthorized access.

Storage Gateway Module

module "sgw" {
  source             = "aws-ia/storagegateway/aws//modules/aws-sgw"
  name               = "my-sgw"
  gateway_ip_address = module.vsphere.vm_ip
  join_smb_domain    = true
  domain_name        = var.domain_name
  domain_username    = var.domain_username
  domain_password    = var.domain_password
  domain_controllers = var.domain_controllers
  gateway_type       = "FILE_S3"       
}

Note that variable "join_smb_domain" is set to true by default and therefore optional. To create a Storage Gateway that is not joined to the domain set "join_smb_domain" to false.

Example :

module "sgw" {
  source             = "aws-ia/storagegateway/aws//modules/aws-sgw"
  name               = "my-sgw"
  gateway_ip_address = module.vsphere.vm_ip
  join_smb_domain    = false
  gateway_type       = "FILE_S3"       
}

Refer to to the S3 NFS Storage Gateway example for VMware for an end to end example: s3-nfs-filegateway-vmware

Setting up S3 buckets for S3 File Gateway

module "s3_bucket" {
  source                   = "terraform-aws-modules/s3-bucket/aws"
  version                  = ">=3.5.0"
  bucket                   = "bucket-name"
  control_object_ownership = true
  object_ownership         = "BucketOwnerEnforced"
  block_public_acls        = true
  block_public_policy      = true
  ignore_public_acls       = true
  restrict_public_buckets  = true

  server_side_encryption_configuration = {
    rule = {
      apply_server_side_encryption_by_default = {
        kms_master_key_id = "kms_key_id"
        sse_algorithm     = "aws:kms"
      }
    }
  }
  logging = {
    target_bucket = "log-delivery-bucket"
    target_prefix = "log/"
  }

  versioning = {
    enabled = false
  }
}

Note that versioning is set to false by default for the S3 bucket for the file share for Storage Gateway. Enabling S3 Versioning can increase storage costs within Amazon S3. Please see here for further information on whether S3 Versioning is right for your workload.

Setting up SMB File shares

module "smb_share" {
  source        = "aws-ia/storagegateway/aws//modules/s3-smb-share"
  share_name    = "smb_share_name"
  gateway_arn   = module.sgw.storage_gateway.arn
  bucket_arn    = module.s3_bucket.s3_bucket_arn
  role_arn      = "iam-role-for-sgw-s3"
  log_group_arn = "log-group-arn"
}

Setting up NFS File shares

module "nfs_share" {
  source        = "aws-ia/storagegateway/aws//modules/s3-nfs-share"
  share_name    = "nfs_share_name"
  gateway_arn   = module.sgw.storage_gateway.arn
  bucket_arn    = module.s3_bucket.s3_bucket_arn
  role_arn      = "iam-role-for-sgw-s3"
  log_group_arn = "log-group-arn"
  client_list   = ["10.0.0.0/24","10.0.1.0/24"]
}

The examples also includes "aws_kms_key" resource block to create a KMS key. For production deployments, you should pass in a key policy that restricts the use of the key based on your access requirements. Refer to this link for information.

Support & Feedback

Storage Gateway module for Terraform is maintained by AWS Solution Architects. It is not part of an AWS service and support is provided best-effort by the AWS Storage community.

To post feedback, submit feature ideas, or report bugs, please use the Issues section of this GitHub repo.

If you are interested in contributing to the Storage Gateway module, see the Contribution guide.

Requirements

Name	Version
terraform	>= 1.0.7
aws	>= 4.0.0, < 5.0.0
awscc	>= 0.24.0

Providers

Name	Version
aws	>= 4.0.0, < 5.0.0

Modules

No modules.

Resources

Name	Type
aws_storagegateway_cache.sgw	resource
aws_storagegateway_gateway.mysgw	resource
aws_storagegateway_local_disk.sgw	data source

Inputs

Name	Description	Type	Default	Required
gateway_ip_address	IP Address of the SGW appliance in vSphere	`string`	n/a	yes
gateway_name	Storage Gateway Name	`string`	n/a	yes
disk_path	Path on the SGW appliance in vsphere where the cache disk resides on the OS	`string`	`"/dev/sdb"`	no
domain_controllers	List of IPv4 addresses, NetBIOS names, or host names of your domain server. If you need to specify the port number include it after the colon (“:”). For example, mydc.mydomain.com:389.	`list(any)`	`[]`	no
domain_name	The name of the domain that you want the gateway to join	`string`	`""`	no
domain_password	The password for the service account on your self-managed AD domain that SGW will use to join to your AD domain	`string`	`""`	no
domain_username	The user name for the service account on your self-managed AD domain that SGW use to join to your AD domain	`string`	`""`	no
gateway_type	Type of the gateway. Valid options are FILE_S3, FILE_FSX_SMB, VTL, CACHED, STORED	`string`	`"FILE_S3"`	no
join_smb_domain	Setting for controlling whether to join the Storage gateway to an Active Directory (AD) domain for Server Message Block (SMB) file shares. Variables domain_controllers, domain_name, password and username should also be specified to join AD domain.	`bool`	`true`	no
organizational_unit	The organizational unit (OU) is a container in an Active Directory that can hold users, groups, computers, and other OUs and this parameter specifies the OU that the gateway will join within the AD domain.	`string`	`""`	no
timeout_in_seconds	Specifies the time in seconds, in which the JoinDomain operation must complete. The default is 20 seconds.	`number`	`-1`	no
timezone	Time zone for the gateway. The time zone is of the format GMT, GMT-hr:mm, or GMT+hr:mm.For example, GMT-4:00 indicates the time is 4 hours behind GMT. Avoid prefixing with 0	`string`	`"GMT"`	no

Outputs

Name	Description
storage_gateway	Storage Gateway Name

kawsark/terraform-aws-storagegateway