/terraform-aws-arc-opensearch

Repo for managing the OpenSearch Terraform Module.

Primary LanguageHCLApache License 2.0Apache-2.0

Module Structure

terraform-aws-arc-opensearch

Latest Release Last Updated Terraform GitHub Actions

Quality gate

Known Vulnerabilities

Overview

Terraform module for Amazon OpenSearch provides a flexible and scalable way to deploy and manage OpenSearch clusters, with support for both serverless and managed (provisioned) deployment options.

Usage

See the example/vpc folder for a working module example.

################################################################################
## opensearch
################################################################################
module "opensearch" {
  source                       = "sourcefuse/arc-opensearch/aws"
  version                      = "0.1.2"
  namespace                    = var.namespace
  environment                  = var.environment
  name                         = var.name
  engine_version               = var.engine_version
  instance_type                = var.instance_type
  instance_count               = var.instance_count
  enable_vpc_options           = true
  vpc_id                       = data.aws_vpc.default.id
  subnet_ids                   = local.private_subnet_ids
  ingress_rules                = local.ingress_rules
  egress_rules                 = local.egress_rules

  tags = module.tags.tags
}

################################################################################
## opensearch serverless
################################################################################
 module "opensearch_serverless" {
  source                       = "sourcefuse/arc-opensearch/aws"
  version                      = "0.1.2"
  enable_serverless            = true
  namespace                    = var.namespace
  environment                  = var.environment
  name                         = var.name
  ingress_rules                = local.ingress_rules
  egress_rules                 = local.egress_rules
  subnet_ids                   = local.private_subnet_ids
  vpc_id                       = data.aws_vpc.default.id
  data_lifecycle_policy_rules  = local.data_lifecycle_policy_rules
  access_policy_rules          = local.access_policy_rules
  tags                         = module.tags.tags

}

See the example/public folder if you want your os to be public

################################################################################
## opensearch
################################################################################
module "opensearch" {
  source                         = "sourcefuse/arc-opensearch/aws"
  version                        = "1.0.3"
  namespace                      = var.namespace
  environment                    = var.environment
  name                           = var.name
  engine_version                 = var.engine_version
  instance_type                  = var.instance_type
  instance_count                 = var.instance_count
  enable_encrypt_at_rest         = true
  enable_domain_endpoint_options = true
  advanced_security_enabled      = true

  tags = module.tags.tags
}

################################################################################
## opensearch serverless
################################################################################
module "opensearch_serverless" {
  source                       = "sourcefuse/arc-opensearch/aws"
  version                      = "1.0.3"
  enable_serverless            = true
  namespace                    = var.namespace
  environment                  = var.environment
  name                         = var.name
  enable_public_access         = true
  data_lifecycle_policy_rules  = local.data_lifecycle_policy_rules
  access_policy_rules          = local.access_policy_rules

  tags = module.tags.tags
}

Requirements

Name Version
terraform >= 1.5.0
aws ~> 5.0

Providers

Name Version
aws 5.74.0

Modules

Name Source Version
opensearch ./modules/opensearch-domain n/a
opensearch_serverless ./modules/opensearch-serverless n/a

Resources

Name Type
aws_caller_identity.current data source

Inputs

Name Description Type Default Required
access_policies Custom access policy for OpenSearch domain. If empty, default policy will be used string "" no
access_policy_rules List of rules for the access policy. 
list(object({
    resource_type = string
    resource      = list(string)
    permissions   = list(string)
    principal     = optional(list(string), [])
  }))
 [] no
advanced_security_enabled Enable advanced security options (fine-grained access control) bool false no
anonymous_auth_enabled Enable anonymous authentication bool false no
auto_software_update_enabled Enable automatic software updates for OpenSearch bool false no
auto_tune_cron_expression Cron expression for Auto-Tune maintenance schedule string "0 1 * * ?" no
auto_tune_desired_state Desired state of Auto-Tune string "ENABLED" no
auto_tune_duration_unit Duration unit for Auto-Tune maintenance string "HOURS" no
auto_tune_duration_value Duration value for Auto-Tune maintenance number 1 no
auto_tune_start_at Start time for Auto-Tune maintenance string "2024-10-23T01:00:00Z" no
availability_zone_count The number of availability zones to use for zone awareness. number 2 no
cognito_identity_pool_id Cognito Identity Pool ID string "" no
cognito_user_pool_id Cognito User Pool ID string "" no
create_access_policy Flag to determine if access policy should be created. bool true no
create_data_lifecycle_policy Flag to determine if data lifecycle policy should be created. bool true no
create_encryption_policy Flag to determine if encryption policy should be created. bool true no
custom_certificate_arn ARN of the ACM certificate for the custom endpoint string "" no
custom_hostname Custom domain name for the OpenSearch endpoint string "" no
data_lifecycle_policy_rules Data lifecycle policy rules for the indices. 
list(object({
    indexes   = list(string)
    retention = string
  }))
 
[
  {
    "indexes": [
      "*"
    ],
    "retention": "Unlimited"
  }
]
 no
dedicated_master_count Number of dedicated master instances number 3 no
dedicated_master_enabled Whether dedicated master is enabled bool false no
dedicated_master_type Instance type for the dedicated master node string "m5.large.search" no
description A description for the OpenSearch collection. string "OpenSearch collection domain for logs and search" no
ebs_enabled Whether EBS is enabled for the domain bool true no
egress_rules A list of egress rules for the security group. 
list(object({
    from_port   = number
    to_port     = number
    protocol    = string
    cidr_blocks = list(string)
  }))
 [] no
enable_auto_tune Enable Auto-Tune for the domain bool false no
enable_cognito_options Enable Cognito authentication for the OpenSearch domain bool false no
enable_custom_endpoint Enable custom domain endpoint bool false no
enable_domain_endpoint_options Enable custom domain endpoint options for the OpenSearch domain. bool false no
enable_encrypt_at_rest Enable encryption at rest for the OpenSearch domain. bool false no
enable_off_peak_window_options Enable off-peak window options for the domain bool false no
enable_public_access Enable public access for the OpenSearch collection. If false, private access will be used. bool false no
enable_serverless Enable OpenSearch Serverless. If true, creates the serverless module; if false, creates the standard module. bool false no
enable_snapshot_options Enable snapshot options for the domain bool false no
enable_vpc_options Enable VPC options for the OpenSearch domain. bool false no
enable_zone_awareness Enable zone awareness for the OpenSearch domain. bool false no
encrypt_at_rest_enabled Enable encryption at rest bool true no
enforce_https Force HTTPS on the OpenSearch endpoint bool true no
engine_version OpenSearch or Elasticsearch engine version string "OpenSearch_1.0" no
environment Name of the environment, i.e. dev, stage, prod string n/a yes
ingress_rules A list of ingress rules for the security group. 
list(object({
    from_port   = number
    to_port     = number
    protocol    = string
    cidr_blocks = list(string)
  }))
 [] no
instance_count Number of instances in the cluster number 2 no
instance_type Instance type for the OpenSearch domain string "m5.large.search" no
internal_user_database_enabled Enable internal user database for fine-grained access control bool true no
iops Provisioned IOPS for the volume number null no
kms_key_id KMS key ID for encryption at rest string "" no
log_publishing_enabled Whether to enable the log publishing option. bool true no
log_types List of log types to publish to CloudWatch (Valid values: INDEX_SLOW_LOGS, SEARCH_SLOW_LOGS, ES_APPLICATION_LOGS, AUDIT_LOGS) list(string) 
[
  "INDEX_SLOW_LOGS",
  "SEARCH_SLOW_LOGS"
]
 no
master_user_arn The ARN of the IAM role for fine-grained access control. Required if use_iam_arn_as_master_user is true. string "" no
master_user_name Master user name for OpenSearch string "admin" no
name Name of the OpenSearch domain string n/a yes
namespace Namespace of the project, i.e. arc string n/a yes
node_to_node_encryption_enabled Enable node-to-node encryption bool true no
off_peak_hours Off-peak window start time (hours) number 0 no
off_peak_minutes Off-peak window start time (minutes) number 0 no
retention_in_days The number of days to retain log events in the log group number 7 no
saml_options Configuration block for SAML options in the OpenSearch domain. 
object({
    enabled                 = bool
    idp_entity_id           = optional(string)
    idp_metadata_content    = optional(string)
    roles_key               = optional(string)
    session_timeout_minutes = optional(number)
    subject_key             = optional(string)
  })
 
{
  "enabled": false,
  "idp_entity_id": null,
  "idp_metadata_content": null,
  "roles_key": null,
  "session_timeout_minutes": null,
  "subject_key": null
}
 no
security_group_name Name for the security group string "" no
snapshot_start_hour Start hour for the automated snapshot number 0 no
subnet_ids List of subnet IDs for the OpenSearch domain list(string) [] no
tags Tags to apply to resources map(string) n/a yes
throughput Provisioned throughput for the volume number null no
tls_security_policy TLS security policy for HTTPS endpoints string "Policy-Min-TLS-1-2-PFS-2023-10" no
type The type of OpenSearch collection. string "TIMESERIES" no
use_iam_arn_as_master_user Set to true to use IAM ARN as the master user, false to create a master user. bool false no
use_standby_replicas Flag to enable or disable standby replicas. bool true no
use_ultrawarm Whether to enable UltraWarm nodes bool false no
volume_size EBS volume size in GB number 20 no
volume_type EBS volume type string "gp2" no
vpc_id ID of the VPC for OpenSearch domain string null no
warm_count Number of UltraWarm instances number 2 no
warm_type UltraWarm node instance type string "ultrawarm1.medium.search" no
zone_awareness_enabled Whether zone awareness is enabled bool true no

Outputs

Name Description
opensearch_collection_endpoint The Endpoint of the OpenSearch collection
opensearch_domain_arn The ARN of the OpenSearch domain.
opensearch_domain_endpoint The endpoint of the OpenSearch domain.
opensearch_domain_id The unique identifier for the OpenSearch domain.
opensearch_serverless_collection_arn The ARN of the OpenSearch Serverless collection
opensearch_serverless_collection_id The ID of the OpenSearch Serverless collection

Versioning

This project uses a .version file at the root of the repo which the pipeline reads from and does a git tag.

When you intend to commit to main, you will need to increment this version. Once the project is merged, the pipeline will kick off and tag the latest git commit.

Development

Prerequisites

Configurations

  • Configure pre-commit hooks 
    pre-commit install

Tests

  • Tests are available in test directory
  • Configure the dependencies 
    cd test/
go mod init github.com/sourcefuse/terraform-aws-refarch-opensearch
go get github.com/gruntwork-io/terratest/modules/terraform
  • Now execute the test 
    go test -timeout  30m

Authors

This project is authored by:

  • SourceFuse ARC Team