/terraform-aws-lambda-es-cleaner

Terraform module and Lambda for auto-deleting old indices in Elasticsearch

Primary LanguageHCLMIT LicenseMIT

AWS Elasticsearch Service index cleaner Terraform module

Terraform module for automatically deleting Elasticsearch index exceeding maximum retention period.

terraform v0.12.x Language grade: Python

Prerequisites

Index name in Elasticsearch Service must include YYYYMMDD timeformat key.

Usage

module "es_cleaner" {
  source  = "baikonur-oss/lambda-es-cleaner/aws"

  lambda_package_url = "https://github.com/baikonur-oss/terraform-aws-lambda-es-cleaner/releases/download/v1.0.0/lambda_package.zip"
  name               = "es_cleaner"

  elasticsearch_host  = "search-dev-elasticsearch-xxxxxxxx.ap-northeast-1.es.amazonaws.com"
  elasticsearch_arn   = "arn:aws:es:ap-northeast-1:0123456789:domain/elasticsearch"
  schedule_expression = "cron(0 1 * * ? *)"
}

Warning: use same module and package versions!

Version pinning

Terraform Module Registry

Use version parameter to pin to a specific version, or to specify a version constraint when pulling from Terraform Module Registry (source = baikonur-oss/lambda-es-cleaner/aws). For more information, refer to Module Versions section of Terraform Modules documentation.

GitHub URI

Make sure to use ?ref= version pinning in module source URI when pulling from GitHub. Pulling from GitHub is especially useful for development, as you can pin to a specific branch, tag or commit hash. Example: source = github.com/baikonur-oss/terraform-aws-lambda-es-cleaner?ref=v1.0.0

For more information on module version pinning, see Selecting a Revision section of Terraform Modules documentation.

Inputs

Name Description Type Default Required
dry_run_only Dry run option for testing purpose string "false" no
elasticsearch_arn Elasticsearch Service ARN string n/a yes
elasticsearch_host Elasticsearch Service endpoint (without https://) string n/a yes
handler Lambda Function handler (entrypoint) string "main.handler" no
lambda_package_url Lambda package URL (see Usage in README) string n/a yes
log_retention_in_days Lambda Function log retention in days string "30" no
max_age_days retention period of Elasticsearch Service index (days). Older indexes will be removed. string "60" no
memory Lambda Function memory in megabytes string "256" no
name Resource name string n/a yes
runtime Lambda Function runtime string "python3.7" no
schedule_expression Lambda Schedule Expressions for Rules (https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/events/ScheduledEvents.html) string n/a yes
tags Tags for Lambda Function map(string) {} no
timeout Lambda Function timeout in seconds string "60" no
timezone tz database timezone name (e.g. Asia/Tokyo) string "UTC" no
tracing_mode X-Ray tracing mode (see: https://docs.aws.amazon.com/lambda/latest/dg/API_TracingConfig.html ) string "PassThrough" no

Contributing

Make sure to have following tools installed:

macOS

brew install pre-commit terraform terraform-docs

# set up pre-commit hooks by running below command in repository root
pre-commit install