elasticsearch-snapshot

This container can be used to easily manage Elasticsearch snapshots.

Environment Variables

ELASTICSEARCH_URL (default: http://elasticsearch-9200.service.consul:9200)
- The URL to reach your Elasticsearch server.
- NOTE: This must be the entire hostname or IP. Hostnames that expect a search domain to be added will not work.
ESS_ABORT_IF_EMPTY (default: true)
- This will abort the snapshot process if there are no indices on the server.
- This is useful if you just bootstrapped a cluster, and want to restore before doing any backups. The repository will still be created, but no snapshots will be created until there are indices on the server.
ESS_CREATE_IF_MISSING (defaut: false)
- If this is true and the snapshot repository does not exist, then attempt to create it. See the Repository Auto-Creation section for more information.
ESS_MAX_SNAPSHOTS (defaut: 0)
- The maximum number of snapshots to allow. Set to 0 to disable entirely.
- Note: Only snapshots with a scheduled- prefix will be counted and deleted. All other snapshots will not count against this limit.
ESS_REPO_NAME required
- The name of the repository to create snapshots under.
ESS_WAIT_FOR_COMPLETION (default: true)
- Whether the execution should wait until the snapshot has been created before exiting.

Repository Auto-Creation

If ESS_CREATE_IF_MISSING is set to true then the following are relevant:

ESS_REPO_TYPE required
- The repository type field. Common ones include fs and s3.
ESS_REPO_SETTINGS_<SETTING> optional
- Add an environment variable per setting key to build the repository creation payload.
- If there are no ESS_REPO_SETTINGS_* variables found, then the settings hash will be {}.

An example creation payload construction:

ESS_CREATE_IF_MISSING=true
ESS_REPO_TYPE=s3
ESS_REPO_SETTINGS_BUCKET=mybucket
ESS_REPO_SETTINGS_COMPRESS=true
ESS_REPO_SETTINGS_BASE_PATH=/path/to/put/snapshots

Would result in a payload of:

{
  "type":"s3",
  "settings": {
    "bucket": "mybucket",
    "compress": "true",
    "base_path": "/path/to/put/snapshots"
  }
}

For more details about repository creation, see the Elasticsearch documentation

Usage

This is best used with a timer and service combination in your cluster.

# elasticsearch-snapshot.timer
[Unit]
Description=Elasticsearch Scheduled Snapshot Timer

[Timer]
OnCalendar=hourly
Persistent=false

[Install]
WantedBy=multi-user.target

[X-Fleet]
Conflicts=%p.timer
MachineOf=%p.service

# elasticsearch-snapshot.service
[Unit]
Description=Elasticsearch Scheduled Snapshot
Requires=docker.service
After=docker.service

[Service]
TimeoutStartSec=0
Type=simple

ExecStartPre=-/usr/bin/docker pull webdestroya/elasticsearch-snapshot

ExecStart=/usr/bin/docker run --rm=true \
  -e ELASTICSEARCH_URL=http://elasticsearch-9200.service.consul:9200 \
  -e ESS_REPO_NAME=s3_repository \
  -e ESS_MAX_SNAPSHOTS=100 \
  -e ESS_WAIT_FOR_COMPLETION=true \
  webdestroya/elasticsearch-snapshot

[Install]
WantedBy=multi-user.target

[X-Fleet]
Conflicts=%p.service

Useful Plugins

This tool was built primarily for the cloud-aws Elasticsearch plugin.