ELK Stack — Backup & Restore

3 minute read

Backup & Restore in Elasticsearch

Elasticsearch snapshots are the only reliable way to back up a cluster. Snapshots capture the state of indices, data streams, and cluster metadata at a point in time and store them in an external repository. This section covers repository types, creating and restoring snapshots, and automating the entire process with Snapshot Lifecycle Management (SLM).

Snapshot Repository Types

Type Plugin Required Use Case
Shared file system (fs) None NFS mount shared across all nodes
AWS S3 repository-s3 Cloud-native backup on AWS
Google Cloud Storage repository-gcs Cloud-native backup on GCP
Azure Blob Storage repository-azure Cloud-native backup on Azure
HDFS repository-hdfs Hadoop environments

Tip: Always prefer object-store repositories (S3, GCS, Azure) over shared filesystems in production. They scale better and eliminate the NFS single point of failure.

Registering a Snapshot Repository

Shared File System

Every node must have the path registered in elasticsearch.yml:

# elasticsearch.yml
path.repo: ["/mnt/es-snapshots"]

Register the repository:

curl -X PUT "localhost:9200/_snapshot/my_fs_repo" -H 'Content-Type: application/json' -d'
{
  "type": "fs",
  "settings": {
    "location": "/mnt/es-snapshots",
    "compress": true,
    "max_snapshot_bytes_per_sec": "200mb",
    "max_restore_bytes_per_sec": "200mb"
  }
}'

AWS S3

Install the plugin on every node:

bin/elasticsearch-plugin install repository-s3

Add credentials to the Elasticsearch keystore:

bin/elasticsearch-keystore add s3.client.default.access_key
bin/elasticsearch-keystore add s3.client.default.secret_key

Register the repository:

curl -X PUT "localhost:9200/_snapshot/my_s3_repo" -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "my-es-snapshots",
    "region": "us-east-1",
    "base_path": "production",
    "compress": true,
    "server_side_encryption": true,
    "storage_class": "standard_ia"
  }
}'

Google Cloud Storage

bin/elasticsearch-plugin install repository-gcs
bin/elasticsearch-keystore add-file gcs.client.default.credentials_file /path/to/service-account.json

curl -X PUT "localhost:9200/_snapshot/my_gcs_repo" -H 'Content-Type: application/json' -d'
{
  "type": "gcs",
  "settings": {
    "bucket": "my-es-snapshots",
    "base_path": "production",
    "compress": true
  }
}'

Azure Blob Storage

bin/elasticsearch-plugin install repository-azure
bin/elasticsearch-keystore add azure.client.default.account
bin/elasticsearch-keystore add azure.client.default.key

curl -X PUT "localhost:9200/_snapshot/my_azure_repo" -H 'Content-Type: application/json' -d'
{
  "type": "azure",
  "settings": {
    "container": "es-snapshots",
    "base_path": "production",
    "compress": true
  }
}'

Creating Snapshots

Full Cluster Snapshot

curl -X PUT "localhost:9200/_snapshot/my_s3_repo/snapshot-$(date +%Y%m%d-%H%M%S)?wait_for_completion=false"

Snapshot Specific Indices

curl -X PUT "localhost:9200/_snapshot/my_s3_repo/logs-snapshot-2026-04" -H 'Content-Type: application/json' -d'
{
  "indices": "logs-2026.04.*",
  "ignore_unavailable": true,
  "include_global_state": false
}'

Snapshot Data Streams

curl -X PUT "localhost:9200/_snapshot/my_s3_repo/ds-snapshot" -H 'Content-Type: application/json' -d'
{
  "indices": "logs-*",
  "include_global_state": false,
  "feature_states": ["none"]
}'

Snapshot Lifecycle Management (SLM)

SLM automates snapshot creation, retention, and deletion on a schedule.

Create an SLM Policy

curl -X PUT "localhost:9200/_slm/policy/nightly-snapshots" -H 'Content-Type: application/json' -d'
{
  "schedule": "0 30 2 * * ?",
  "name": "<nightly-snap-{now/d}>",
  "repository": "my_s3_repo",
  "config": {
    "indices": ["*"],
    "ignore_unavailable": true,
    "include_global_state": false
  },
  "retention": {
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }
}'

The schedule uses a cron expression — 0 30 2 * * ? runs at 02:30 UTC daily.

Execute a Policy Immediately

curl -X POST "localhost:9200/_slm/policy/nightly-snapshots/_execute"

View SLM Policy Status

curl -s "localhost:9200/_slm/policy/nightly-snapshots" | jq .

View SLM Stats

curl -s "localhost:9200/_slm/stats" | jq .

Restoring Snapshots

Full Restore

curl -X POST "localhost:9200/_snapshot/my_s3_repo/nightly-snap-2026.04.01/_restore" -H 'Content-Type: application/json' -d'
{
  "indices": "*",
  "include_global_state": true
}'

Partial Restore (Specific Indices)

curl -X POST "localhost:9200/_snapshot/my_s3_repo/nightly-snap-2026.04.01/_restore" -H 'Content-Type: application/json' -d'
{
  "indices": "logs-2026.03.*",
  "ignore_unavailable": true,
  "include_global_state": false
}'

Restore with Rename

Useful when restoring to the same cluster without deleting the original index:

curl -X POST "localhost:9200/_snapshot/my_s3_repo/nightly-snap-2026.04.01/_restore" -H 'Content-Type: application/json' -d'
{
  "indices": "logs-2026.03.31",
  "rename_pattern": "(.+)",
  "rename_replacement": "restored-$1",
  "include_global_state": false
}'

Restore to a Different Cluster

  1. Register the same repository on the target cluster (point to the same S3 bucket/path).
  2. List available snapshots: GET _snapshot/my_s3_repo/*
  3. Restore as above.

Important: The target cluster must be the same or newer major version as the cluster that created the snapshot.

Snapshot Status & Monitoring

Check Snapshot Progress

curl -s "localhost:9200/_snapshot/my_s3_repo/nightly-snap-2026.04.01/_status" | jq .

Response includes shard-level progress:

{
  "snapshots": [{
    "snapshot": "nightly-snap-2026.04.01",
    "state": "IN_PROGRESS",
    "shards_stats": {
      "total": 50,
      "done": 35,
      "started": 15
    }
  }]
}

List All Snapshots

curl -s "localhost:9200/_snapshot/my_s3_repo/*?verbose=false" | jq '.snapshots | length'

Delete Old Snapshots

curl -X DELETE "localhost:9200/_snapshot/my_s3_repo/nightly-snap-2026.03.01"

Searchable Snapshots

Searchable snapshots let you query data directly from the snapshot repository without fully restoring it, saving storage on your data nodes.

Mount a Searchable Snapshot

curl -X POST "localhost:9200/_snapshot/my_s3_repo/nightly-snap-2026.04.01/_mount?storage=shared_cache" -H 'Content-Type: application/json' -d'
{
  "index": "logs-2026.03.01",
  "renamed_index": "searchable-logs-2026.03.01"
}'

Storage options:

  • full_copy — Downloads all data to local storage (fast, uses more disk).
  • shared_cache — Uses a local cache; fetches from the repository on demand (slower, uses less disk).

Quick Reference

Task Command
Register FS repository PUT _snapshot/repo { "type": "fs", ... }
Register S3 repository PUT _snapshot/repo { "type": "s3", ... }
Create snapshot PUT _snapshot/repo/snap_name
Create SLM policy PUT _slm/policy/policy_name { ... }
Execute SLM now POST _slm/policy/policy_name/_execute
Restore snapshot POST _snapshot/repo/snap_name/_restore
Restore with rename Add rename_pattern + rename_replacement
Check snapshot status GET _snapshot/repo/snap_name/_status
Mount searchable snapshot POST _snapshot/repo/snap_name/_mount
Delete snapshot DELETE _snapshot/repo/snap_name
SLM stats GET _slm/stats

Next Steps

Backups protect against data loss, but you also need to know when something is going wrong before it becomes an emergency. Continue to Cluster Monitoring.

You may also enjoy