ELK Stack — Index Lifecycle Management

5 minute read

Index Lifecycle Management

Index Lifecycle Management (ILM) automates the process of managing indices as they age. You define a policy that describes what should happen at each phase of an index’s life, and Elasticsearch handles the transitions for you.

Why ILM Matters

Without ILM you must manually:

Roll over indices when they get too large.
Move old indices to cheaper storage.
Force-merge read-only indices to reclaim space.
Delete indices after the retention period.

ILM eliminates this operational burden and ensures consistent behavior across your cluster.

ILM Phases Lifecycle

┌──────────┐     ┌──────────┐     ┌──────────┐     ┌──────────┐     ┌──────────┐
│          │     │          │     │          │     │          │     │          │
│   HOT    │────▶│   WARM   │────▶│   COLD   │────▶│  FROZEN  │────▶│  DELETE  │
│          │     │          │     │          │     │          │     │          │
│ Active   │     │ Read-    │     │ Infre-   │     │ Rarely   │     │ Remove   │
│ indexing │     │ heavy    │     │ quent    │     │ accessed │     │ index    │
│ & search │     │ queries  │     │ access   │     │ (search- │     │          │
│          │     │          │     │          │     │  able     │     │          │
│          │     │          │     │          │     │  snapshot)│     │          │
└──────────┘     └──────────┘     └──────────┘     └──────────┘     └──────────┘
   Day 0          Day 7           Day 30           Day 90           Day 365

Each phase is optional except hot. You can skip any phase that does not apply to your use case.

Phase Overview

Phase	Purpose	Typical Actions
Hot	Active writes and frequent queries	Rollover, set priority
Warm	No more writes, still queried regularly	Shrink, forcemerge, allocate, read-only
Cold	Infrequent access, lower-cost hardware	Allocate to cold nodes, freeze
Frozen	Rare access via searchable snapshots	Searchable snapshot mount
Delete	Data no longer needed	Delete index

Creating an ILM Policy

curl -s -X PUT "localhost:9200/_ilm/policy/logs-policy" \
  -H "Content-Type: application/json" \
  -d '{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_primary_shard_size": "50gb",
            "max_age": "7d"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "warm": {
        "min_age": "7d",
        "actions": {
          "shrink": {
            "number_of_shards": 1
          },
          "forcemerge": {
            "max_num_segments": 1
          },
          "allocate": {
            "require": {
              "data": "warm"
            }
          },
          "set_priority": {
            "priority": 50
          }
        }
      },
      "cold": {
        "min_age": "30d",
        "actions": {
          "allocate": {
            "require": {
              "data": "cold"
            }
          },
          "set_priority": {
            "priority": 0
          }
        }
      },
      "delete": {
        "min_age": "365d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}'

{
  "acknowledged": true
}

Actions by Phase

Hot Phase Actions

Action	Description
`rollover`	Create a new index when the current one hits a size, age, or document count threshold
`set_priority`	Set the index priority for recovery ordering

Warm Phase Actions

Action	Description
`shrink`	Reduce the number of primary shards
`forcemerge`	Merge segments to reduce storage and improve query speed
`allocate`	Move index to specific nodes (e.g., warm-tier hardware)
`readonly`	Set the index to read-only
`set_priority`	Adjust recovery priority

Cold Phase Actions

Action	Description
`allocate`	Move to cold-tier nodes
`freeze`	Freeze the index to reduce memory overhead (deprecated in 8.x, use frozen tier)
`searchable_snapshot`	Mount as a searchable snapshot
`set_priority`	Adjust recovery priority

Delete Phase Actions

Action	Description
`delete`	Permanently remove the index
`wait_for_snapshot`	Wait for a snapshot before deleting

Attaching an ILM Policy to an Index Template

The recommended approach is to reference the ILM policy in your index template so every new index automatically gets the policy:

curl -s -X PUT "localhost:9200/_index_template/logs-template" \
  -H "Content-Type: application/json" \
  -d '{
  "index_patterns": ["logs-*"],
  "template": {
    "settings": {
      "number_of_shards": 2,
      "number_of_replicas": 1,
      "index.lifecycle.name": "logs-policy",
      "index.lifecycle.rollover_alias": "logs-write"
    },
    "aliases": {
      "logs-write": {
        "is_write_index": true
      }
    }
  },
  "priority": 200
}'

{
  "acknowledged": true
}

Bootstrap the First Index

For rollover to work with aliases, bootstrap the first index with a numeric suffix:

curl -s -X PUT "localhost:9200/logs-000001" \
  -H "Content-Type: application/json" \
  -d '{
  "aliases": {
    "logs-write": {
      "is_write_index": true
    }
  }
}'

{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "logs-000001"
}

Monitoring ILM Status

Check ILM Status for an Index

curl -s -X GET "localhost:9200/logs-000001/_ilm/explain?pretty"

{
  "indices": {
    "logs-000001": {
      "index": "logs-000001",
      "managed": true,
      "policy": "logs-policy",
      "lifecycle_date_millis": 1711958400000,
      "age": "3d",
      "phase": "hot",
      "phase_time_millis": 1711958400000,
      "action": "complete",
      "action_time_millis": 1711958400000,
      "step": "complete",
      "step_time_millis": 1711958400000,
      "phase_execution": {
        "policy": "logs-policy",
        "phase_definition": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "7d"
            },
            "set_priority": {
              "priority": 100
            }
          }
        }
      }
    }
  }
}

Check Global ILM Operating Mode

curl -s -X GET "localhost:9200/_ilm/status?pretty"

{
  "operation_mode": "RUNNING"
}

Get a Specific ILM Policy

curl -s -X GET "localhost:9200/_ilm/policy/logs-policy?pretty"

Manual ILM Operations

Manually Move an Index to the Next Phase

If you need to force an index into a different phase immediately:

curl -s -X POST "localhost:9200/_ilm/move/logs-000001" \
  -H "Content-Type: application/json" \
  -d '{
  "current_step": {
    "phase": "hot",
    "action": "complete",
    "name": "complete"
  },
  "next_step": {
    "phase": "warm",
    "action": "shrink",
    "name": "shrink"
  }
}'

{
  "acknowledged": true
}

Retry a Failed ILM Step

When an ILM step fails (e.g., allocation timeout), retry it:

curl -s -X POST "localhost:9200/_ilm/retry/logs-000001"

{
  "acknowledged": true
}

Stop and Start ILM

# Stop ILM (useful during maintenance)
curl -s -X POST "localhost:9200/_ilm/stop"

# Start ILM
curl -s -X POST "localhost:9200/_ilm/start"

Practical Example: Application Logs

Here is a complete ILM setup for application logs that:

Rolls over every 30 GB or 1 day.
Shrinks to 1 shard and force-merges after 2 days.
Moves to cold storage after 14 days.
Deletes after 90 days.

curl -s -X PUT "localhost:9200/_ilm/policy/app-logs-policy" \
  -H "Content-Type: application/json" \
  -d '{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_primary_shard_size": "30gb",
            "max_age": "1d"
          }
        }
      },
      "warm": {
        "min_age": "2d",
        "actions": {
          "shrink": { "number_of_shards": 1 },
          "forcemerge": { "max_num_segments": 1 }
        }
      },
      "cold": {
        "min_age": "14d",
        "actions": {
          "allocate": {
            "require": { "data": "cold" }
          }
        }
      },
      "delete": {
        "min_age": "90d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}'

{
  "acknowledged": true
}

Deleting an ILM Policy

curl -s -X DELETE "localhost:9200/_ilm/policy/logs-policy"

{
  "acknowledged": true
}

Note: You cannot delete a policy that is currently in use by any index. Remove the policy reference from all indices first.

Quick Reference

Task	Command
Create / update ILM policy	`PUT _ilm/policy/<policy-name>`
Get ILM policy	`GET _ilm/policy/<policy-name>`
Delete ILM policy	`DELETE _ilm/policy/<policy-name>`
Explain ILM for an index	`GET <index>/_ilm/explain`
Check ILM operating mode	`GET _ilm/status`
Manually move to next step	`POST _ilm/move/<index>`
Retry a failed step	`POST _ilm/retry/<index>`
Stop ILM	`POST _ilm/stop`
Start ILM	`POST _ilm/start`

Next → Aliases

Share on

Twitter Facebook LinkedIn