HashiCorp Vault — Raft Storage

2 minute read

Raft is Vault’s integrated storage backend — no external dependencies like Consul or etcd. It provides both data persistence and HA leader election.

Why Raft?

External Storage (Consul) Integrated Storage (Raft)
Requires separate Consul cluster No external dependencies
More infrastructure to manage Self-contained
Proven at scale Simpler operations
Separate failure domain Vault manages its own data

How Raft Consensus Works

1. Client sends write to Active node
2. Active node writes to local WAL (Write-Ahead Log)
3. Active replicates to standbys
4. Majority acknowledges (quorum)
5. Write is committed
6. Client gets success response

Quorum = (N/2) + 1
  3 nodes → need 2 (tolerate 1 failure)
  5 nodes → need 3 (tolerate 2 failures)

Cluster Sizing

Nodes Quorum Failure Tolerance Recommended For
1 1 0 Dev/test only
3 2 1 Small production
5 3 2 Standard production
7 4 3 Large production

Recommendation: Use 3 or 5 nodes. More than 7 increases write latency without significant benefit.

Configuration

Node 1 (Initial Leader)

# /etc/vault/vault.hcl — Node 1

storage "raft" {
  path    = "/vault/data"
  node_id = "vault-node-1"
}

listener "tcp" {
  address         = "0.0.0.0:8200"
  cluster_address = "0.0.0.0:8201"
  tls_cert_file   = "/etc/vault/tls/cert.pem"
  tls_key_file    = "/etc/vault/tls/key.pem"
}

api_addr     = "https://vault-1.internal:8200"
cluster_addr = "https://vault-1.internal:8201"
cluster_name = "vault-prod"

Node 2 and 3

Same config but with different node_id, api_addr, and cluster_addr.

Bootstrap the Cluster

# Start all 3 nodes, then initialize node 1
vault operator init -key-shares=5 -key-threshold=3

# Unseal node 1
vault operator unseal <key1>
vault operator unseal <key2>
vault operator unseal <key3>

# Join node 2 to the cluster
VAULT_ADDR=https://vault-2.internal:8200 \
vault operator raft join https://vault-1.internal:8200

# Unseal node 2
VAULT_ADDR=https://vault-2.internal:8200 \
vault operator unseal <key1>
vault operator unseal <key2>
vault operator unseal <key3>

# Join node 3
VAULT_ADDR=https://vault-3.internal:8200 \
vault operator raft join https://vault-1.internal:8200

# Unseal node 3
VAULT_ADDR=https://vault-3.internal:8200 \
vault operator unseal <key1>
vault operator unseal <key2>
vault operator unseal <key3>

Manage Raft Cluster

# List cluster members
vault operator raft list-peers

# Expected output:
# Node          Address                   State       Voter
# ----          -------                   -----       -----
# vault-node-1  vault-1.internal:8201     leader      true
# vault-node-2  vault-2.internal:8201     follower    true
# vault-node-3  vault-3.internal:8201     follower    true

# Remove a node (if decommissioning)
vault operator raft remove-peer vault-node-3

Raft Snapshots

# Create a snapshot (backup)
vault operator raft snapshot save /tmp/vault-snapshot.snap

# Restore from snapshot
vault operator raft snapshot restore /tmp/vault-snapshot.snap

# Automate backups
cat <<'EOF' > /usr/local/bin/vault-backup.sh
#!/bin/bash
DATE=$(date +%Y%m%d-%H%M%S)
vault operator raft snapshot save "/backups/vault-${DATE}.snap"
# Keep last 30 days
find /backups -name "vault-*.snap" -mtime +30 -delete
EOF
chmod +x /usr/local/bin/vault-backup.sh

# Crontab: daily backup at 2 AM
# 0 2 * * * /usr/local/bin/vault-backup.sh

Raft Autopilot

Vault includes autopilot features for Raft cluster management:

# View autopilot configuration
vault operator raft autopilot get-config

# Update autopilot settings
vault operator raft autopilot set-config \
    cleanup_dead_servers=true \
    dead_server_last_contact_threshold=24h \
    min_quorum=3

# Check autopilot cluster state
vault operator raft autopilot state

Quick Reference

Command Description
vault operator raft list-peers List cluster members
vault operator raft join URL Join a node to the cluster
vault operator raft remove-peer ID Remove a node
vault operator raft snapshot save FILE Create backup
vault operator raft snapshot restore FILE Restore backup
vault operator raft autopilot state Cluster health

Next: Auto-Unseal

You may also enjoy