Amazon EKS — Spot Instances Strategy

2 minute read

Spot Instances let you use unused EC2 capacity at up to 90% discount. For fault-tolerant Kubernetes workloads — web servers, batch jobs, CI runners — Spot is the single largest cost saver.

How Spot Works

  1. You request Spot capacity from a pool (instance type + AZ)
  2. AWS provides it at the Spot price (fluctuating, but typically 60-90% off On-Demand)
  3. If AWS needs the capacity back, you get a 2-minute interruption notice
  4. The instance is terminated (your pods are evicted)

Spot-Friendly Workloads

Workload Spot-Friendly? Notes
Stateless web servers ✅ Yes Multiple replicas handle interruptions
Batch processing ✅ Yes Jobs can restart
CI/CD runners ✅ Yes Builds can be retried
Dev/test environments ✅ Yes Interruptions acceptable
Databases ❌ No Use On-Demand or Reserved
Single-replica critical services ❌ No No redundancy during interruption
Long-running ML training ⚠️ Maybe Use checkpointing

Capacity-Optimized Allocation

Use capacity-optimized allocation strategy — AWS selects pools with the most available capacity, reducing interruption rates:

eksctl create nodegroup \
  --cluster my-cluster \
  --name spot-workers \
  --instance-types m6i.large,m6i.xlarge,m5.large,m5.xlarge,m5a.large,m5a.xlarge \
  --spot \
  --nodes-min 2 \
  --nodes-max 20 \
  --node-labels "lifecycle=spot" \
  --node-taints "spot=true:NoSchedule"

Critical: Diversify across at least 6-10 instance types and all AZs. Don’t rely on a single instance type.

Mixed On-Demand + Spot Strategy

Run critical workloads on On-Demand, everything else on Spot:

# On-Demand node group (critical)
eksctl create nodegroup \
  --cluster my-cluster \
  --name on-demand-critical \
  --instance-types m6i.xlarge \
  --nodes-min 2 \
  --nodes-max 5 \
  --node-labels "lifecycle=on-demand,workload=critical"

# Spot node group (non-critical)
eksctl create nodegroup \
  --cluster my-cluster \
  --name spot-general \
  --instance-types m6i.large,m6i.xlarge,m5.large,m5.xlarge,m5a.large \
  --spot \
  --nodes-min 2 \
  --nodes-max 30 \
  --node-labels "lifecycle=spot,workload=general" \
  --node-taints "spot=true:NoSchedule"

Schedule workloads with tolerations and affinity:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: non-critical-app
spec:
  replicas: 4
  template:
    spec:
      tolerations:
        - key: spot
          operator: Equal
          value: "true"
          effect: NoSchedule
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              preference:
                matchExpressions:
                  - key: lifecycle
                    operator: In
                    values: ["spot"]
      containers:
        - name: app
          image: my-app:latest

AWS Node Termination Handler

Catches Spot interruption notices and gracefully drains nodes:

helm repo add eks https://aws.github.io/eks-charts

helm install aws-node-termination-handler eks/aws-node-termination-handler \
  --namespace kube-system \
  --set enableSpotInterruptionDraining=true \
  --set enableScheduledEventDraining=true \
  --set enableRebalanceMonitoring=true \
  --set enableRebalanceDraining=true

What it does:

  1. Monitors EC2 metadata for Spot interruption warnings (2 min notice)
  2. Cordons the node (no new pods scheduled)
  3. Drains the node (evicts pods, respecting PDBs)
  4. Pods reschedule on remaining nodes

Karpenter Spot Integration

Karpenter handles Spot natively and is the recommended approach:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values: ["m6i.large", "m6i.xlarge", "m5.large", "m5.xlarge", "m5a.large", "c6i.large", "c6i.xlarge", "r6i.large"]
        - key: topology.kubernetes.io/zone
          operator: In
          values: ["us-east-1a", "us-east-1b", "us-east-1c"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m

Karpenter with Spot:

  • Automatically replaces Spot instances when interrupted
  • Tries multiple instance types and AZs simultaneously
  • Falls back to On-Demand if no Spot available (if configured)

Savings Comparison

Strategy Monthly Compute Cost Savings
All On-Demand (10 × m6i.xlarge) $1,400 Baseline
All Spot (10 × m6i.xlarge) $420-560 60-70%
Mixed (3 OD + 7 Spot) $714-812 42-49%
Reserved (1yr) $910 35%
Savings Plans (1yr) $952 32%

Best Practices

  1. Diversify instance types — at least 6-10 types across 3 AZs
  2. Use capacity-optimized allocation — not lowest-price
  3. Install Node Termination Handler (if not using Karpenter) — graceful drains
  4. Taint Spot nodes — only schedule tolerant workloads
  5. Always run PDBs — prevent all replicas from being evicted simultaneously
  6. Use Karpenter — it handles Spot more intelligently than ASG-based approaches

Updated:

You may also enjoy