Amazon EKS — Spot and Graviton Instances

4 minute read

Running EKS workloads on Spot instances can save up to 90% compared to On-Demand pricing. Graviton (ARM) instances offer up to 40% better price-performance. Combining both gives you the most cost-effective EKS clusters.

Spot Instances for EKS

How Spot Instances Work

  • Spot instances use spare EC2 capacity at steep discounts (typically 60-90% off)
  • AWS can reclaim Spot capacity with a 2-minute interruption notice
  • You need to design workloads to handle interruptions gracefully

Creating a Spot Node Group

eksctl create nodegroup \
  --cluster my-cluster \
  --name spot-nodes \
  --instance-types m6i.large,m5.large,m6a.large,m5a.large \
  --spot \
  --nodes 3 \
  --nodes-min 1 \
  --nodes-max 10

eksctl ClusterConfig

managedNodeGroups:
  - name: spot-nodes
    instanceTypes:
      - m6i.large
      - m5.large
      - m6a.large
      - m5a.large
      - c6i.large
      - c5.large
    spot: true
    desiredCapacity: 3
    minSize: 1
    maxSize: 10
    labels:
      capacity-type: spot
    taints:
      - key: spot
        value: "true"
        effect: NoSchedule

Terraform

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.0"

  eks_managed_node_groups = {
    spot_nodes = {
      instance_types = ["m6i.large", "m5.large", "m6a.large"]
      capacity_type  = "SPOT"
      min_size       = 1
      max_size       = 10
      desired_size   = 3

      labels = {
        "capacity-type" = "spot"
      }

      taints = {
        spot = {
          key    = "spot"
          value  = "true"
          effect = "NO_SCHEDULE"
        }
      }
    }
  }
}

Handling Spot Interruptions

AWS Node Termination Handler

NTH watches for Spot interruption notices and gracefully drains pods before the node is reclaimed:

helm repo add eks https://aws.github.io/eks-charts
helm install aws-node-termination-handler eks/aws-node-termination-handler \
  --namespace kube-system \
  --set enableSpotInterruptionDraining=true \
  --set enableScheduledEventDraining=true

How NTH Works

  1. Watches EC2 metadata for interruption notices
  2. Cordons the node (stops new pods from scheduling)
  3. Drains existing pods (respects PodDisruptionBudgets)
  4. Pods get rescheduled on other available nodes
  5. Node is terminated by AWS after 2 minutes

Design Patterns for Spot

Pattern How
Multiple instance types Specify 5-10 types for Spot diversity
Capacity-optimized allocation AWS picks instances from the deepest Spot pools
Spread across AZs Reduces blast radius of Spot reclamation
Tolerations on Spot-friendly workloads Only place interruptible workloads on Spot
PodDisruptionBudgets Protect critical workloads during draining

Mixed Instance Strategy

Combine On-Demand and Spot in the same cluster:

managedNodeGroups:
  # On-Demand for critical workloads
  - name: on-demand-nodes
    instanceType: m6i.xlarge
    desiredCapacity: 3
    minSize: 3
    maxSize: 6
    labels:
      capacity-type: on-demand

  # Spot for non-critical workloads
  - name: spot-nodes
    instanceTypes:
      - m6i.large
      - m5.large
      - m6a.large
    spot: true
    desiredCapacity: 5
    minSize: 0
    maxSize: 20
    labels:
      capacity-type: spot
    taints:
      - key: spot
        value: "true"
        effect: NoSchedule

Schedule Spot-Friendly Workloads

apiVersion: apps/v1
kind: Deployment
metadata:
  name: batch-processor
spec:
  replicas: 5
  template:
    spec:
      tolerations:
        - key: spot
          operator: Equal
          value: "true"
          effect: NoSchedule
      nodeSelector:
        capacity-type: spot
      containers:
        - name: processor
          image: my-app:latest
          resources:
            requests:
              cpu: "500m"
              memory: "512Mi"

Graviton (ARM) Instances

Why Graviton?

  • Up to 40% better price-performance than comparable x86 instances
  • Lower energy consumption
  • Same or better performance for most workloads
  • Available in all major EKS regions

Graviton Instance Types

Type vCPU Memory Use Case
m7g.medium 1 4 GB Dev/test
m7g.xlarge 4 16 GB General purpose
c7g.xlarge 4 8 GB Compute-intensive
r7g.xlarge 4 32 GB Memory-intensive
t4g.medium 2 4 GB Burstable, dev/test

Creating a Graviton Node Group

managedNodeGroups:
  - name: graviton-nodes
    instanceType: m7g.xlarge
    desiredCapacity: 3
    minSize: 2
    maxSize: 10
    labels:
      arch: arm64

Multi-Architecture Container Images

Your containers must support ARM to run on Graviton. Use multi-arch manifests:

# Build for both architectures
# In your CI/CD pipeline:
docker buildx create --name multi-arch --use
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --tag your-ecr-repo/app:latest \
  --push .

Check if your image supports ARM:

docker manifest inspect your-ecr-repo/app:latest
# Look for: "architecture": "arm64"

Running Mixed Architecture Clusters

managedNodeGroups:
  - name: x86-nodes
    instanceType: m6i.xlarge
    desiredCapacity: 3
    labels:
      arch: amd64

  - name: arm-nodes
    instanceType: m7g.xlarge
    desiredCapacity: 3
    labels:
      arch: arm64

Schedule pods on the correct architecture:

spec:
  nodeSelector:
    kubernetes.io/arch: arm64    # or amd64

Or let Kubernetes handle it with multi-arch images — the scheduler picks any available node.

Spot + Graviton = Maximum Savings

Combine both for the lowest cost:

managedNodeGroups:
  - name: spot-graviton
    instanceTypes:
      - m7g.large
      - m6g.large
      - c7g.large
      - c6g.large
      - r7g.large
    spot: true
    desiredCapacity: 5
    minSize: 0
    maxSize: 20
    labels:
      capacity-type: spot
      arch: arm64

Savings stack:

  • Graviton: ~20-40% cheaper than x86
  • Spot: ~60-90% cheaper than On-Demand
  • Combined: up to 90%+ savings vs x86 On-Demand

Cost Comparison Example

Running 10 nodes, 24/7 for a month (m-type, xlarge, us-east-1):

Configuration Instance Price/hr Monthly (10 nodes)
x86 On-Demand m6i.xlarge $0.192 ~$1,382
Graviton On-Demand m7g.xlarge $0.163 ~$1,174
x86 Spot m6i.xlarge ~$0.058 ~$418
Graviton Spot m7g.xlarge ~$0.049 ~$353

Savings from x86 On-Demand to Graviton Spot: ~75%.

Best Practices

  1. Use multiple instance types for Spot — diversify across at least 5 types to reduce interruption risk
  2. Taint Spot nodes — prevent critical workloads from accidentally scheduling on Spot
  3. Install Node Termination Handler — essential for graceful Spot interruption handling
  4. Design for interruption — stateless workloads, checkpointing, retries
  5. Start with Graviton for new workloads — price-performance is consistently better
  6. Build multi-arch images from the start — it’s harder to add later
  7. Monitor Spot interruption frequency — CloudWatch metrics and Spot placement scores
  8. Use On-Demand for stateful workloads — databases, queues, persistent services

Updated:

You may also enjoy