Amazon EKS — GPU & ARM Instances

1 minute read

GPU & ARM Instances on EKS

EKS supports specialized instance types for GPU workloads (ML inference, training) and ARM-based Graviton processors (cost optimization).

GPU Instances

Instance Family GPU Use Case Cost (On-Demand, us-east-1)
p3.2xlarge 1x V100 (16 GB) Training $3.06/hr
p3.8xlarge 4x V100 Distributed training $12.24/hr
g4dn.xlarge 1x T4 (16 GB) Inference $0.526/hr
g5.xlarge 1x A10G (24 GB) Inference + training $1.006/hr

GPU Node Group

managedNodeGroups:
  - name: gpu
    instanceType: g4dn.xlarge
    desiredCapacity: 1
    minSize: 0
    maxSize: 4
    labels:
      nvidia.com/gpu: "true"
    taints:
      - key: nvidia.com/gpu
        value: "true"
        effect: NoSchedule

Deploy GPU Workload

apiVersion: v1
kind: Pod
metadata:
  name: gpu-test
spec:
  tolerations:
    - key: nvidia.com/gpu
      operator: Exists
      effect: NoSchedule
  containers:
    - name: cuda
      image: nvidia/cuda:12.0.0-base-ubuntu22.04
      command: ["nvidia-smi"]
      resources:
        limits:
          nvidia.com/gpu: 1

ARM / Graviton Instances

AWS Graviton processors offer up to 40% better price-performance than x86:

Instance vCPU Memory x86 Equivalent Savings
m6g.large 2 8 GB m5.large ~20%
m7g.large 2 8 GB m5.large ~30%
c7g.xlarge 4 8 GB c5.xlarge ~30% 
managedNodeGroups:
  - name: graviton
    instanceType: m7g.large
    desiredCapacity: 3
    labels:
      arch: arm64

Important: Your container images must be built for ARM (multi-arch). Most official images support both amd64 and arm64.

Common Mistakes

Mistake Fix
GPU nodes without taint Non-GPU pods scheduled on expensive GPU nodes
x86-only images on Graviton Build multi-arch images
GPU nodes always running Use minSize: 0 with Karpenter/Cluster Autoscaler

What’s Next?

Save even more: Spot Instances.

You may also enjoy