Amazon EKS — Cluster Autoscaler

Cluster Autoscaler adjusts the number of nodes by modifying AWS Auto Scaling Group (ASG) desired capacity. When pods can’t be scheduled due to insufficient resources, it scales up. When nodes are underutilized, it scales down.

---

How It Works

1. Pod is created but stays Pending (no node has capacity)
2. Cluster Autoscaler detects the unschedulable pod
3. It evaluates which ASG can accommodate the pod
4. It increases the ASG’s desired count
5. ASG launches a new EC2 instance
6. Node joins the cluster → pod gets scheduled

Scale-down works in reverse:

1. CA checks if nodes are underutilized (< 50% utilization by default)
2. It verifies pods can be rescheduled elsewhere
3. It cordons and drains the node
4. Decreases ASG desired count → EC2 terminates

---

Installation

IRSA Setup

cat > ca-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DescribeAutoScalingInstances",
        "autoscaling:DescribeLaunchConfigurations",
        "autoscaling:DescribeScalingActivities",
        "autoscaling:DescribeTags",
        "autoscaling:SetDesiredCapacity",
        "autoscaling:TerminateInstanceInAutoScalingGroup",
        "ec2:DescribeLaunchTemplateVersions",
        "ec2:DescribeInstanceTypes",
        "ec2:DescribeImages",
        "ec2:GetInstanceTypesFromInstanceRequirements",
        "eks:DescribeNodegroup"
      ],
      "Resource": "*"
    }
  ]
}
EOF

aws iam create-policy \
  --policy-name ClusterAutoscalerPolicy \
  --policy-document file://ca-policy.json

eksctl create iamserviceaccount \
  --cluster my-cluster \
  --namespace kube-system \
  --name cluster-autoscaler \
  --attach-policy-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):policy/ClusterAutoscalerPolicy \
  --approve

Helm Install

helm repo add autoscaler https://kubernetes.github.io/autoscaler

helm install cluster-autoscaler autoscaler/cluster-autoscaler \
  --namespace kube-system \
  --set autoDiscovery.clusterName=my-cluster \
  --set awsRegion=us-east-1 \
  --set rbac.serviceAccount.create=false \
  --set rbac.serviceAccount.name=cluster-autoscaler \
  --set extraArgs.balance-similar-node-groups=true \
  --set extraArgs.skip-nodes-with-system-pods=false

---

ASG Discovery Tags

Cluster Autoscaler discovers ASGs using tags:

k8s.io/cluster-autoscaler/enabled = true
k8s.io/cluster-autoscaler/my-cluster = owned

These are automatically added by eksctl and the EKS Terraform module for managed node groups.

---

Configuration Parameters

Parameter	Default	Description
--scale-down-utilization-threshold	0.5	Scale down when node < 50% utilized
--scale-down-unneeded-time	10m	Wait time before scale down
--scale-down-delay-after-add	10m	Cooldown after adding a node
--max-graceful-termination-sec	600	Max time to drain a node
--balance-similar-node-groups	false	Balance nodes across ASGs (set to true)
--skip-nodes-with-system-pods	true	Don’t scale down nodes with kube-system pods
--expander	random	Strategy for choosing which ASG to scale: random, most-pods, least-waste, priority

---

Multiple Node Groups

# Scale parameters per node group via ASG tags
# Or use priority expander
cat > priority-expander.yaml << 'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-autoscaler-priority-expander
  namespace: kube-system
data:
  priorities: |
    50:
      - .*spot.*
    30:
      - .*on-demand.*
EOF

kubectl apply -f priority-expander.yaml

---

Limitations

Limitation	Impact
Scales via ASG (indirect)	Slower — must wait for ASG, EC2, node join
One instance type per ASG	Less flexible — can’t mix instance types in one group
No bin-packing optimization	Doesn’t choose optimal instance type per workload
Scale-up latency	2-5 minutes from pending pod to running
ASG sizing constraints	Must pre-define min/max for each ASG

Recommendation: For new clusters, use Karpenter instead — it addresses all these limitations.

---

Best Practices

1. Set balance-similar-node-groups: true — distribute across AZs evenly
2. Use least-waste expander — better bin-packing than random
3. Set appropriate min/max on ASGs — CA won’t exceed these limits
4. Don’t set CPU limits too low — CA uses requests for scheduling, not limits
5. Add PodDisruptionBudgets — CA respects PDBs during scale-down
6. Consider migrating to Karpenter — faster, smarter, more cost-effective

---