Amazon EKS — Amazon VPC CNI

The Amazon VPC CNI plugin is the default networking plugin for EKS. It gives each pod a real VPC IP address — not an overlay address. This means pods communicate directly within the VPC without encapsulation.

---

How VPC CNI Works

1. Each EC2 node has one or more Elastic Network Interfaces (ENIs)
2. Each ENI has a primary IP (for the node) and multiple secondary IPs
3. The VPC CNI assigns these secondary IPs to pods as they’re scheduled
4. Pod-to-pod traffic uses native VPC routing — no tunneling, no overlay

Node (m5.large)
├── ENI-0 (primary)
│   ├── Primary IP: 10.0.1.10 (node)
│   ├── Secondary IP: 10.0.1.11 → Pod A
│   ├── Secondary IP: 10.0.1.12 → Pod B
│   └── Secondary IP: 10.0.1.13 → Pod C
├── ENI-1
│   ├── Primary IP: 10.0.1.20
│   ├── Secondary IP: 10.0.1.21 → Pod D
│   └── Secondary IP: 10.0.1.22 → Pod E

---

Max Pods Per Node

The maximum number of pods per node depends on the instance type:

Max Pods = (Number of ENIs × (IPs per ENI - 1)) + 2

The -1 accounts for the primary IP on each ENI. The +2 accounts for the host networking pods.

Common Instance Limits

Instance Type	Max ENIs	IPs per ENI	Max Pods (default)
t3.medium	3	6	17
t3.large	3	12	35
m5.large	3	10	29
m5.xlarge	4	15	58
m5.2xlarge	4	15	58
c5.xlarge	4	15	58
r5.xlarge	4	15	58

Tip: Check the full list with: aws ec2 describe-instance-types --instance-types m5.large --query 'InstanceTypes[0].NetworkInfo'

---

WARM_ENI_TARGET and WARM_IP_TARGET

The VPC CNI pre-allocates IPs to reduce pod startup latency.

Variable	Default	Description
WARM_ENI_TARGET	1	Number of spare ENIs to keep ready
WARM_IP_TARGET	None	Number of spare IPs to keep ready
MINIMUM_IP_TARGET	None	Minimum IPs always available

Problem with Defaults

With WARM_ENI_TARGET=1, the CNI keeps an entire spare ENI attached. On an m5.large (10 IPs per ENI), this reserves 9 IPs per node even if they’re not being used.

For 10 nodes: 90 wasted IPs.

Optimized Configuration

kubectl set env daemonset aws-node -n kube-system \
  WARM_IP_TARGET=2 \
  MINIMUM_IP_TARGET=5

This tells the CNI to keep only 2 warm IPs (not a full ENI), reducing IP waste.

---

VPC CNI vs Overlay CNIs

Feature	VPC CNI	Overlay CNIs (Calico, Cilium)
Pod IPs	Real VPC IPs	Overlay IPs (virtual)
Performance	No encapsulation overhead	Slight overhead (VXLAN/Geneve)
AWS Integration	Native (security groups, VPC flow logs)	Requires additional config
IP Capacity	Limited by VPC CIDR	Virtually unlimited
Debugging	Easy (VPC Flow Logs, tcpdump)	Harder (overlay network)
Network Policies	Supported (VPC CNI v1.14+)	Native support

---

VPC CNI Configuration

View Current Configuration

kubectl describe daemonset aws-node -n kube-system

Important Environment Variables

Variable	Purpose
AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG	Enable custom networking (secondary CIDR for pods)
ENABLE_PREFIX_DELEGATION	Enable /28 prefix delegation (16 IPs per slot)
ENABLE_POD_ENI	Enable security groups for pods
POD_SECURITY_GROUP_ENFORCING_MODE	Strict or standard SG enforcement
WARM_ENI_TARGET	Spare ENIs to keep ready
WARM_IP_TARGET	Spare IPs to keep ready

---

CNI Metrics and Monitoring

The VPC CNI exposes Prometheus metrics:

# Port-forward to CNI metrics endpoint
kubectl port-forward -n kube-system \
  $(kubectl get pods -n kube-system -l k8s-app=aws-node -o name | head -1) \
  61678:61678

# Check metrics
curl http://localhost:61678/metrics | grep awscni

Key metrics to monitor:

• awscni_assigned_ip_addresses — Currently assigned IPs
• awscni_total_ip_addresses — Total IPs available on the node
• awscni_eni_allocated — Number of ENIs attached
• awscni_ip_max — Maximum IPs the node supports

---

What’s Next?

When you run out of VPC IPs, you need custom networking or prefix delegation. Let’s learn about both.

Next: Custom Networking & Prefix Delegation →