Amazon EKS — Pod Debugging

1 minute read

Pod Debugging

A systematic approach to diagnosing pod issues on EKS.

The Debug Flowchart

Pod Not Working?
│
├── kubectl get pod → STATUS?
│   │
│   ├── Pending
│   │   └── kubectl describe pod → Events?
│   │       ├── Insufficient cpu/memory → Node scaling issue
│   │       ├── No nodes match → Check nodeSelector/affinity
│   │       └── PVC not bound → Check storage class
│   │
│   ├── CrashLoopBackOff
│   │   └── kubectl logs pod (--previous) → Error?
│   │       ├── Application error → Fix code/config
│   │       ├── OOMKilled → Increase memory limit
│   │       └── Permission denied → Check security context
│   │
│   ├── ImagePullBackOff
│   │   └── kubectl describe pod → Image?
│   │       ├── Wrong image name → Fix image reference
│   │       ├── Auth error → Check imagePullSecrets
│   │       └── ECR → Check IRSA/node IAM role
│   │
│   ├── Running but not ready
│   │   └── Check readiness probe
│   │       ├── Wrong port/path → Fix probe config
│   │       └── App slow to start → Increase initialDelay
│   │
│   └── ErrImageNeverPull
│       └── imagePullPolicy: Never + image not on node
│           └── Change to Always or IfNotPresent

Essential Commands

# Status overview
$ kubectl get pods -o wide

# Detailed events
$ kubectl describe pod <name>

# Current logs
$ kubectl logs <name>

# Previous container logs (after crash)
$ kubectl logs <name> --previous

# Multi-container pod
$ kubectl logs <name> -c <container>

# Interactive debugging
$ kubectl exec -it <name> -- /bin/sh

# Ephemeral debug container (if no shell in image)
$ kubectl debug -it <name> --image=busybox --target=<container>

# Resource usage
$ kubectl top pod <name>

CrashLoopBackOff: Common Causes

Cause How to Identify Fix
App error Check logs Fix application bug
OOMKilled describe pod shows OOMKilled Increase memory limit
Missing config Logs show “config not found” Check ConfigMap/Secret mounts
Wrong command Logs show “exec format error” Check container command/args
Permission Logs show “permission denied” Fix securityContext or file permissions

Common Mistakes

Mistake Impact
Not checking --previous logs Missing the actual crash error
Debugging in production namespace Risk to live traffic
Not using kubectl debug Stuck when container has no shell

What’s Next?

Node problems: Node Issues.

You may also enjoy