CKA — Troubleshoot Control Plane

8 minute read

CKA Exam Domain

Domain Weight
Troubleshooting 30%

Control plane troubleshooting is one of the most challenging — and most rewarding — skills on the CKA exam. When the API server, scheduler, controller manager, or etcd breaks, the entire cluster suffers. You need to know how to find and fix these issues even when kubectl itself isn’t working.

Control Plane Troubleshooting Flow

Follow this systematic approach:

kubectl cluster-info → Check static Pod manifests → kubectl get pods -n kube-system → Logs / journalctl
  1. Can you reach the API server? — kubectl cluster-info
  2. Are control plane Pods running? — kubectl get pods -n kube-system
  3. Check static Pod manifests — /etc/kubernetes/manifests/
  4. Read the logs — kubectl logs or crictl logs if kubectl is down

Exam Tip: Control plane components in kubeadm clusters run as static Pods. Their manifests live in /etc/kubernetes/manifests/. If you break the manifest, the kubelet automatically stops the Pod. Fix the manifest and the kubelet restarts it automatically.

API Server Not Responding

When kubectl commands hang or return connection errors, the API server is likely down.

Symptoms

kubectl get pods
# The connection to the server 10.0.0.1:6443 was refused

Debugging Steps

# Step 1: Check the static Pod manifest
cat /etc/kubernetes/manifests/kube-apiserver.yaml

# Step 2: Check if the API server container is running
crictl ps | grep apiserver
crictl ps -a | grep apiserver  # include stopped containers

# Step 3: If it's not running, check why
crictl logs <container-id>

# Step 4: Check kubelet logs (kubelet manages static Pods)
journalctl -u kubelet | grep "apiserver" | tail -20

Common API Server Issues

Wrong flag or argument:

# BROKEN — wrong etcd endpoint
spec:
  containers:
  - command:
    - kube-apiserver
    - --etcd-servers=https://127.0.0.1:2378  # wrong port! should be 2379

# FIXED
    - --etcd-servers=https://127.0.0.1:2379

Wrong certificate path:

# BROKEN — typo in certificate path
    - --tls-cert-file=/etc/kubernetes/pki/apiserverr.crt  # extra 'r'

# FIXED
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt

etcd unreachable:

# If the API server logs say etcd is unreachable, troubleshoot etcd first
crictl logs <apiserver-container-id> 2>&1 | grep etcd

Exam Tip: After editing a static Pod manifest in /etc/kubernetes/manifests/, the kubelet automatically restarts the Pod. You don’t need to run any restart command. But wait 10–30 seconds and verify.

Scheduler Failures

When the scheduler is broken, new Pods stay in Pending state and you won’t see any scheduling events.

Symptoms

kubectl get pods
# NAME     READY   STATUS    RESTARTS   AGE
# my-app   0/1     Pending   0          5m

kubectl describe pod my-app
# Events section is empty — no scheduling attempt!

Debugging Steps

# Check if the scheduler Pod is running
kubectl get pods -n kube-system -l component=kube-scheduler

# Get scheduler logs
kubectl logs -n kube-system kube-scheduler-controlplane

# Check the static Pod manifest
cat /etc/kubernetes/manifests/kube-scheduler.yaml

Common Scheduler Issues

Wrong config path:

# BROKEN
spec:
  containers:
  - command:
    - kube-scheduler
    - --kubeconfig=/etc/kubernetes/schedulerr.conf  # typo!

# FIXED
    - --kubeconfig=/etc/kubernetes/scheduler.conf

Scheduler bound to wrong port:

# BROKEN
    - --secure-port=10260  # wrong port

# FIXED
    - --secure-port=10259

Controller Manager Failures

The controller manager runs controllers for Deployments, ReplicaSets, Services, and more. When it’s broken, higher-level resources stop working.

Symptoms

  • Deployments don’t create ReplicaSets
  • ReplicaSets don’t create Pods
  • Services don’t get Endpoints
  • Nodes don’t get updated lease objects

Debugging Steps

# Check if the controller manager Pod is running
kubectl get pods -n kube-system -l component=kube-controller-manager

# Get controller manager logs
kubectl logs -n kube-system kube-controller-manager-controlplane

# Check the static Pod manifest
cat /etc/kubernetes/manifests/kube-controller-manager.yaml

Common Controller Manager Issues

# BROKEN — wrong kubeconfig path
spec:
  containers:
  - command:
    - kube-controller-manager
    - --kubeconfig=/etc/kubernetes/controller-managerr.conf  # typo!

# FIXED
    - --kubeconfig=/etc/kubernetes/controller-manager.conf

# BROKEN — wrong CA certificate path
    - --root-ca-file=/etc/kubernetes/pki/caa.crt  # extra 'a'

# FIXED
    - --root-ca-file=/etc/kubernetes/pki/ca.crt

etcd Issues

etcd is the cluster’s database. If etcd is down, the API server can’t read or write anything, and effectively the entire cluster is down.

Debugging Steps

# Check if etcd Pod is running
kubectl get pods -n kube-system -l component=etcd

# Get etcd logs
kubectl logs -n kube-system etcd-controlplane

# If kubectl doesn't work (API server down because etcd is down):
crictl ps | grep etcd
crictl ps -a | grep etcd
crictl logs <etcd-container-id>

Common etcd Issues

Wrong data directory:

# BROKEN
spec:
  containers:
  - command:
    - etcd
    - --data-dir=/var/lib/etcdd  # typo — directory doesn't exist

# FIXED
    - --data-dir=/var/lib/etcd

Volume mount mismatch:

The --data-dir flag must match the volume mount path. This is a classic CKA trap:

# BROKEN — data-dir and mount path don't match
    - --data-dir=/var/lib/etcd-custom
    volumeMounts:
    - mountPath: /var/lib/etcd      # doesn't match data-dir!
      name: etcd-data

# FIXED — both paths match
    - --data-dir=/var/lib/etcd
    volumeMounts:
    - mountPath: /var/lib/etcd
      name: etcd-data

Wrong certificate paths:

# Check that all certificate files referenced in the etcd manifest actually exist
ls -la /etc/kubernetes/pki/etcd/

Static Pod Manifest Debugging

All control plane components in a kubeadm cluster are static Pods managed by the kubelet. Their manifests are in:

/etc/kubernetes/manifests/
├── etcd.yaml
├── kube-apiserver.yaml
├── kube-controller-manager.yaml
└── kube-scheduler.yaml

Common Mistakes in Static Pod Manifests

  1. Typos in file paths — certificate paths, kubeconfig paths, data directories
  2. Wrong port numbers — etcd port, API server port, scheduler port
  3. Missing or wrong volume mounts — the mount path doesn’t match the flag
  4. Wrong image tag — pointing to a non-existent version

Debugging Pattern

# Step 1: Check which manifests exist
ls /etc/kubernetes/manifests/

# Step 2: Check the specific manifest for errors
cat /etc/kubernetes/manifests/kube-apiserver.yaml

# Step 3: Check if referenced files exist
ls -la /etc/kubernetes/pki/

# Step 4: Fix the manifest (kubelet auto-restarts the Pod)
vi /etc/kubernetes/manifests/kube-apiserver.yaml

# Step 5: Wait and verify
crictl ps | grep apiserver
# Or after API server recovers:
kubectl get pods -n kube-system

Exam Tip: When editing static Pod manifests, use vi carefully. A syntax error in the YAML will prevent the Pod from starting. After saving, wait 15–30 seconds and verify with crictl ps.

Certificate Expiration

Kubernetes uses TLS certificates everywhere. If certificates expire, components can’t communicate.

Check Certificate Expiration

# Quick check with kubeadm
kubeadm certs check-expiration

# Manual check with openssl
openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text -noout | grep -A2 "Validity"

# Check all certificates in the PKI directory
for cert in /etc/kubernetes/pki/*.crt; do
  echo "=== $cert ==="
  openssl x509 -in "$cert" -noout -dates
done

Renew Certificates

# Renew all certificates
kubeadm certs renew all

# Restart control plane components to pick up new certs
# (kubelet auto-restarts static Pods if manifests are touched)

Sample Output

CERTIFICATE                EXPIRES                  RESIDUAL TIME
admin.conf                 Feb 09, 2026 06:30 UTC   364d
apiserver                  Feb 09, 2026 06:30 UTC   364d
apiserver-etcd-client      Feb 09, 2026 06:30 UTC   364d
apiserver-kubelet-client   Feb 09, 2026 06:30 UTC   364d
controller-manager.conf    Feb 09, 2026 06:30 UTC   364d
etcd-healthcheck-client    Feb 09, 2026 06:30 UTC   364d
etcd-peer                  Feb 09, 2026 06:30 UTC   364d
etcd-server                Feb 09, 2026 06:30 UTC   364d
front-proxy-client         Feb 09, 2026 06:30 UTC   364d
scheduler.conf             Feb 09, 2026 06:30 UTC   364d

Practice Questions

Q1. You cannot run any kubectl commands — all return “connection refused”. The cluster uses kubeadm. Diagnose and fix.

Show Answer 
# Step 1: kubectl is down, so use crictl directly on the control plane node
crictl ps -a | grep apiserver
# If the API server container exited or is not running:

# Step 2: Check API server logs
crictl logs <apiserver-container-id>

# Step 3: Check the static Pod manifest
cat /etc/kubernetes/manifests/kube-apiserver.yaml
# Look for typos in flags, wrong ports, wrong certificate paths

# Step 4: Fix the manifest
vi /etc/kubernetes/manifests/kube-apiserver.yaml

# Step 5: Wait for kubelet to restart the API server (15-30 seconds)
crictl ps | grep apiserver
kubectl get nodes  # should work now

Q2. Pods created by Deployments are stuck in Pending. The scheduler Pod in kube-system is in CrashLoopBackOff. Fix it.

Show Answer 
# Step 1: Check scheduler logs
kubectl logs -n kube-system kube-scheduler-controlplane

# Step 2: Check the scheduler manifest
cat /etc/kubernetes/manifests/kube-scheduler.yaml
# Common issue: wrong --kubeconfig path

# Step 3: Verify the kubeconfig file exists
ls /etc/kubernetes/scheduler.conf

# Step 4: Fix the manifest
vi /etc/kubernetes/manifests/kube-scheduler.yaml
# Correct the path or flag that's wrong

# Step 5: Wait and verify
kubectl get pods -n kube-system -l component=kube-scheduler
# Scheduler should be Running

kubectl get pods
# Previously Pending Pods should now be scheduling

Q3. You created a Deployment but no ReplicaSet or Pods were created. What’s likely wrong?

Show Answer 
# This is a controller manager issue — the Deployment controller creates ReplicaSets

# Step 1: Check the controller manager
kubectl get pods -n kube-system -l component=kube-controller-manager
kubectl logs -n kube-system kube-controller-manager-controlplane

# Step 2: Check the manifest
cat /etc/kubernetes/manifests/kube-controller-manager.yaml
# Look for typos in --kubeconfig, --root-ca-file, or other flags

# Step 3: Fix the manifest
vi /etc/kubernetes/manifests/kube-controller-manager.yaml

# Step 4: Verify
kubectl get pods -n kube-system -l component=kube-controller-manager
kubectl get rs  # ReplicaSet should now be created
kubectl get pods  # Pods should follow

Q4. The etcd Pod keeps crashing. The logs show “cannot access data directory”. Fix it.

Show Answer 
# Step 1: Check etcd logs
crictl ps -a | grep etcd
crictl logs <etcd-container-id>

# Step 2: Check the etcd manifest
cat /etc/kubernetes/manifests/etcd.yaml

# Step 3: Look for mismatch between --data-dir and volumeMounts
# The --data-dir flag must match the mountPath

# Step 4: Also check the hostPath volume
# volumes:
# - hostPath:
#     path: /var/lib/etcd
#   name: etcd-data

# Make sure:
# 1. --data-dir matches the volumeMount mountPath
# 2. The hostPath directory exists on disk

# Step 5: Fix and save
vi /etc/kubernetes/manifests/etcd.yaml

# Step 6: Verify
crictl ps | grep etcd
kubectl get pods -n kube-system -l component=etcd

Q5. Some cluster certificates have expired. Renew them and restore cluster functionality.

Show Answer 
# Step 1: Check which certificates have expired
kubeadm certs check-expiration

# Step 2: Renew all certificates
kubeadm certs renew all

# Step 3: Restart control plane components
# The easiest way is to move manifests out and back:
cd /etc/kubernetes/manifests/
mv *.yaml /tmp/
# Wait 10 seconds for Pods to stop
mv /tmp/kube-*.yaml /tmp/etcd.yaml .
# Wait 30 seconds for Pods to restart

# Step 4: Update kubeconfig (admin.conf was also renewed)
cp /etc/kubernetes/admin.conf ~/.kube/config

# Step 5: Verify
kubectl get nodes
kubectl get pods -n kube-system

What’s Next

Control plane issues can cascade into networking problems. Next, we’ll learn how to troubleshoot DNS, Services, NetworkPolicies, and kube-proxy.

Next: Troubleshoot Networking →

Updated:

You may also enjoy