CKA — Scheduling Lab

Lab Overview

In this hands-on lab you’ll apply everything from the Scheduling chapter. By the end, you’ll have practiced:

• Labeling nodes and using nodeSelector
• Configuring node affinity (hard and soft)
• Applying taints and tolerations
• Creating PriorityClasses and observing preemption
• Deploying DaemonSets
• Creating static Pods

Environment: These tasks assume a cluster with at least one control plane node and two worker nodes (worker-1, worker-2). Adjust node names to match your setup.

---

Task 1: nodeSelector

Objective: Label a node and deploy a Pod that uses nodeSelector to land on that node.

Step 1 — Label the node:

kubectl label nodes worker-1 disktype=ssd

Step 2 — Verify the label:

kubectl get nodes worker-1 --show-labels | grep disktype

Expected output should include disktype=ssd.

Step 3 — Create the Pod:

# task1-nodeselector.yaml
apiVersion: v1
kind: Pod
metadata:
  name: ssd-pod
spec:
  containers:
    - name: nginx
      image: nginx:1.27
  nodeSelector:
    disktype: ssd

kubectl apply -f task1-nodeselector.yaml

Step 4 — Verify placement:

kubectl get pod ssd-pod -o wide

Expected output:

NAME      READY   STATUS    RESTARTS   AGE   IP           NODE       ...
ssd-pod   1/1     Running   0          10s   10.244.1.5   worker-1   ...

The Pod should be running on worker-1.

Troubleshooting

If the Pod is Pending, check events:

kubectl describe pod ssd-pod

Common cause: No node has the disktype=ssd label. Verify with:

kubectl get nodes -l disktype=ssd

---

Task 2: Node Affinity

Objective: Create Pods with required and preferred node affinity rules.

Task 2a — Required (Hard) Affinity

Step 1 — Label nodes:

kubectl label nodes worker-1 zone=us-east-1a
kubectl label nodes worker-2 zone=us-east-1b

Step 2 — Create a Pod with required affinity:

# task2a-required-affinity.yaml
apiVersion: v1
kind: Pod
metadata:
  name: affinity-required
spec:
  containers:
    - name: nginx
      image: nginx:1.27
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: zone
                operator: In
                values:
                  - us-east-1a

kubectl apply -f task2a-required-affinity.yaml
kubectl get pod affinity-required -o wide

The Pod must land on worker-1 (zone=us-east-1a).

Task 2b — Preferred (Soft) Affinity

# task2b-preferred-affinity.yaml
apiVersion: v1
kind: Pod
metadata:
  name: affinity-preferred
spec:
  containers:
    - name: nginx
      image: nginx:1.27
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          preference:
            matchExpressions:
              - key: zone
                operator: In
                values:
                  - us-west-2a

kubectl apply -f task2b-preferred-affinity.yaml
kubectl get pod affinity-preferred -o wide

Since no node has zone=us-west-2a, the scheduler assigns the Pod to any available node — the soft preference is honored on a best-effort basis.

Verify

kubectl describe pod affinity-preferred | grep -A 5 "Node-Selectors\|Tolerations\|Events"

The Pod should be Running on one of the worker nodes, even without a matching label.

---

Task 3: Taints & Tolerations

Objective: Taint a node, observe that a Pod without a toleration stays Pending, then add a toleration so it gets scheduled.

Step 1 — Taint the node:

kubectl taint nodes worker-2 env=prod:NoSchedule

Step 2 — Verify the taint:

kubectl describe node worker-2 | grep Taints

Expected: Taints: env=prod:NoSchedule

Step 3 — Deploy a Pod without a toleration:

# task3-no-toleration.yaml
apiVersion: v1
kind: Pod
metadata:
  name: no-toleration-pod
spec:
  containers:
    - name: nginx
      image: nginx:1.27
  nodeSelector:
    zone: us-east-1b

kubectl apply -f task3-no-toleration.yaml
kubectl get pod no-toleration-pod

Expected: The Pod is Pending because worker-2 is tainted and the Pod has no toleration.

kubectl describe pod no-toleration-pod | grep -A 3 Events

Step 4 — Deploy a Pod with a toleration:

# task3-with-toleration.yaml
apiVersion: v1
kind: Pod
metadata:
  name: toleration-pod
spec:
  containers:
    - name: nginx
      image: nginx:1.27
  nodeSelector:
    zone: us-east-1b
  tolerations:
    - key: "env"
      operator: "Equal"
      value: "prod"
      effect: "NoSchedule"

kubectl apply -f task3-with-toleration.yaml
kubectl get pod toleration-pod -o wide

Expected: The Pod is Running on worker-2.

Understanding the Difference

• no-toleration-pod — Pending because nodeSelector targets worker-2, but the taint blocks it.
• toleration-pod — Running because it both targets worker-2 (nodeSelector) and tolerates the taint.

---

Task 4: Pod Priority

Objective: Create two PriorityClasses and observe scheduling priority.

Step 1 — Create PriorityClasses:

# task4-priority-classes.yaml
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: high-priority
value: 1000000
globalDefault: false
description: "High priority for critical workloads."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: low-priority
value: 100
globalDefault: false
description: "Low priority for background tasks."

kubectl apply -f task4-priority-classes.yaml
kubectl get priorityclasses

Step 2 — Deploy Pods with different priorities:

# task4-low-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: low-pod
spec:
  priorityClassName: low-priority
  containers:
    - name: nginx
      image: nginx:1.27

# task4-high-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: high-pod
spec:
  priorityClassName: high-priority
  containers:
    - name: nginx
      image: nginx:1.27

kubectl apply -f task4-low-pod.yaml
kubectl apply -f task4-high-pod.yaml

Step 3 — Verify priorities:

kubectl describe pod low-pod | grep Priority
kubectl describe pod high-pod | grep Priority

Expected output:

Priority:             100
Priority Class Name:  low-priority

Priority:             1000000
Priority Class Name:  high-priority

Preemption in Action

To observe preemption, you’d need a resource-constrained cluster where the high-priority Pod can’t be scheduled without evicting the low-priority one. In a lab with ample resources, both Pods run without preemption.

To simulate, you can set tight resource requests and limits on a single-node setup.

---

Task 5: DaemonSet

Objective: Create a DaemonSet, verify it runs on all worker nodes, then limit it with nodeSelector.

Step 1 — Create a DaemonSet:

# task5-daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: monitor-agent
spec:
  selector:
    matchLabels:
      app: monitor
  template:
    metadata:
      labels:
        app: monitor
    spec:
      containers:
        - name: agent
          image: busybox:1.36
          command: ["sleep", "infinity"]
      tolerations:
        - operator: "Exists"

kubectl apply -f task5-daemonset.yaml

Step 2 — Verify it runs on all nodes:

kubectl get pods -l app=monitor -o wide

Expected: One Pod on each node (including the control plane, due to the toleration).

NAME                  READY   STATUS    NODE
monitor-agent-abc12   1/1     Running   control-plane
monitor-agent-def34   1/1     Running   worker-1
monitor-agent-ghi56   1/1     Running   worker-2

Step 3 — Limit with nodeSelector:

Update the DaemonSet to only run on worker nodes:

kubectl edit daemonset monitor-agent

Add under .spec.template.spec:

nodeSelector:
  node-role.kubernetes.io/worker: ""

Or, label your worker nodes first:

kubectl label nodes worker-1 node-role.kubernetes.io/worker=""
kubectl label nodes worker-2 node-role.kubernetes.io/worker=""

Step 4 — Verify:

kubectl get pods -l app=monitor -o wide

The control plane Pod should be terminating, leaving only worker node Pods.

DaemonSet Status

kubectl get daemonset monitor-agent

NAME            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                        AGE
monitor-agent   2         2         2       2            2           node-role.kubernetes.io/worker=       60s

---

Task 6: Static Pod

Objective: Create a static Pod manifest on a worker node and verify it appears in kubectl.

Step 1 — SSH to a worker node:

ssh worker-1

Step 2 — Verify the static Pod path:

cat /var/lib/kubelet/config.yaml | grep staticPodPath

Expected: staticPodPath: /etc/kubernetes/manifests

Step 3 — Create a static Pod manifest:

cat <<EOF > /etc/kubernetes/manifests/static-web.yaml
apiVersion: v1
kind: Pod
metadata:
  name: static-web
spec:
  containers:
    - name: nginx
      image: nginx:1.27
      ports:
        - containerPort: 80
EOF

Step 4 — Verify from the control plane:

# Exit back to control plane or open another terminal
kubectl get pods -A | grep static-web

Expected output:

default       static-web-worker-1   1/1     Running   0          15s

The Pod name has the node name appended (static-web-worker-1).

Step 5 — Try deleting the mirror Pod:

kubectl delete pod static-web-worker-1
kubectl get pods | grep static-web

The kubelet recreates it immediately.

Cleanup the Static Pod

SSH to the node and remove the manifest:

ssh worker-1
rm /etc/kubernetes/manifests/static-web.yaml

Verify:

kubectl get pods | grep static-web
# Should be gone after a few seconds

---

Cleanup

Remove all resources and configurations created during this lab:

# Delete Pods
kubectl delete pod ssd-pod affinity-required affinity-preferred \
  no-toleration-pod toleration-pod low-pod high-pod --ignore-not-found

# Delete DaemonSet
kubectl delete daemonset monitor-agent --ignore-not-found

# Delete PriorityClasses
kubectl delete priorityclass high-priority low-priority --ignore-not-found

# Remove taints
kubectl taint nodes worker-2 env=prod:NoSchedule- 2>/dev/null || true

# Remove labels
kubectl label nodes worker-1 disktype- zone- node-role.kubernetes.io/worker- 2>/dev/null || true
kubectl label nodes worker-2 zone- node-role.kubernetes.io/worker- 2>/dev/null || true

# Remove static Pod (on the node)
# ssh worker-1 "rm -f /etc/kubernetes/manifests/static-web.yaml"

Verify everything is cleaned up:

kubectl get pods
kubectl get daemonsets
kubectl get priorityclasses | grep -v system
kubectl describe node worker-1 | grep Taints
kubectl describe node worker-2 | grep Taints

You’ve completed the Scheduling lab. Head back to the Scheduling hub to review the concepts.