Health Checks — Liveness, Readiness, and Startup Probes

How does Kubernetes know if your application is healthy? Or if it’s ready to receive traffic? Or if it’s still starting up? Probes are Kubernetes’ built-in health check mechanism that ensures only healthy containers serve traffic and unhealthy ones are automatically restarted.

The Three Types of Probes

Probe	Question It Answers	Action on Failure
Liveness	“Is the container still alive?”	Restart the container
Readiness	“Is the container ready for traffic?”	Remove from Service endpoints
Startup	“Has the container finished starting up?”	Restart the container

Container Start
      │
      ▼
┌─────────────┐    Failed    ┌───────────┐
│   Startup   │─────────────►│  Restart   │
│   Probe     │              │ Container  │
└──────┬──────┘              └───────────┘
       │ Passed
       ▼
┌─────────────┐    Failed    ┌───────────┐
│  Liveness   │─────────────►│  Restart   │
│  Probe      │              │ Container  │
└──────┬──────┘              └───────────┘
       │ Running
       ▼
┌─────────────┐    Failed    ┌──────────────────┐
│  Readiness  │─────────────►│ Remove from      │
│  Probe      │              │ Service endpoints │
└─────────────┘              └──────────────────┘

Probe Methods

Kubernetes supports three probe methods:

1. HTTP GET

Sends an HTTP GET request. Success = 200-399 status code.

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
    httpHeaders:
    - name: Custom-Header
      value: probe-check
  initialDelaySeconds: 10
  periodSeconds: 15
  timeoutSeconds: 3
  failureThreshold: 3

2. TCP Socket

Tries to open a TCP connection. Success = connection established.

livenessProbe:
  tcpSocket:
    port: 3306
  initialDelaySeconds: 15
  periodSeconds: 20

3. Command Execution

Runs a command inside the container. Success = exit code 0.

livenessProbe:
  exec:
    command:
    - cat
    - /tmp/healthy
  initialDelaySeconds: 5
  periodSeconds: 10

Probe Configuration Parameters

Parameter	Default	Description
initialDelaySeconds	0	Wait before first probe
periodSeconds	10	Time between probes
timeoutSeconds	1	Probe timeout
successThreshold	1	Consecutive successes needed (after failure)
failureThreshold	3	Consecutive failures before action

Liveness Probe

Purpose: Detect and restart containers that are stuck (deadlocked, frozen, unresponsive).

apiVersion: v1
kind: Pod
metadata:
  name: liveness-demo
spec:
  containers:
  - name: app
    image: busybox
    command: ["sh", "-c", "touch /tmp/healthy; sleep 30; rm /tmp/healthy; sleep 600"]
    livenessProbe:
      exec:
        command: ["cat", "/tmp/healthy"]
      initialDelaySeconds: 5
      periodSeconds: 5

What happens:

1. Container starts, creates /tmp/healthy
2. Liveness probe checks every 5s — passes (cat /tmp/healthy succeeds)
3. After 30s, /tmp/healthy is deleted
4. Liveness probe fails 3 times (failureThreshold)
5. Kubernetes restarts the container
6. Cycle repeats

kubectl apply -f liveness-demo.yaml
kubectl describe pod liveness-demo
# Events show: Liveness probe failed → Container restarted

kubectl get pod liveness-demo
# RESTARTS column increases

Readiness Probe

Purpose: Control when a container is ready to receive traffic. Unlike liveness, failing a readiness probe does NOT restart the container — it just removes it from the Service endpoints.

apiVersion: v1
kind: Pod
metadata:
  name: readiness-demo
  labels:
    app: readiness-demo
spec:
  containers:
  - name: app
    image: nginx:1.27
    ports:
    - containerPort: 80
    readinessProbe:
      httpGet:
        path: /
        port: 80
      initialDelaySeconds: 5
      periodSeconds: 10

Use cases for readiness probes:

• Application is loading data into cache
• Application is waiting for a database connection
• Application is performing startup initialization
• Application is under heavy load and can’t accept more traffic

Startup Probe

Purpose: Give slow-starting containers time to initialize before liveness probes start. Without a startup probe, a slow application might be killed by the liveness probe before it finishes starting.

apiVersion: v1
kind: Pod
metadata:
  name: slow-start-app
spec:
  containers:
  - name: app
    image: my-heavy-app:1.0
    ports:
    - containerPort: 8080
    startupProbe:
      httpGet:
        path: /healthz
        port: 8080
      failureThreshold: 30     # 30 × 10s = 300s (5 minutes) to start
      periodSeconds: 10
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      periodSeconds: 15
    readinessProbe:
      httpGet:
        path: /ready
        port: 8080
      periodSeconds: 10

Flow:

1. Startup probe runs first — checks every 10s, up to 30 failures (5 minutes max)
2. Once startup probe passes, it’s disabled permanently for this container
3. Liveness and readiness probes take over

Complete Production Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web-api
  template:
    metadata:
      labels:
        app: web-api
    spec:
      containers:
      - name: api
        image: my-api:1.0
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: "250m"
            memory: "256Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"

        # Startup probe: Wait up to 2 minutes for app to start
        startupProbe:
          httpGet:
            path: /health/startup
            port: 8080
          failureThreshold: 12
          periodSeconds: 10

        # Liveness probe: Restart if unhealthy for 45 seconds
        livenessProbe:
          httpGet:
            path: /health/live
            port: 8080
          initialDelaySeconds: 0
          periodSeconds: 15
          timeoutSeconds: 5
          failureThreshold: 3

        # Readiness probe: Remove from traffic if not ready
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8080
          initialDelaySeconds: 0
          periodSeconds: 10
          timeoutSeconds: 3
          failureThreshold: 3
          successThreshold: 1

Typical Health Endpoints

# Example health endpoints (Python/Flask)
@app.route('/health/startup')
def startup():
    """Check if app has finished initialization."""
    if app_initialized:
        return 'OK', 200
    return 'Starting', 503

@app.route('/health/live')
def liveness():
    """Check if app is alive (not deadlocked)."""
    return 'OK', 200

@app.route('/health/ready')
def readiness():
    """Check if app can handle traffic."""
    if db_connected and cache_loaded:
        return 'OK', 200
    return 'Not Ready', 503

Hands-On: Probe Behavior

# probe-demo.yaml
apiVersion: v1
kind: Pod
metadata:
  name: probe-demo
  labels:
    app: probe-demo
spec:
  containers:
  - name: app
    image: nginx:1.27
    ports:
    - containerPort: 80
    livenessProbe:
      httpGet:
        path: /healthz
        port: 80
      initialDelaySeconds: 5
      periodSeconds: 5
      failureThreshold: 3
    readinessProbe:
      httpGet:
        path: /ready
        port: 80
      initialDelaySeconds: 5
      periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: probe-demo
spec:
  selector:
    app: probe-demo
  ports:
  - port: 80

kubectl apply -f probe-demo.yaml

# Both /healthz and /ready return 404 from nginx (not 200-399)
# Readiness fails → Pod removed from Service endpoints
# Liveness fails → Pod restarted

# Watch events
kubectl describe pod probe-demo

# Fix: Create the health endpoints
kubectl exec probe-demo -- sh -c 'echo ok > /usr/share/nginx/html/healthz'
kubectl exec probe-demo -- sh -c 'echo ok > /usr/share/nginx/html/ready'

# Now probes pass, Pod becomes Ready
kubectl get pods probe-demo

Common Pitfalls

Pitfall	Problem	Fix
Liveness probe too aggressive	Pod restart loop	Increase initialDelaySeconds and failureThreshold
No readiness probe	Traffic sent to starting Pods	Always set readiness probes
Same endpoint for liveness and readiness	Can’t distinguish “alive” from “ready”	Use separate endpoints
Missing startup probe for slow apps	Killed before initialization	Add startup probe
Probe timeout too short	False failures under load	Increase timeoutSeconds

Best Practices

1. Always set readiness probes — Prevents traffic to unready Pods
2. Use startup probes for slow applications — Protect against premature liveness kills
3. Keep liveness probes simple — Just check “is the process alive?”
4. Make readiness probes meaningful — Check database, cache, dependencies
5. Don’t use the same endpoint for liveness and readiness
6. Be generous with timeouts — Network and IO latency can cause false failures

Summary

Probe	Checks	On Failure
Startup	“Finished starting?”	Restart (blocks liveness/readiness)
Liveness	“Still alive?”	Restart container
Readiness	“Ready for traffic?”	Remove from Service

• Use HTTP, TCP, or exec probe methods
• Configure initialDelaySeconds, periodSeconds, failureThreshold appropriately
• Always set readiness probes in production
• Use startup probes for applications with long initialization times

What’s Next?

Now let’s learn about Helm — the package manager for Kubernetes that simplifies deploying complex applications with reusable, configurable charts.

Next: Helm Package Manager →