Kubernetes On-Premises — Introduction
Welcome to the On-Premises Kubernetes Tutorial
Running Kubernetes on-premises means you manage everything — hardware, networking, storage, the control plane, and worker nodes. There’s no cloud provider to manage infrastructure for you. This tutorial covers building production-grade self-managed Kubernetes clusters.
Why On-Premises?
| Reason | Description |
|---|---|
| Data sovereignty | Data must stay in your data center (regulatory/compliance) |
| Cost at scale | Large clusters can be cheaper on owned hardware |
| Low latency | Edge computing, manufacturing, IoT workloads |
| Security isolation | Air-gapped or restricted network environments |
| Existing infrastructure | Leverage existing bare-metal or VMware investments |
On-Prem vs Cloud Managed Kubernetes
| Aspect | On-Premises | Cloud Managed (AKS/GKE/EKS) |
|---|---|---|
| Control plane | You manage | Provider manages |
| Hardware | You provision | Provider provisions |
| Networking | You configure (CNI, LB, DNS) | Integrated cloud networking |
| Storage | You set up (NFS, Ceph, etc.) | Cloud storage classes |
| Upgrades | Manual with kubeadm | Automated/assisted |
| HA | You architect | Built-in options |
| Cost model | CapEx + OpEx | OpEx (pay-as-you-go) |
Architecture Overview
┌──────────────┐
│ Load Balancer│
│ (HAProxy/ │
│ keepalived) │
└──────┬───────┘
┌───────────────┼───────────────┐
│ │ │
┌──────┴──────┐ ┌──────┴──────┐ ┌──────┴──────┐
│ Control │ │ Control │ │ Control │
│ Plane 1 │ │ Plane 2 │ │ Plane 3 │
│ API+etcd │ │ API+etcd │ │ API+etcd │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
┌──────┴──────┐ ┌──────┴──────┐ ┌──────┴──────┐
│ Worker 1 │ │ Worker 2 │ │ Worker 3 │
│ kubelet │ │ kubelet │ │ kubelet │
│ Pods │ │ Pods │ │ Pods │
└─────────────┘ └─────────────┘ └─────────────┘
Prerequisites
Before starting this tutorial, you should have:
- Completed the Kubernetes Foundation Tutorial and ideally the CKA Tutorial
- At least 3 Linux machines (VMs or bare-metal) with:
- 2 CPU cores, 4 GB RAM minimum per node
- Ubuntu 22.04 LTS or RHEL 8/9
- Network connectivity between all nodes
- Basic Linux administration skills (systemd, networking, firewall)
- Root/sudo access on all nodes
Key Decisions
Before building your cluster, decide on:
| Decision | Options |
|---|---|
| Installation tool | kubeadm (standard), k3s (lightweight), Kubespray (Ansible) |
| CNI plugin | Calico (policy + routing), Flannel (simple), Cilium (eBPF-based) |
| Storage | NFS, Ceph (Rook), Longhorn, local-path-provisioner |
| Load balancer | MetalLB (bare-metal LB), HAProxy + keepalived |
| Ingress controller | NGINX, Traefik, HAProxy |
| Container runtime | containerd (recommended), CRI-O |
What You’ll Learn
- Plan infrastructure for production on-prem Kubernetes
- Install a multi-node cluster with kubeadm
- Choose and deploy a CNI plugin (Calico, Flannel, Cilium)
- Set up persistent storage (NFS, Ceph/Rook, Longhorn)
- Build a highly available control plane
- Deploy monitoring with Prometheus + Grafana
- Implement backup and disaster recovery with Velero
Practice Questions
Q1: What are the main differences between running Kubernetes on-premises vs using a managed service?
Show Answer
On-premises, you are responsible for: - **Hardware provisioning** and maintenance - **Control plane management** (API server, etcd, scheduler) - **Networking** (CNI plugin, load balancer, DNS) - **Storage** (NFS, Ceph, or other storage backends) - **Upgrades** (manual kubeadm upgrade process) - **High availability** (designing and implementing HA) With managed services (AKS, GKE, EKS), the cloud provider handles the control plane, integrates networking/storage, and simplifies upgrades.Q2: Name three CNI plugins and their key characteristics.
