KCNA — Observability Fundamentals
Observability Fundamentals
Observability is the ability to understand a system’s internal state by examining its outputs. In cloud-native systems with many microservices, observability is essential for debugging, performance tuning, and reliability.
Observability vs Monitoring
| Aspect | Monitoring | Observability |
|---|---|---|
| Approach | Predefined checks — “Is the server up?” | Exploratory — “Why did latency spike at 3 PM?” |
| Questions | Known in advance | Unknown and novel |
| Scope | Dashboards and alerts for known issues | Understand any system behavior, including unknown issues |
| Data | Metrics and uptime checks | Metrics, logs, AND traces correlated together |
Monitoring tells you when something is wrong.
Observability helps you understand why it’s wrong.
The Three Pillars of Observability
Observability
┌─────────┼─────────┐
▼ ▼ ▼
Metrics Logs Traces
(numbers) (events) (request flow)
1. Metrics
Numerical measurements collected over time.
| Type | Example | Description |
|---|---|---|
| Counter | http_requests_total |
Always increases — total count of events |
| Gauge | memory_usage_bytes |
Goes up and down — current value |
| Histogram | request_duration_seconds |
Distribution of values in buckets |
| Summary | request_latency_p99 |
Pre-calculated percentiles |
Tool: Prometheus (CNCF Graduated)
2. Logs
Timestamped, text-based records of discrete events.
2025-02-09T10:15:30Z INFO [order-service] Order #12345 created for user alice
2025-02-09T10:15:31Z ERROR [payment-service] Payment failed: card declined for order #12345
| Property | Description |
|---|---|
| Structured | JSON format with consistent fields — easy to query |
| Unstructured | Free-text format — harder to parse and query |
Tools: Fluentd, Fluent Bit (CNCF Graduated)
3. Traces
Records of a request’s journey across multiple services.
User Request
│
├── API Gateway (2ms)
│ ├── Auth Service (5ms)
│ └── Order Service (50ms)
│ ├── Inventory Service (20ms)
│ └── Payment Service (25ms)
│ └── Bank API (15ms)
│
Total: 117ms
Each unit of work is called a span. The complete request path is a trace.
Tools: Jaeger (CNCF Graduated), OpenTelemetry (CNCF Incubating)
SLIs, SLOs, and SLAs
These terms define service reliability and are important for the KCNA exam.
SLI — Service Level Indicator
A measurable metric that indicates service health.
| SLI | Measurement |
|---|---|
| Availability | Percentage of successful requests |
| Latency | Time to process a request (e.g., p99 latency) |
| Throughput | Requests per second |
| Error Rate | Percentage of failed requests |
SLO — Service Level Objective
An internal target for an SLI.
| SLO | Target |
|---|---|
| Availability SLO | 99.9% uptime (allows ~8.7 hours downtime/year) |
| Latency SLO | p99 latency < 200ms |
| Error Rate SLO | Less than 0.1% error rate |
SLA — Service Level Agreement
A contractual agreement with customers, with consequences for violations.
| SLA | Agreement |
|---|---|
| Availability SLA | 99.9% uptime — if violated, customer gets credits |
| Response Time SLA | Support response within 4 hours |
Relationship
SLI (what you measure) → SLO (what you target) → SLA (what you promise)
- SLI: “Our p99 latency is currently 150ms”
- SLO: “We target p99 latency under 200ms”
- SLA: “We guarantee p99 latency under 500ms — or you get refunded”
SLOs are always tighter than SLAs. You aim for a higher standard internally.
Error Budgets
An error budget is the acceptable amount of unreliability before violating an SLO.
Error Budget = 1 - SLO
If SLO = 99.9% availability:
Error Budget = 0.1% = ~43 minutes/month of downtime
When the error budget is consumed:
- Stop feature work — focus on reliability
- Investigate — what caused the errors?
- Fix — address reliability issues before deploying more changes
This balances velocity (shipping features) with reliability (keeping services healthy).
Observability in Kubernetes
| K8s Feature | Observability Type |
|---|---|
| metrics-server | Basic CPU/memory metrics for HPA |
| Container logs (stdout/stderr) | Collected by logging agents |
| kube-state-metrics | Kubernetes object state as Prometheus metrics |
| Liveness/Readiness probes | Basic health monitoring |
| Events | Kubernetes resource lifecycle events |
KCNA Exam Tips
- Know the three pillars: Metrics, Logs, Traces
- Understand the difference between observability and monitoring
- Know the relationship: SLI → SLO → SLA (measure → target → promise)
- Error budgets balance feature velocity with reliability
- Know metric types: Counter, Gauge, Histogram, Summary
- Structured logs are preferred over unstructured in cloud-native systems
What’s Next?
Dive into Prometheus — the standard monitoring tool for Kubernetes.
