ELK Stack — ELK Use Cases

5 minute read

ELK Stack Use Cases

The ELK Stack isn’t just a logging tool — it’s a versatile platform for searching, analyzing, and visualizing any kind of data. Let’s explore the real-world scenarios where organizations rely on ELK every day.

Use Case 1 — Centralized Log Management

This is the most common use case. Instead of SSH-ing into 50 servers to grep through logs, you send all logs to one central Elasticsearch cluster and search from Kibana.

The Problem:

Developer: "The API returned a 500 error at 2:15 PM"
Ops Team:  "Which server? We have 50."
Developer: "I don't know, the load balancer picked one."
Ops Team:  *SSH into all 50 servers, grep through logs*
            ... 45 minutes later ...
Ops Team:  "Found it on server 37."

With ELK:

Developer: "The API returned a 500 error at 2:15 PM"
Ops Team:  *Opens Kibana, types: status_code:500 AND @timestamp > "2025-03-15T14:15:00"*
            ... 2 seconds later ...
Ops Team:  "Found it. Server 37, line 4521, NullPointerException in UserService.java:142"

What you collect:

  • Application logs (Spring Boot, Django, Express, etc.)
  • Web server logs (NGINX, Apache, HAProxy)
  • Database logs (MySQL, PostgreSQL, MongoDB)
  • System logs (syslog, auth.log, kernel messages)
  • Container logs (Docker, Kubernetes pods)

Use Case 2 — Application Performance Monitoring (APM)

Track response times, error rates, and throughput across your applications.

Typical dashboard metrics:

Metric Example Query
Average response time Aggregation on response_time field
Error rate (5xx) Count where status_code >= 500 / total count
Top slowest endpoints Top 10 by response_time descending
Requests per second Date histogram with 1-second intervals
Error distribution Terms aggregation on error.type

Example scenario:

You deploy a new version of your API. Within minutes, Kibana shows the error rate jumped from 0.1% to 5%. You filter by status_code:500 and version:2.3.1 to see exactly which endpoints are failing and what error messages they’re returning.

Use Case 3 — Security Information and Event Management (SIEM)

Elastic SIEM (now part of Elastic Security) uses the ELK Stack to detect threats, investigate incidents, and respond to security events.

What you monitor:

┌──────────────────────────────────────────────┐
│              Security Data Sources            │
│                                               │
│  Firewall Logs       ──▶ Detect port scans   │
│  Auth Logs           ──▶ Detect brute force  │
│  DNS Logs            ──▶ Detect DNS tunneling│
│  Network Flow Logs   ──▶ Detect exfiltration │
│  Endpoint Logs       ──▶ Detect malware      │
│  Cloud Audit Logs    ──▶ Detect privilege    │
│                          escalation          │
└──────────────────────────────────────────────┘

Example detection rule:

“Alert if any single IP has more than 100 failed SSH login attempts within 5 minutes” — this is a classic brute-force detection that ELK can handle with a simple threshold alert in Kibana.

Use Case 4 — Infrastructure Monitoring

While Prometheus is great for metrics, ELK handles log-based infrastructure monitoring beautifully.

What you track:

  • Disk space warnings from syslog
  • Out-of-memory (OOM) kills from kernel logs
  • Service restart events from systemd
  • Network errors from interface logs
  • Certificate expiry warnings

Example dashboard:

Panel Data Source Visualization
OOM Kills Today syslog with “oom-killer” Metric (count)
Failed Services systemd with “Failed” Table
Disk Warnings syslog with “No space left” Timeline
SSH Login Map auth.log with GeoIP Coordinate Map

Use Case 5 — Business Analytics

ELK isn’t limited to technical logs. You can index any JSON data and analyze it.

Examples:

  • E-commerce: Track order events, search queries, product views. Build a dashboard showing popular products, search-to-purchase conversion, and regional sales trends.
  • SaaS platforms: Analyze user activity logs — logins, feature usage, API calls. Identify power users and features with low adoption.
  • Healthcare: Aggregate patient admission logs and track wait times, bed occupancy, and resource utilization.

Use Case 6 — Compliance and Audit Logging

Many regulations (HIPAA, PCI-DSS, SOC 2, GDPR) require you to retain and search audit logs.

ELK helps with:

  • Storing audit logs for required retention periods (1 year, 7 years, etc.)
  • Searching through historical logs for investigations
  • Generating compliance reports from Kibana dashboards
  • Alerting on policy violations in real-time
  • Index Lifecycle Management (ILM) to automatically archive and delete old data

Use Case 7 — Kubernetes Cluster Observability

For teams running Kubernetes, ELK is a natural fit for centralized logging.

Architecture:

┌──────────────────────────────────────────────┐
│           Kubernetes Cluster                  │
│                                               │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐      │
│  │  Pod A  │  │  Pod B  │  │  Pod C  │      │
│  │  (app)  │  │  (api)  │  │  (web)  │      │
│  └────┬────┘  └────┬────┘  └────┬────┘      │
│       │            │            │             │
│       ▼            ▼            ▼             │
│  ┌──────────────────────────────────────┐    │
│  │     Filebeat DaemonSet               │    │
│  │     (runs on every node)             │    │
│  │     reads: /var/log/containers/*.log │    │
│  └──────────────────┬───────────────────┘    │
└─────────────────────┼────────────────────────┘
                      │
                      ▼
              ┌───────────────┐
              │ Elasticsearch │──▶ Kibana
              └───────────────┘

You deploy Filebeat as a DaemonSet — it runs on every Kubernetes node and automatically collects container logs from all pods. Kubernetes metadata (pod name, namespace, labels) is added to every log event, making it easy to filter logs by deployment, service, or namespace.

Real-World Example — Debugging a Production Incident

Let’s walk through a realistic debugging scenario:

Situation: Users report that the checkout page is slow.

Step 1: Open Kibana → Discover → Filter: service:checkout AND response_time > 5000

Step 2: You see 500 slow requests in the last hour. Most come from the /checkout/payment endpoint.

Step 3: Drill down into a slow request:

{
  "service": "checkout",
  "endpoint": "/checkout/payment",
  "response_time": 8500,
  "trace_id": "abc-123-def",
  "error": "Connection timeout to payment-gateway:3000",
  "host": "checkout-pod-7f8b9"
}

Step 4: Search for the trace_id across all services: trace_id:abc-123-def

Step 5: You find the payment gateway service has a connection pool exhaustion error — it can’t handle the traffic spike.

Step 6: Fix: increase the connection pool size, deploy, and monitor in Kibana to confirm response times are back to normal.

Total debugging time: 5 minutes instead of hours.

Quick Reference

Use Case What ELK Does
Centralized Logging Collect all logs in one searchable place
APM Track response times, error rates, throughput
SIEM Detect security threats from log patterns
Infrastructure Monitor disk, memory, services via logs
Business Analytics Analyze user activity and business events
Compliance Retain and search audit logs
Kubernetes Centralize container logs with metadata

Next Step

You’ve seen what the ELK Stack can do. Now let’s explore all the components of the modern Elastic Stack in detail.

You may also enjoy