ELK Stack — Introduction Exercises

4 minute read

Practice Exercises — Introduction

Test your understanding of the ELK Stack concepts covered in this chapter. Try to answer each question before revealing the solution.

Exercise 1 — Name the Components

Question: What does each letter in “ELK” stand for, and what is each component’s primary role?

Show Solution
Letter Component Primary Role
E Elasticsearch Store, search, and analyze data (distributed search engine)
L Logstash Collect, transform, and route data (processing pipeline)
K Kibana Visualize, explore, and manage data (web UI)

The modern Elastic Stack also includes:

  • Beats — lightweight data shippers (Filebeat, Metricbeat, etc.)
  • Elastic Agent — unified data collection agent
  • Fleet — centralized agent management in Kibana

Exercise 2 — Trace the Data Flow

Question: Describe step by step what happens when an NGINX server writes a new line to its access log, assuming the full ELK Stack is configured (Filebeat → Logstash → Elasticsearch → Kibana).

Show Solution
  1. NGINX writes a log line to /var/log/nginx/access.log
  2. Filebeat (running on the NGINX server) detects the new line and reads it
  3. Filebeat sends the raw log event to Logstash on port 5044
  4. Logstash receives the event via the beats input plugin
  5. Logstash applies filters — grok parsing extracts fields (IP, status code, path, timestamp)
  6. Logstash sends the structured document to Elasticsearch via the elasticsearch output plugin
  7. Elasticsearch indexes the document into an index like nginx-logs-2025.03.15
  8. The document becomes searchable within ~1 second (near real-time)
  9. A user opens Kibana and searches for status_code:500 in the Discover view
  10. Kibana queries Elasticsearch and displays matching log events

Exercise 3 — Choose the Right Architecture

Question: You’re setting up logging for a startup with 5 servers. You only need to collect NGINX and application logs with no custom parsing required. Which architecture pattern would you recommend and why?

Show Solution

Recommended: Pattern 1 — Simple (Filebeat → Elasticsearch → Kibana)

Reasons:

  • Only 5 servers — no need for the complexity of Logstash
  • No custom parsing — Filebeat has built-in NGINX and application log modules that handle parsing automatically
  • Fewer moving parts — no Logstash to manage, monitor, or scale
  • Lower resource usage — Filebeat is lightweight (~30 MB RAM) vs Logstash (~500 MB RAM)
Server 1: [Filebeat] ──┐
Server 2: [Filebeat] ──┤
Server 3: [Filebeat] ──┼──▶ Elasticsearch ──▶ Kibana
Server 4: [Filebeat] ──┤
Server 5: [Filebeat] ──┘

You can always add Logstash later if your parsing needs grow.

Exercise 4 — ELK vs Alternatives

Question: Your company is choosing between ELK and Grafana Loki for centralized logging. The main requirements are: (a) fast full-text search across all logs, (b) aggregation queries to compute error rates per service, (c) budget is limited. Which would you recommend and why?

Show Solution

Recommended: ELK Stack

Reasons:

  • Requirement (a) — Fast full-text search: Elasticsearch indexes every word in every log line, making full-text search nearly instant. Grafana Loki does NOT index log content — it only indexes labels (pod name, namespace, etc.). Searching through log content in Loki requires scanning compressed chunks, which is significantly slower.

  • Requirement (b) — Aggregations: Elasticsearch has powerful aggregation capabilities — you can compute error rates, percentiles, histograms, and group by any field. Loki’s LogQL has limited metric capabilities in comparison.

  • Requirement (c) — Budget: Both are open source and free. However, ELK requires more storage (because it indexes everything) and more RAM (Elasticsearch needs memory for its JVM). Loki is cheaper on storage. The trade-off is search speed vs storage cost.

Verdict: Since fast search and aggregations are primary requirements, ELK is the better fit. The higher storage cost is worth the significantly faster search and richer analytics.

Exercise 5 — Design a Use Case

Question: A healthcare company needs to: (a) collect audit logs from 20 application servers, (b) retain logs for 7 years for HIPAA compliance, (c) search through historical logs during investigations, (d) alert on suspicious access patterns. Design the ELK architecture for this scenario, including which components you’d use and why.

Show Solution

Proposed Architecture:

┌──────────────────────────────────────────────────┐
│  20 Application Servers                           │
│  [Filebeat on each] ──▶ ships audit logs         │
└───────────────┬──────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────┐
│  Logstash (2 instances for HA)                    │
│  - Parse audit log format with grok              │
│  - Enrich with user details                      │
│  - Route to appropriate index                    │
└───────────────┬──────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────┐
│  Elasticsearch Cluster (3-node minimum)           │
│  - Hot-Warm-Cold architecture for cost savings   │
│  - ILM policy:                                    │
│    Hot:  0-30 days  (fast SSD storage)           │
│    Warm: 30-180 days (standard storage)          │
│    Cold: 180 days-7 years (cheap storage)        │
│  - Snapshot to S3 for disaster recovery          │
└───────────────┬──────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────┐
│  Kibana                                           │
│  - SIEM dashboards for suspicious access          │
│  - Alerting rules:                                │
│    • Alert on after-hours access                 │
│    • Alert on bulk data downloads                │
│    • Alert on failed authentication spikes       │
│  - Spaces: separate views for security vs ops    │
└──────────────────────────────────────────────────┘

Key design decisions:

  • Filebeat on each server — lightweight, reliable audit log shipping
  • Logstash for parsing — audit logs often have custom formats needing grok
  • 3-node ES cluster — minimum for high availability (data replication)
  • Hot-Warm-Cold ILM — keeps recent data on fast storage, archives old data on cheap storage to meet 7-year retention affordably
  • S3 snapshots — disaster recovery backup for compliance
  • Kibana Alerting — real-time detection of suspicious patterns
  • RBAC in Kibana — restrict who can access audit log data

What’s Next?

Great work completing the Introduction chapter! You now understand:

  • ✅ What the ELK Stack is and why it exists
  • ✅ How data flows through the architecture
  • ✅ Real-world use cases
  • ✅ All Elastic Stack components
  • ✅ How ELK compares to alternatives

Next up: Chapter 2 — Installation — where you’ll get the entire ELK Stack running on your machine.

You may also enjoy