ELK Stack — Kibana Advanced Exercises

Introduction

These five practice exercises cover the advanced Kibana features from this chapter — data views with runtime fields, Canvas workpads, alerting rules, Spaces management, and RBAC configuration. Each exercise increases in complexity and mirrors real-world production scenarios.

Prerequisites:

• Running Elasticsearch 8.x and Kibana instance
• Kibana sample data loaded (web logs, ecommerce, flights) — load via Home → Try sample data
• Admin access to Kibana

---

Exercise 1 — Data View with Runtime Fields

Difficulty: ⭐⭐ Intermediate

Scenario

Your team ingests web server logs into kibana_sample_data_logs. The bytes field stores response size as a raw number. Product managers want to see response sizes categorized as Small (< 5000 bytes), Medium (5000–10000 bytes), and Large (> 10000 bytes) without reindexing.

Requirements

1. Create a data view for kibana_sample_data_logs
2. Add a runtime field called response_size_category of type Keyword
3. The runtime field should classify documents based on the bytes field
4. Verify the runtime field appears in Discover and can be used in a bar chart visualization

Show Solution

Step 1 — Create the Data View

1. Open Stack Management → Data Views
2. Click Create data view
3. Name: Web Logs
4. Index pattern: kibana_sample_data_logs
5. Timestamp field: @timestamp
6. Click Save data view to Kibana

Step 2 — Add the Runtime Field

1. In the data view editor, click Add field
2. Name: response_size_category
3. Type: Keyword
4. Toggle Set value on
5. Enter the Painless script:

if (doc['bytes'].size() > 0) {
  long b = doc['bytes'].value;
  if (b < 5000) emit('Small');
  else if (b <= 10000) emit('Medium');
  else emit('Large');
}

1. Click Save

Step 3 — Verify in Discover

1. Open Discover
2. Select the Web Logs data view
3. Add the response_size_category column
4. Verify you see Small, Medium, and Large values

Step 4 — Create a Visualization

1. Open Visualize Library → Create visualization → Lens
2. Select the Web Logs data view
3. Drag response_size_category to the horizontal axis
4. The vertical axis defaults to Count
5. Choose Bar vertical chart type
6. Save as Response Size Distribution

---

Exercise 2 — Build a Canvas Workpad

Difficulty: ⭐⭐ Intermediate

Scenario

The VP of Engineering wants a live status board displayed on a monitor in the office. It should show key metrics from the sample web logs data — total requests, error rate, top 5 countries by traffic, and a time-series chart of requests over time.

Requirements

1. Create a Canvas workpad named Engineering Status Board with a 1920×1080 layout
2. Add a metric element showing total request count (last 24 hours)
3. Add a metric element showing error rate (percentage of 5xx responses)
4. Add a data table showing top 5 countries by request count
5. Set auto-refresh to 30 seconds
6. Use a dark background color

Show Solution

Step 1 — Create the Workpad

1. Open Canvas from the side navigation
2. Click Create workpad
3. Name: Engineering Status Board
4. Click the settings gear icon
5. Set page size to 1920 × 1080
6. Set background color to #1a1a2e

Step 2 — Total Requests Metric

1. Click Add element → Metric
2. Click the element, then open the Expression editor (bottom of screen)
3. Replace the expression with:

filters
| essql query="SELECT COUNT(*) AS total FROM \"kibana_sample_data_logs\" WHERE @timestamp > NOW() - INTERVAL 24 HOUR"
| math "total"
| metric "Total Requests (24h)"
  metricFont={font size=48 family="'Open Sans', Helvetica, Arial, sans-serif" color="#00d4ff" align="center"}
  labelFont={font size=14 family="'Open Sans', Helvetica, Arial, sans-serif" color="#AAAAAA" align="center"}
| render

1. Position in the top-left area of the workpad

Step 3 — Error Rate Metric

1. Click Add element → Metric
2. Open the Expression editor and use:

filters
| essql query="SELECT ROUND(CAST(SUM(CASE WHEN response >= 500 THEN 1 ELSE 0 END) AS DOUBLE) / COUNT(*) * 100, 2) AS error_pct FROM \"kibana_sample_data_logs\" WHERE @timestamp > NOW() - INTERVAL 24 HOUR"
| math "error_pct"
| formatnumber "0.00"
| metric "Error Rate %"
  metricFont={font size=48 family="'Open Sans', Helvetica, Arial, sans-serif" color="#FF4136" align="center"}
  labelFont={font size=14 family="'Open Sans', Helvetica, Arial, sans-serif" color="#AAAAAA" align="center"}
| render

1. Position next to the total requests metric

Step 4 — Top 5 Countries Table

1. Click Add element → Data table
2. Open the Expression editor:

filters
| essql query="SELECT geo.src AS country, COUNT(*) AS requests FROM \"kibana_sample_data_logs\" WHERE @timestamp > NOW() - INTERVAL 24 HOUR GROUP BY geo.src ORDER BY requests DESC LIMIT 5"
| table
| render

1. Style the table with light text on dark background

Step 5 — Set Auto-Refresh

1. Click the clock icon in the toolbar
2. Set refresh interval to 30 seconds
3. Click Full screen to test the display

---

Exercise 3 — Set Up Alerting Rules

Difficulty: ⭐⭐⭐ Advanced

Scenario

The SRE team needs to be alerted when the web logs show more than 20 requests with HTTP status code 503 (Service Unavailable) within a 5-minute window. The alert should post to a Slack channel and also write alert data to an Elasticsearch index for historical tracking.

Requirements

1. Create a Server log connector for testing (since Slack requires external setup)
2. Create an Index connector that writes to alert-history
3. Create an Index threshold rule:
   • Name: 503 Service Unavailable Spike
   • Check every: 1 minute
   • Index: kibana_sample_data_logs
   • Condition: count of documents where response is 503, is above 20 in the last 5 minutes
4. Add two actions:
   • Server log action on Active status
   • Index action on Active and Recovered status
5. Verify the rule appears in the rules list with correct configuration

Show Solution

Step 1 — Create the Server Log Connector

1. Open Stack Management → Connectors
2. Click Create connector
3. Select Server log
4. Name: SRE Alert Log
5. Click Save

Step 2 — Create the Index Connector

1. Click Create connector
2. Select Index
3. Name: Alert History
4. Index: alert-history
5. Check Define time field for each document → field name: @timestamp
6. Click Save

Step 3 — Create the Alert Rule

1. Open Stack Management → Rules
2. Click Create rule
3. Name: 503 Service Unavailable Spike
4. Tags: production, availability
5. Check every: 1 minute

Step 4 — Configure the Condition

1. Select rule type: Index threshold
2. Index: kibana_sample_data_logs
3. Timestamp field: @timestamp
4. Configure:
   • WHEN count()
   • OF all documents (with KQL filter: response: 503)
   • IS ABOVE 20
   • FOR THE LAST 5 minutes

Step 5 — Add the Server Log Action

1. Click Add action → Server log
2. Select the SRE Alert Log connector
3. Run when: Alert is active
4. Message:

🚨 503 SPIKE DETECTED
Value: {{context.value}} occurrences in last 5 minutes
Threshold: {{context.threshold}}
Time: {{context.date}}

Step 6 — Add the Index Action

1. Click Add action → Index
2. Select the Alert History connector
3. Run when: Alert is active
4. Document:

{
  "rule_name": "503 Service Unavailable Spike",
  "status": "active",
  "value": "",
  "threshold": "",
  "timestamp": ""
}

1. Add another Index action for Recovered status:

{
  "rule_name": "503 Service Unavailable Spike",
  "status": "recovered",
  "timestamp": ""
}

Step 7 — Save and Verify

1. Click Save
2. Verify the rule appears in Stack Management → Rules
3. Check the rule status — it should show as OK or Active depending on current data

---

Exercise 4 — Configure Spaces and Copy Objects

Difficulty: ⭐⭐⭐ Advanced

Scenario

Your organization has two teams — Platform Engineering and Business Analytics. Each team needs its own Kibana Space with only the features they use. A shared dashboard [Logs] Web Traffic should be available in both Spaces.

Requirements

1. Create a Space called Platform Engineering (ID: platform) with features: Discover, Dashboard, Alerting, Dev Tools
2. Create a Space called Business Analytics (ID: analytics) with features: Dashboard, Canvas, Visualize
3. Copy the [Logs] Web Traffic dashboard from the Default Space to both new Spaces
4. Verify the dashboard and its dependencies appear in each Space

Show Solution

Step 1 — Create the Platform Engineering Space

1. Open Stack Management → Spaces
2. Click Create a space
3. Name: Platform Engineering
4. URL identifier: platform
5. Description: Infrastructure monitoring and log analysis
6. Color: Choose a blue shade
7. Under Customize features for this space:
   • Enable: Discover, Dashboard, Alerting, Dev Tools
   • Disable: Canvas, Maps, Machine Learning, Visualize, Graph
8. Click Create space

Step 2 — Create the Business Analytics Space

1. Click Create a space
2. Name: Business Analytics
3. URL identifier: analytics
4. Description: Business KPI dashboards and presentations
5. Color: Choose a green shade
6. Under Customize features:
   • Enable: Dashboard, Canvas, Visualize
   • Disable: Discover, Alerting, Dev Tools, Maps, Machine Learning, Graph
7. Click Create space

Step 3 — Copy the Dashboard

1. Make sure you are in the Default space
2. Open Stack Management → Saved Objects
3. Search for [Logs] Web Traffic
4. Select the dashboard checkbox
5. Click Copy to space
6. Select both Platform Engineering and Business Analytics
7. Enable Include related objects (this copies dependent visualizations and data views)
8. Click Copy

Step 4 — Verify

1. Switch to the Platform Engineering space:
   • Click the space selector (top-left Kibana logo area) → Platform Engineering
   • Or navigate to: https://localhost:5601/s/platform/app/dashboards
2. Open Dashboards → verify [Logs] Web Traffic is present
3. Open the dashboard → verify all visualizations render
4. Repeat for the Business Analytics space

Verify via API

# List objects in the platform space
curl -X GET "https://localhost:5601/s/platform/api/saved_objects/_find?type=dashboard" \
  -H "kbn-xsrf: true" \
  -u elastic:changeme

# List objects in the analytics space
curl -X GET "https://localhost:5601/s/analytics/api/saved_objects/_find?type=dashboard" \
  -H "kbn-xsrf: true" \
  -u elastic:changeme

---

Exercise 5 — Set Up RBAC with Custom Roles

Difficulty: ⭐⭐⭐⭐ Expert

Scenario

Using the Spaces created in Exercise 4, implement the following access control:

• Platform engineers should have full read-write access to the platform Space and read-only access to logs-* and metrics-* indices. They must NOT see the user.email field (PII protection).
• Business analysts should have read-only access to the analytics Space and read access to kibana_sample_data_ecommerce index only.
• A test user for each role should be created and verified.

Requirements

1. Create a custom role platform-engineer with:
   • Cluster privilege: monitor
   • Index privileges: read, view_index_metadata on logs-*, metrics-*, kibana_sample_data_logs
   • Field-level security: exclude user.email
   • Kibana privileges: all on features in the platform Space
2. Create a custom role business-analyst with:
   • No cluster privileges
   • Index privileges: read, view_index_metadata on kibana_sample_data_ecommerce
   • Kibana privileges: read on features in the analytics Space
3. Create user platform-user with role platform-engineer
4. Create user analyst-user with role business-analyst
5. Verify access by logging in as each user

Show Solution

Step 1 — Create the platform-engineer Role

curl -X PUT "https://localhost:9200/_security/role/platform-engineer" \
  -H "Content-Type: application/json" \
  -u elastic:changeme \
  -d '{
    "cluster": ["monitor"],
    "indices": [
      {
        "names": ["logs-*", "metrics-*", "kibana_sample_data_logs"],
        "privileges": ["read", "view_index_metadata"],
        "field_security": {
          "grant": ["*"],
          "except": ["user.email"]
        }
      }
    ],
    "applications": [
      {
        "application": "kibana-.kibana",
        "privileges": [
          "feature_discover.all",
          "feature_dashboard.all",
          "feature_alerting.all",
          "feature_dev_tools.all"
        ],
        "resources": ["space:platform"]
      }
    ]
  }'

Step 2 — Create the business-analyst Role

curl -X PUT "https://localhost:9200/_security/role/business-analyst" \
  -H "Content-Type: application/json" \
  -u elastic:changeme \
  -d '{
    "cluster": [],
    "indices": [
      {
        "names": ["kibana_sample_data_ecommerce"],
        "privileges": ["read", "view_index_metadata"]
      }
    ],
    "applications": [
      {
        "application": "kibana-.kibana",
        "privileges": [
          "feature_dashboard.read",
          "feature_canvas.read",
          "feature_visualize.read"
        ],
        "resources": ["space:analytics"]
      }
    ]
  }'

Step 3 — Create the Test Users

# Platform user
curl -X POST "https://localhost:9200/_security/user/platform-user" \
  -H "Content-Type: application/json" \
  -u elastic:changeme \
  -d '{
    "password": "Plat4rm!User",
    "roles": ["platform-engineer"],
    "full_name": "Platform User",
    "email": "platform@company.com"
  }'

# Analyst user
curl -X POST "https://localhost:9200/_security/user/analyst-user" \
  -H "Content-Type: application/json" \
  -u elastic:changeme \
  -d '{
    "password": "An4lyst!User",
    "roles": ["business-analyst"],
    "full_name": "Analyst User",
    "email": "analyst@company.com"
  }'

Step 4 — Verify Platform User Access

1. Open a private/incognito browser window
2. Navigate to https://localhost:5601/s/platform/app/discover
3. Log in as platform-user / Plat4rm!User
4. Verify:
   • ✅ Can access Discover, Dashboard, Alerting, Dev Tools in the platform Space
   • ✅ Can search kibana_sample_data_logs
   • ✅ The user.email field is NOT visible in Discover
   • ❌ Cannot access the analytics Space
   • ❌ Cannot access Canvas or Maps

Step 5 — Verify Analyst User Access

1. Open another private window
2. Navigate to https://localhost:5601/s/analytics/app/dashboards
3. Log in as analyst-user / An4lyst!User
4. Verify:
   • ✅ Can view dashboards and Canvas workpads in the analytics Space
   • ✅ Can read ecommerce data
   • ❌ Cannot create or edit dashboards (read-only)
   • ❌ Cannot access the platform Space
   • ❌ Cannot access Discover or Dev Tools

Cleanup (Optional)

# Delete test users
curl -X DELETE "https://localhost:9200/_security/user/platform-user" -u elastic:changeme
curl -X DELETE "https://localhost:9200/_security/user/analyst-user" -u elastic:changeme

# Delete custom roles
curl -X DELETE "https://localhost:9200/_security/role/platform-engineer" -u elastic:changeme
curl -X DELETE "https://localhost:9200/_security/role/business-analyst" -u elastic:changeme

---

Summary

Exercise	Topic	Difficulty
1	Data view with runtime fields	⭐⭐
2	Canvas workpad for status board	⭐⭐
3	Alerting rules with connectors	⭐⭐⭐
4	Spaces and copying objects	⭐⭐⭐
5	RBAC with custom roles and FLS	⭐⭐⭐⭐

These exercises simulate real-world scenarios you will encounter when managing a multi-team Kibana deployment. Practice them in a development environment before applying the concepts to production.

---

← Back to: Kibana Advanced