ELK Stack — Metricbeat and Heartbeat

5 minute read

Metricbeat Overview

Metricbeat is a lightweight shipper that collects system-level and service-level metrics from your servers and ships them to Elasticsearch or Logstash. It provides out-of-the-box modules for operating system metrics, Docker, Kubernetes, databases, web servers, and more.

┌─────────────────────────────────────────────────────────┐
│                     Your Server                          │
│                                                          │
│  ┌────────────────────────────────────────────────────┐ │
│  │                   Metricbeat                        │ │
│  │                                                     │ │
│  │  ┌────────────┐  ┌────────────┐  ┌──────────────┐ │ │
│  │  │  System     │  │  Docker    │  │ Kubernetes   │ │ │
│  │  │  Module     │  │  Module    │  │ Module       │ │ │
│  │  │            │  │            │  │              │ │ │
│  │  │ ● CPU       │  │ ● container│  │ ● pod        │ │ │
│  │  │ ● Memory    │  │ ● cpu      │  │ ● node       │ │ │
│  │  │ ● Disk      │  │ ● memory   │  │ ● container  │ │ │
│  │  │ ● Network   │  │ ● diskio   │  │ ● volume     │ │ │
│  │  │ ● Process   │  │ ● network  │  │ ● event      │ │ │
│  │  │ ● Filesystem│  │ ● info     │  │ ● deployment │ │ │
│  │  └─────┬───────┘  └─────┬─────┘  └──────┬───────┘ │ │
│  │        │                │                │         │ │
│  │        └────────────────┼────────────────┘         │ │
│  │                         │                          │ │
│  │                         ▼                          │ │
│  │              ┌──────────────────┐                  │ │
│  │              │ Elasticsearch /  │                  │ │
│  │              │ Logstash Output  │                  │ │
│  │              └──────────────────┘                  │ │
│  └────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘

Metricbeat Installation

# Debian/Ubuntu
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.17.0-amd64.deb
sudo dpkg -i metricbeat-8.17.0-amd64.deb

# RHEL/CentOS
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.17.0-x86_64.rpm
sudo rpm -vi metricbeat-8.17.0-x86_64.rpm

# macOS
brew install elastic/tap/metricbeat-full

# Docker
docker pull docker.elastic.co/beats/metricbeat:8.17.0

Complete metricbeat.yml Configuration

# ========================= Metricbeat Modules ============================
metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 10s

# ========================= System Module =================================
metricbeat.modules:
  - module: system
    metricsets:
      - cpu
      - memory
      - network
      - process
      - process_summary
      - filesystem
      - diskio
      - load
      - socket_summary
      - uptime
    enabled: true
    period: 10s
    # Monitor top N processes by CPU and memory
    processes: ['.*']
    process.include_top_n:
      by_cpu: 10
      by_memory: 10
    cpu.metrics: ["percentages", "normalized_percentages"]
    core.metrics: ["percentages"]

# ========================= Elasticsearch Output ==========================
output.elasticsearch:
  hosts: ["https://es-node-1:9200"]
  username: "metricbeat_writer"
  password: "${ES_PASSWORD}"
  ssl:
    enabled: true
    certificate_authorities: ["/etc/metricbeat/certs/ca.crt"]

# ============================== Kibana ===================================
setup.kibana:
  host: "https://kibana-host:5601"

# ============================= Dashboards ================================
setup.dashboards.enabled: true

# ============================== Processors ===============================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~

# ============================== Logging ==================================
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/metricbeat
  name: metricbeat
  keepfiles: 7
  permissions: 0644

System Module Metricsets

The system module provides granular control over which metrics to collect:

Metricset	Metrics collected
`cpu`	CPU usage (user, system, idle, iowait, IRQ, steal) per core
`memory`	Total, used, free, cached, swap, hugepages
`load`	1-min, 5-min, 15-min load averages
`network`	Bytes in/out, packets in/out, errors, drops per interface
`diskio`	Read/write bytes, IOPS, queue length per device
`filesystem`	Total, used, free, usage percentage per mount point
`process`	CPU, memory, file descriptors, state per process
`process_summary`	Count of processes by state (running, sleeping, zombie)
`socket_summary`	Open TCP/UDP sockets count
`uptime`	System uptime duration

Docker Module

# /etc/metricbeat/modules.d/docker.yml
- module: docker
  metricsets:
    - container
    - cpu
    - diskio
    - healthcheck
    - info
    - memory
    - network
  hosts: ["unix:///var/run/docker.sock"]
  period: 10s
  enabled: true
  # Filter by container labels
  # labels.dedot: true

Kubernetes Module

# /etc/metricbeat/modules.d/kubernetes.yml
- module: kubernetes
  metricsets:
    - node
    - system
    - pod
    - container
    - volume
    - event
  period: 10s
  host: ${NODE_NAME}
  hosts: ["https://${NODE_NAME}:10250"]
  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  ssl.verification_mode: none
  add_metadata: true

Heartbeat Overview

Heartbeat is a lightweight daemon that periodically checks whether your services are reachable. It supports HTTP, TCP, and ICMP monitors. Think of it as your uptime monitoring agent — deployed externally to your services to verify they are healthy.

Unlike Metricbeat (which runs on the server), Heartbeat is typically deployed outside the monitored services to simulate real user access patterns.

Heartbeat Installation

# Debian/Ubuntu
curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-8.17.0-amd64.deb
sudo dpkg -i heartbeat-8.17.0-amd64.deb

# RHEL/CentOS
curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-8.17.0-x86_64.rpm
sudo rpm -vi heartbeat-8.17.0-x86_64.rpm

Complete heartbeat.yml Configuration

# =========================== Heartbeat Monitors ==========================

heartbeat.monitors:

  # --- HTTP Monitor: check web applications ---
  - type: http
    id: web-app-monitor
    name: "Production Web App"
    enabled: true
    urls:
      - "https://myapp.example.com"
      - "https://api.example.com/health"
    schedule: '@every 30s'
    timeout: 10s
    # Check response body
    check.response:
      status: [200]
      body:
        - "healthy"
        - "ok"
    # Check response headers
    check.response.headers:
      Content-Type: "application/json"
    # TLS settings
    ssl:
      verification_mode: full
    tags: ["web", "production"]

  # --- HTTP Monitor: check Elasticsearch cluster ---
  - type: http
    id: es-cluster-health
    name: "Elasticsearch Cluster"
    enabled: true
    urls:
      - "https://es-node-1:9200/_cluster/health"
    schedule: '@every 60s'
    timeout: 10s
    username: "heartbeat_reader"
    password: "${ES_MONITOR_PASSWORD}"
    check.response:
      status: [200]
      json:
        - description: "Cluster status is green or yellow"
          condition:
            contains:
              status: "green"
    ssl:
      certificate_authorities: ["/etc/heartbeat/certs/ca.crt"]

  # --- TCP Monitor: check database ports ---
  - type: tcp
    id: postgres-monitor
    name: "PostgreSQL"
    enabled: true
    hosts: ["db-primary.internal:5432", "db-replica.internal:5432"]
    schedule: '@every 30s'
    timeout: 5s
    tags: ["database", "postgresql"]

  - type: tcp
    id: redis-monitor
    name: "Redis Cache"
    enabled: true
    hosts: ["redis.internal:6379"]
    schedule: '@every 15s'
    timeout: 3s
    tags: ["cache", "redis"]

  # --- ICMP Monitor: check host reachability ---
  - type: icmp
    id: host-ping
    name: "Server Ping Check"
    enabled: true
    hosts: ["10.0.1.10", "10.0.1.11", "10.0.1.12"]
    schedule: '@every 60s'
    timeout: 5s
    wait: 1s
    tags: ["infrastructure", "ping"]

# ========================= Elasticsearch Output ==========================
output.elasticsearch:
  hosts: ["https://es-node-1:9200"]
  username: "heartbeat_writer"
  password: "${ES_PASSWORD}"
  ssl:
    enabled: true
    certificate_authorities: ["/etc/heartbeat/certs/ca.crt"]

# ============================== Kibana ===================================
setup.kibana:
  host: "https://kibana-host:5601"

# ============================= Dashboards ================================
setup.dashboards.enabled: true

# ============================== Processors ===============================
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

# ============================== Logging ==================================
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/heartbeat
  name: heartbeat
  keepfiles: 7

Monitor Types Comparison

Monitor	Protocol	Use Case	Example
`http`	HTTP/HTTPS	Web apps, APIs, health endpoints	Check `https://api.example.com/health` returns 200
`tcp`	TCP	Databases, message brokers, any TCP service	Verify PostgreSQL port 5432 is open
`icmp`	ICMP (ping)	Host reachability, network connectivity	Ping servers on the internal network

Uptime Dashboard

After loading dashboards (heartbeat setup --dashboards), open Kibana and navigate to Uptime (or Observability → Uptime) to see:

Monitor status — up/down for each configured monitor
Availability percentage — over configurable time windows
Response time trends — latency graphs for HTTP and TCP monitors
Certificate expiration — TLS certificate monitoring for HTTPS endpoints
Alerts — configure alerts when monitors go down

Starting and Managing Metricbeat and Heartbeat

# --- Metricbeat ---
# Test config
metricbeat test config -e
metricbeat test output

# Start
sudo systemctl start metricbeat
sudo systemctl enable metricbeat

# --- Heartbeat ---
# Test config
heartbeat test config -e
heartbeat test output

# Start
sudo systemctl start heartbeat-elastic
sudo systemctl enable heartbeat-elastic

# --- Load dashboards ---
metricbeat setup --dashboards
heartbeat setup --dashboards

Quick Reference

Task	Metricbeat	Heartbeat
Config file	`/etc/metricbeat/metricbeat.yml`	`/etc/heartbeat/heartbeat.yml`
Modules dir	`/etc/metricbeat/modules.d/`	—
Enable module	`metricbeat modules enable docker`	—
Test config	`metricbeat test config -e`	`heartbeat test config -e`
Test output	`metricbeat test output`	`heartbeat test output`
Start service	`systemctl start metricbeat`	`systemctl start heartbeat-elastic`
Load dashboards	`metricbeat setup --dashboards`	`heartbeat setup --dashboards`
Default period	`10s`	`@every 30s`
Typical memory	~30-50 MB	~20-30 MB

Next: Processors — enrich, filter, and transform data at the edge before shipping.

Share on

Twitter Facebook LinkedIn