ELK Stack — Docker Compose Stack

5 minute read

Docker Compose ELK Stack

The fastest way to get the entire ELK Stack running is with Docker Compose. One command spins up Elasticsearch, Kibana, Logstash, and Filebeat — all pre-configured to talk to each other.

This is the recommended approach for learning throughout this tutorial.

Prerequisites

Docker installed — version 20.10+
Docker Compose installed — version 2.0+ (included with Docker Desktop)
8 GB RAM minimum allocated to Docker (16 GB recommended)
20 GB free disk space

Check your Docker version:

$ docker --version
Docker version 24.0.7, build afdd53b

$ docker compose version
Docker Compose version v2.23.3

Project Structure

Create a project directory:

$ mkdir -p ~/elk-stack && cd ~/elk-stack

The final structure will look like:

elk-stack/
├── docker-compose.yml
├── .env
├── logstash/
│   └── pipeline/
│       └── logstash.conf
├── filebeat/
│   └── filebeat.yml
└── sample-logs/
    └── access.log

Step 1 — Create the Environment File

$ cat > .env << 'EOF'
ELASTIC_VERSION=8.15.0
ELASTIC_PASSWORD=changeme123
KIBANA_PASSWORD=changeme123
STACK_VERSION=8.15.0
CLUSTER_NAME=elk-learning
LICENSE=basic
ES_PORT=9200
KIBANA_PORT=5601
ES_MEM_LIMIT=2147483648
KB_MEM_LIMIT=1073741824
LS_MEM_LIMIT=1073741824
EOF

Step 2 — Create docker-compose.yml

$ cat > docker-compose.yml << 'COMPOSE'
version: "3.8"

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTIC_VERSION}
    container_name: elasticsearch
    environment:
      - node.name=es-node-1
      - cluster.name=${CLUSTER_NAME}
      - discovery.type=single-node
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=false
      - xpack.security.transport.ssl.enabled=false
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - es-data:/usr/share/elasticsearch/data
    ports:
      - "${ES_PORT}:9200"
      - "9300:9300"
    mem_limit: ${ES_MEM_LIMIT}
    networks:
      - elk
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -u elastic:${ELASTIC_PASSWORD} http://localhost:9200/_cluster/health | grep -q '\"status\":\"green\"\\|\"status\":\"yellow\"'",
        ]
      interval: 10s
      timeout: 10s
      retries: 30

  kibana:
    image: docker.elastic.co/kibana/kibana:${ELASTIC_VERSION}
    container_name: kibana
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - xpack.security.enabled=true
    ports:
      - "${KIBANA_PORT}:5601"
    mem_limit: ${KB_MEM_LIMIT}
    networks:
      - elk
    depends_on:
      elasticsearch:
        condition: service_healthy

  logstash:
    image: docker.elastic.co/logstash/logstash:${ELASTIC_VERSION}
    container_name: logstash
    environment:
      - "LS_JAVA_OPTS=-Xms512m -Xmx512m"
    volumes:
      - ./logstash/pipeline/:/usr/share/logstash/pipeline/:ro
    ports:
      - "5044:5044"
      - "9600:9600"
    mem_limit: ${LS_MEM_LIMIT}
    networks:
      - elk
    depends_on:
      elasticsearch:
        condition: service_healthy

  filebeat:
    image: docker.elastic.co/beats/filebeat:${ELASTIC_VERSION}
    container_name: filebeat
    user: root
    command: filebeat -e -strict.perms=false
    volumes:
      - ./filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
      - ./sample-logs/:/var/log/sample/:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      - elk
    depends_on:
      elasticsearch:
        condition: service_healthy

volumes:
  es-data:
    driver: local

networks:
  elk:
    driver: bridge
COMPOSE

Step 3 — Create the Logstash Pipeline

$ mkdir -p logstash/pipeline

$ cat > logstash/pipeline/logstash.conf << 'EOF'
input {
  beats {
    port => 5044
  }
}

filter {
  if [fields][log_type] == "nginx" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
    date {
      match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
      target => "@timestamp"
    }
    mutate {
      convert => {
        "response" => "integer"
        "bytes" => "integer"
      }
    }
  }
}

output {
  elasticsearch {
    hosts => ["http://elasticsearch:9200"]
    user => "elastic"
    password => "changeme123"
    index => "logs-%{+YYYY.MM.dd}"
  }
  stdout {
    codec => rubydebug
  }
}
EOF

Step 4 — Create the Filebeat Configuration

$ mkdir -p filebeat

$ cat > filebeat/filebeat.yml << 'EOF'
filebeat.inputs:
  - type: filestream
    id: sample-nginx
    enabled: true
    paths:
      - /var/log/sample/access.log
    fields:
      log_type: nginx
    fields_under_root: true

output.logstash:
  hosts: ["logstash:5044"]

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_docker_metadata: ~
EOF

Step 5 — Create Sample Logs

$ mkdir -p sample-logs

$ cat > sample-logs/access.log << 'EOF'
168.1.10 - - [15/Mar/2025:10:00:01 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0"
168.1.11 - - [15/Mar/2025:10:00:02 +0000] "GET /api/users HTTP/1.1" 200 1234 "-" "curl/7.88"
168.1.12 - - [15/Mar/2025:10:00:03 +0000] "POST /api/login HTTP/1.1" 401 89 "-" "Mozilla/5.0"
0.0.50 - - [15/Mar/2025:10:00:04 +0000] "GET /images/logo.png HTTP/1.1" 304 0 "-" "Mozilla/5.0"
168.1.13 - - [15/Mar/2025:10:00:05 +0000] "GET /api/products HTTP/1.1" 500 45 "-" "Python/3.11"
168.1.10 - - [15/Mar/2025:10:00:06 +0000] "DELETE /api/users/42 HTTP/1.1" 403 67 "-" "curl/7.88"
0.0.100 - - [15/Mar/2025:10:00:07 +0000] "GET /health HTTP/1.1" 200 15 "-" "kube-probe/1.28"
168.1.14 - - [15/Mar/2025:10:00:08 +0000] "PUT /api/users/1 HTTP/1.1" 200 234 "-" "Mozilla/5.0"
168.1.15 - - [15/Mar/2025:10:00:09 +0000] "GET /api/orders HTTP/1.1" 200 5678 "-" "Mozilla/5.0"
0.0.50 - - [15/Mar/2025:10:00:10 +0000] "GET /favicon.ico HTTP/1.1" 404 0 "-" "Mozilla/5.0"
EOF

Step 6 — Set the kibana_system Password

Before starting, we need to set up the kibana_system user password. First, start only Elasticsearch:

$ docker compose up -d elasticsearch

Wait for it to be healthy:

$ docker compose ps

Set the kibana_system password:

$ curl -s -X POST -u elastic:changeme123 \
  "http://localhost:9200/_security/user/kibana_system/_password" \
  -H "Content-Type: application/json" \
  -d '{"password": "changeme123"}'

Step 7 — Start the Full Stack

$ docker compose up -d

Expected output:

[+] Running 4/4
 ✔ Container elasticsearch  Started
 ✔ Container kibana          Started
 ✔ Container logstash        Started
 ✔ Container filebeat        Started

Step 8 — Verify Everything Is Running

Check all containers:

$ docker compose ps

Expected output:

NAME            IMAGE                                          STATUS
elasticsearch   docker.elastic.co/elasticsearch/...            Up (healthy)
kibana          docker.elastic.co/kibana/...                   Up
logstash        docker.elastic.co/logstash/...                 Up
filebeat        docker.elastic.co/beats/filebeat:...           Up

Test Elasticsearch:

$ curl -u elastic:changeme123 http://localhost:9200

Test Kibana (wait 1-2 minutes for startup):

$ curl -s http://localhost:5601/api/status | python3 -m json.tool | head -5

Step 9 — View Logs in Kibana

Open http://localhost:5601 in your browser
Log in with elastic / changeme123
Go to Discover (hamburger menu → Analytics → Discover)
Create a data view: logs-*
You should see the sample NGINX log events

Useful Docker Compose Commands

Command	Description
`docker compose up -d`	Start all services in background
`docker compose down`	Stop and remove all containers
`docker compose down -v`	Stop and remove containers + volumes (deletes data!)
`docker compose ps`	Show running containers
`docker compose logs -f`	Follow all container logs
`docker compose logs -f elasticsearch`	Follow one container’s logs
`docker compose restart logstash`	Restart a single service
`docker compose exec elasticsearch bash`	Shell into a container

Adding More Sample Logs

To generate continuous log data for testing:

$ while true; do
  echo "192.168.1.$((RANDOM % 255)) - - [$(date '+%d/%b/%Y:%H:%M:%S %z')] \"GET /api/endpoint$((RANDOM % 10)) HTTP/1.1\" $((RANDOM % 2 == 0 ? 200 : 500)) $((RANDOM % 5000))\" \"-\" \"TestAgent/1.0\"" >> ~/elk-stack/sample-logs/access.log
  sleep 1
done

Press Ctrl+C to stop. Check Kibana — new events should appear within seconds.

Resource Usage

Service	RAM (typical)	CPU (typical)
Elasticsearch	1-2 GB	Low-Medium
Kibana	500 MB-1 GB	Low
Logstash	500 MB-1 GB	Low-Medium
Filebeat	30-50 MB	Very Low
Total	~3-4 GB	Low-Medium

Quick Reference

Task Command

Start stack docker compose up -d

Stop stack docker compose down

Reset everything docker compose down -v

View logs docker compose logs -f

Elasticsearch http://localhost:9200

Kibana http://localhost:5601

Credentials elastic / changeme123

Logstash API http://localhost:9600

Task	Command
Start stack	`docker compose up -d`
Stop stack	`docker compose down`
Reset everything	`docker compose down -v`
View logs	`docker compose logs -f`
Elasticsearch	`http://localhost:9200`
Kibana	`http://localhost:5601`
Credentials	`elastic` / `changeme123`
Logstash API	`http://localhost:9600`

Next Step

You have the entire ELK Stack running! Let’s test your skills with installation exercises.

Share on

Twitter Facebook LinkedIn