ELK Stack — Elastic APM

4 minute read

Elastic APM

Application Performance Monitoring (APM) tracks the performance and availability of your applications in real-time. Elastic APM integrates natively with Elasticsearch and Kibana, giving you distributed tracing, error tracking, and performance metrics alongside your logs.

APM Architecture

┌──────────┐  ┌──────────┐  ┌──────────┐
│  Service  │  │  Service  │  │  Service  │
│  + Agent  │  │  + Agent  │  │  + Agent  │
└─────┬─────┘  └─────┬─────┘  └─────┬─────┘
      │              │              │
      └──────────────┼──────────────┘
                     │
              ┌──────▼──────┐
              │  APM Server  │  (or Elastic Agent
              │              │   with APM integration)
              └──────┬───────┘
                     │
              ┌──────▼──────┐
              │Elasticsearch │
              └──────┬───────┘
                     │
              ┌──────▼──────┐
              │   Kibana     │
              │  (APM App)   │
              └──────────────┘

Components

Component	Purpose
APM Agent	Instruments your application code (auto or manual)
APM Server	Receives, validates, and transforms APM data
Elasticsearch	Stores transactions, spans, errors, and metrics
Kibana APM App	Visualises performance data and service maps

Supported APM Agents

Elastic provides official agents for all major languages:

Language	Agent Package	Auto-Instrumentation
Java	`elastic-apm-agent` (JAR)	✅ Yes (javaagent)
Node.js	`elastic-apm-node`	✅ Yes
Python	`elastic-apm`	✅ Yes (Django, Flask, Starlette)
.NET	`Elastic.Apm`	✅ Yes
Go	`go.elastic.co/apm`	Manual + helpers
Ruby	`elastic-apm`	✅ Yes (Rails, Sinatra)
PHP	`elastic-apm-php`	✅ Yes
RUM JS	`@elastic/apm-rum`	Browser instrumentation

Installing APM Server

Standalone APM Server

# Install APM Server
curl -L -O https://artifacts.elastic.co/downloads/apm-server/apm-server-8.12.0-linux-x86_64.tar.gz
tar xzvf apm-server-8.12.0-linux-x86_64.tar.gz
cd apm-server-8.12.0-linux-x86_64

Configure apm-server.yml:

apm-server:
  host: "0.0.0.0:8200"
  auth:
    secret_token: "your-secret-token"

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: "elastic"
  password: "changeme"
  ssl:
    certificate_authorities: ["/etc/apm-server/ca.crt"]

kibana:
  enabled: true
  host: "https://localhost:5601"

# Start APM Server
./apm-server -e

APM via Elastic Agent (Recommended for 8.x)

In Kibana Fleet:

Navigate to Fleet → Agent Policies
Add the APM integration to your policy
Configure the APM integration settings
Deploy Elastic Agent with the updated policy

Configuring APM Agents

Java Agent

# Download the agent JAR
curl -L -O https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent/1.45.0/elastic-apm-agent-1.45.0.jar

# Run your application with the agent
java -javaagent:/path/to/elastic-apm-agent-1.45.0.jar \
  -Delastic.apm.service_name=my-java-service \
  -Delastic.apm.server_urls=http://apm-server:8200 \
  -Delastic.apm.secret_token=your-secret-token \
  -Delastic.apm.environment=production \
  -Delastic.apm.application_packages=com.mycompany \
  -jar my-app.jar

Python Agent (Flask)

pip install elastic-apm[flask]

from flask import Flask
from elasticapm.contrib.flask import ElasticAPM

app = Flask(__name__)
app.config['ELASTIC_APM'] = {
    'SERVICE_NAME': 'my-python-service',
    'SERVER_URL': 'http://apm-server:8200',
    'SECRET_TOKEN': 'your-secret-token',
    'ENVIRONMENT': 'production',
}
apm = ElasticAPM(app)

@app.route('/api/users')
def get_users():
    # Automatically traced
    users = User.query.all()
    return jsonify([u.to_dict() for u in users])

Node.js Agent

// Must be the very first line in your app
const apm = require('elastic-apm-node').start({
  serviceName: 'my-node-service',
  serverUrl: 'http://apm-server:8200',
  secretToken: 'your-secret-token',
  environment: 'production',
});

const express = require('express');
const app = express();

app.get('/api/orders', async (req, res) => {
  // Automatically traced
  const orders = await db.query('SELECT * FROM orders');
  res.json(orders);
});

app.listen(3000);

APM Data Model

Transactions

A transaction represents a single request or job:

{
  "transaction": {
    "id": "abc123",
    "trace_id": "def456",
    "name": "GET /api/users",
    "type": "request",
    "duration": { "us": 45230 },
    "result": "HTTP 2xx",
    "outcome": "success"
  }
}

Spans

Spans represent individual operations within a transaction:

{
  "span": {
    "id": "span789",
    "transaction_id": "abc123",
    "trace_id": "def456",
    "name": "SELECT FROM users",
    "type": "db",
    "subtype": "postgresql",
    "duration": { "us": 12500 },
    "context": {
      "db": {
        "statement": "SELECT * FROM users WHERE active = true",
        "type": "sql"
      }
    }
  }
}

Errors

Errors are automatically captured:

{
  "error": {
    "id": "err123",
    "trace_id": "def456",
    "transaction_id": "abc123",
    "exception": {
      "message": "Connection refused",
      "type": "ConnectionError",
      "stacktrace": [...]
    }
  }
}

Distributed Tracing

When services communicate, trace context propagates automatically:

User Request
    │
    ▼
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│ API Gateway  │────▶│ Order Service │────▶│  Payment    │
│ trace: abc   │     │ trace: abc    │     │ trace: abc  │
│ span: 001    │     │ span: 002     │     │ span: 003   │
└─────────────┘     └──────┬────────┘     └─────────────┘
                           │
                    ┌──────▼────────┐
                    │ Inventory Svc  │
                    │ trace: abc     │
                    │ span: 004      │
                    └────────────────┘

The traceparent header propagates context:

traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01

Kibana APM App

Service Overview

The APM app provides:

Services list — all instrumented services with latency, throughput, error rate
Service details — transactions, errors, metrics, dependencies
Transaction details — waterfall view of spans, metadata, related logs
Error tracking — grouped errors with occurrence count and stack traces
Service Map — visual dependency graph

Key Metrics

Metric	Description
Latency	Average, p95, p99 transaction duration
Throughput	Requests per minute
Error rate	Percentage of failed transactions
Dependencies	External services called
Instance metrics	CPU, memory per service instance

Correlating APM with Logs

Link APM data to your logs:

import logging
import elasticapm

logger = logging.getLogger(__name__)

@app.route('/api/process')
def process_request():
    # trace.id and transaction.id are automatically added
    # when using the Elastic APM log formatter
    logger.info("Processing request", extra={
        'trace.id': elasticapm.get_trace_id(),
        'transaction.id': elasticapm.get_transaction_id(),
    })

In Kibana APM, click Actions → View related logs to jump from a transaction to its corresponding log entries.

Custom Instrumentation

For operations not auto-instrumented:

Python

import elasticapm

@elasticapm.capture_span('cache_lookup')
def get_from_cache(key):
    return redis.get(key)

# Or manually
client = elasticapm.get_client()
with elasticapm.capture_span('custom_operation', span_type='custom'):
    result = perform_complex_calculation()

Java

import co.elastic.apm.api.ElasticApm;
import co.elastic.apm.api.Span;

Span span = ElasticApm.currentSpan().startSpan("custom", "operation", null);
try {
    span.setName("Complex Calculation");
    // Your code here
} finally {
    span.end();
}

Quick Reference — Elastic APM provides distributed tracing and performance monitoring. Install APM Server (or use Elastic Agent APM integration), add language-specific agents to your services, and use the Kibana APM app for service maps, latency analysis, error tracking, and log correlation. Auto-instrumentation handles most frameworks automatically.

Share on

Twitter Facebook LinkedIn