Amazon EKS — AWS Distro for OpenTelemetry

2 minute read

ADOT is AWS’s distribution of OpenTelemetry — an open-source framework for collecting metrics, traces, and logs. It runs as a DaemonSet or sidecar on EKS and can forward telemetry to multiple AWS and third-party destinations.


Why ADOT?

Without ADOT With ADOT
Separate agents for metrics, logs, traces Single collector for all telemetry
Vendor-locked instrumentation Vendor-neutral OpenTelemetry SDK
Multiple configs to manage One pipeline config

Install as EKS Add-on

aws eks create-addon \
  --cluster-name my-cluster \
  --addon-name adot \
  --service-account-role-arn arn:aws:iam::123456789:role/ADOTCollectorRole

Verify:

kubectl get pods -n opentelemetry-operator-system

IRSA for ADOT

eksctl create iamserviceaccount \
  --cluster my-cluster \
  --namespace opentelemetry-operator-system \
  --name adot-collector-sa \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonPrometheusRemoteWriteAccess \
  --attach-policy-arn arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess \
  --attach-policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \
  --approve

Collector Configuration

The ADOT Collector uses a pipeline model: Receivers → Processors → Exporters.

Metrics to AMP + Traces to X-Ray

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: adot
  namespace: opentelemetry-operator-system
spec:
  mode: daemonset
  serviceAccount: adot-collector-sa
  config: |
    receivers:
      prometheus:
        config:
          scrape_configs:
            - job_name: 'kubernetes-pods'
              kubernetes_sd_configs:
                - role: pod
              relabel_configs:
                - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
                  action: keep
                  regex: true
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
          http:
            endpoint: 0.0.0.0:4318

    processors:
      batch:
        timeout: 30s
        send_batch_size: 1000
      memory_limiter:
        check_interval: 5s
        limit_mib: 512

    exporters:
      prometheusremotewrite:
        endpoint: https://aps-workspaces.us-east-1.amazonaws.com/workspaces/ws-12345678/api/v1/remote_write
        auth:
          authenticator: sigv4auth
      awsxray:
        region: us-east-1

    extensions:
      sigv4auth:
        region: us-east-1

    service:
      extensions: [sigv4auth]
      pipelines:
        metrics:
          receivers: [prometheus]
          processors: [batch, memory_limiter]
          exporters: [prometheusremotewrite]
        traces:
          receivers: [otlp]
          processors: [batch, memory_limiter]
          exporters: [awsxray]

Application Instrumentation

Python (OpenTelemetry SDK)

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Configure OTLP exporter pointing to ADOT Collector
exporter = OTLPSpanExporter(endpoint="http://adot-collector.opentelemetry-operator-system:4317")
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("process-order"):
    # Application logic
    pass

Auto-Instrumentation (No Code Changes)

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: auto-instrumentation
  namespace: production
spec:
  exporter:
    endpoint: http://adot-collector.opentelemetry-operator-system:4317
  propagators:
    - tracecontext
    - baggage
    - xray
  python:
    image: public.ecr.aws/aws-observability/adot-autoinstrumentation-python:latest
  java:
    image: public.ecr.aws/aws-observability/adot-autoinstrumentation-java:latest

Annotate pods for auto-instrumentation:

metadata:
  annotations:
    instrumentation.opentelemetry.io/inject-python: "true"
    # or: instrumentation.opentelemetry.io/inject-java: "true"

Sending to Multiple Destinations

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      processors: [batch]
      exporters: [prometheusremotewrite, cloudwatchlogs]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [awsxray]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [cloudwatchlogs]

ADOT vs Other Collectors

Feature ADOT CloudWatch Agent Fluent Bit Prometheus
Metrics Yes Yes No Yes
Traces Yes No No No
Logs Yes Yes Yes No
Vendor-neutral Yes (OTel) AWS-only Mostly Open source
Managed add-on Yes Yes (Container Insights) No No

Best Practices

  1. Use ADOT for unified telemetry — one collector for metrics, traces, and logs
  2. Use the EKS add-on — managed lifecycle, automatic upgrades
  3. Enable auto-instrumentation — no code changes for basic tracing
  4. Set memory_limiter — prevents collector from consuming excessive memory
  5. Batch exports — reduce API calls to AWS services
  6. Use IRSA — scope permissions to only the destinations you’re exporting to

Updated: