Amazon EKS — Logging with Fluent Bit

1 minute read

Logging with Fluent Bit

Fluent Bit is a lightweight log processor that collects container logs and forwards them to CloudWatch, S3, Elasticsearch, or other destinations.

Architecture

Log Collection
┌─────────────────────────────────────────────────┐
│  Node                                           │
│  ┌────────┐ ┌────────┐                         │
│  │ Pod A  │ │ Pod B  │  → stdout/stderr         │
│  └───┬────┘ └───┬────┘                         │
│      │          │                               │
│      ▼          ▼                               │
│  /var/log/containers/*.log                      │
│      │                                          │
│  ┌───▼──────────────┐                           │
│  │ Fluent Bit       │ (DaemonSet)               │
│  │ (collect, parse, │                           │
│  │  filter, forward)│                           │
│  └──────┬───────────┘                           │
│         │                                       │
└─────────┼───────────────────────────────────────┘
          │
    ┌─────▼─────┐ ┌──────────┐ ┌─────────┐
    │ CloudWatch│ │ S3       │ │ Elastic │
    │ Logs      │ │          │ │ search  │
    └───────────┘ └──────────┘ └─────────┘

Install Fluent Bit on EKS

# AWS-provided Fluent Bit image for EKS
$ kubectl apply -f https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonSet/container-insights-monitoring/fluent-bit/fluent-bit.yaml

Or use the CloudWatch observability addon:

$ aws eks create-addon --cluster-name demo --addon-name amazon-cloudwatch-observability

Log Destinations

Destination Best For Cost
CloudWatch Logs Quick setup, AWS-native $0.50/GB ingestion
S3 Long-term archival $0.023/GB storage
OpenSearch Full-text search, analytics Varies by instance
Datadog/Splunk Enterprise observability Subscription

Fargate Logging

Fargate doesn’t support DaemonSets, so use the built-in log router:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-logging
  namespace: aws-observability
data:
  output.conf: |
    [OUTPUT]
        Name cloudwatch_logs
        Match *
        region us-east-1
        log_group_name /eks/fargate
        log_stream_prefix fargate-
        auto_create_group true

Common Mistakes

Mistake Impact
No log retention policy Costs grow forever
Logging everything at DEBUG level Excessive volume and cost
Not parsing JSON logs Can’t search structured fields

What’s Next?

Request tracing: Distributed Tracing.

You may also enjoy