Amazon EKS — Secrets Management

Kubernetes Secrets are base64-encoded by default — not encrypted. In EKS, you must enable envelope encryption with KMS and use external secret stores for production workloads.

---

EKS Envelope Encryption with KMS

Enable encryption at rest for Kubernetes Secrets using AWS KMS:

aws eks create-cluster \
  --name my-cluster \
  --encryption-config '[{
    "resources": ["secrets"],
    "provider": {
      "keyArn": "arn:aws:kms:us-east-1:123456789:key/abcd-1234-5678"
    }
  }]' \
  ...

For an existing cluster:

aws eks associate-encryption-config \
  --cluster-name my-cluster \
  --encryption-config '[{
    "resources": ["secrets"],
    "provider": {
      "keyArn": "arn:aws:kms:us-east-1:123456789:key/abcd-1234-5678"
    }
  }]'

Note: Enabling encryption on an existing cluster re-encrypts all secrets. This can take several minutes.

---

AWS Secrets and Configuration Provider (ASCP)

ASCP mounts secrets from AWS Secrets Manager or Parameter Store as files in pods.

Install CSI Secrets Store Driver + ASCP

# Secrets Store CSI Driver
helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
helm install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver \
  --namespace kube-system \
  --set syncSecret.enabled=true

# AWS Provider
kubectl apply -f https://raw.githubusercontent.com/aws/secrets-store-csi-driver-provider-aws/main/deployment/aws-provider-installer.yaml

SecretProviderClass

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: db-credentials
  namespace: production
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "prod/db-password"
        objectType: "secretsmanager"
      - objectName: "/app/config/db-host"
        objectType: "ssmparameter"
  # Optional: sync to K8s Secret for env var use
  secretObjects:
    - secretName: db-creds-k8s
      type: Opaque
      data:
        - objectName: "prod/db-password"
          key: password

Use in Pod

apiVersion: v1
kind: Pod
metadata:
  name: app
  namespace: production
spec:
  serviceAccountName: app-sa     # Must have IRSA/Pod Identity for Secrets Manager
  containers:
    - name: app
      image: myapp:latest
      volumeMounts:
        - name: secrets
          mountPath: /mnt/secrets
          readOnly: true
      env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-creds-k8s
              key: password
  volumes:
    - name: secrets
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: db-credentials

---

External Secrets Operator (ESO)

ESO syncs external secrets into native Kubernetes Secrets:

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets \
  --namespace external-secrets --create-namespace

ClusterSecretStore

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: aws-secrets-manager
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        jwt:
          serviceAccountRef:
            name: external-secrets-sa
            namespace: external-secrets

ExternalSecret

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: production
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  target:
    name: db-creds
    creationPolicy: Owner
  data:
    - secretKey: password
      remoteRef:
        key: prod/db-password
        property: password
    - secretKey: username
      remoteRef:
        key: prod/db-password
        property: username

---

Sealed Secrets

For GitOps workflows — encrypt secrets that can be safely committed to Git:

# Install controller
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets sealed-secrets/sealed-secrets -n kube-system

# Install kubeseal CLI
brew install kubeseal

# Seal a secret
kubectl create secret generic my-secret \
  --from-literal=password=secret123 \
  --dry-run=client -o yaml | kubeseal -o yaml > sealed-secret.yaml

kubectl apply -f sealed-secret.yaml

---

Comparison

Feature	K8s Secrets + KMS	ASCP	ESO	Sealed Secrets
Secret storage	etcd (encrypted)	AWS Secrets Manager	AWS Secrets Manager	etcd
Rotation	Manual	Auto-refresh via CSI	refreshInterval	Manual
GitOps-friendly	No (plain Secret)	No (mounted at runtime)	No (synced at runtime)	Yes (encrypted YAML)
Env var support	Native	Via secretObjects sync	Native K8s Secret	Native K8s Secret
File mount	Via volumeMount	Native	Via volumeMount	Via volumeMount
Multi-provider	No	AWS only	AWS, GCP, Azure, Vault, etc.	No

---

Best Practices

1. Enable KMS envelope encryption on all clusters — non-negotiable for production
2. Use ASCP or ESO to pull secrets from Secrets Manager — not hardcoded in manifests
3. Enable automatic rotation in AWS Secrets Manager — ESO picks up changes via refreshInterval
4. Use IRSA/Pod Identity for secret access — scope IAM permissions per application
5. Never commit plain Secrets to Git — use Sealed Secrets or ESO for GitOps workflows

---