AWS for DevOps — Change Sets & Drift Detection

3 minute read

8.4 — Change Sets & Drift Detection

Change Sets — Preview Before You Apply

A Change Set shows you exactly what CloudFormation will change, add, or delete — before it does anything. Think of it as git diff for infrastructure.

  Updated template
       │
       ▼
  Create Change Set  (preview only, no changes yet)
       │
       ▼
  Review changes
  ┌─────────────────────────────────────────────┐
  │ Action    Resource         Replacement?      │
  │ Modify    WebServer        No (tag change)   │
  │ Modify    WebServer        Yes (AMI change!) │
  │ Add       NewAlarm         —                  │
  │ Remove    OldBucket        —                  │
  └─────────────────────────────────────────────┘
       │
  Approve?  ──No──▶  Delete Change Set (no harm done)
       │
      Yes
       │
       ▼
  Execute Change Set (apply changes)

Creating a Change Set

# Create the change set (does NOT apply changes)
$ aws cloudformation create-change-set \
    --stack-name my-app-prod \
    --change-set-name update-instance-type \
    --template-body file://template-v2.yml \
    --parameters ParameterKey=InstanceType,ParameterValue=t3.medium

# Wait for it to be created
$ aws cloudformation wait change-set-create-complete \
    --stack-name my-app-prod \
    --change-set-name update-instance-type

Reviewing the Change Set

$ aws cloudformation describe-change-set \
    --stack-name my-app-prod \
    --change-set-name update-instance-type \
    --query 'Changes[].ResourceChange.{Action:Action,Resource:LogicalResourceId,Type:ResourceType,Replacement:Replacement}'
[
    {
        "Action": "Modify",
        "Resource": "WebServer",
        "Type": "AWS::EC2::Instance",
        "Replacement": "Conditional"
    }
]
Replacement Value Meaning
False In-place update, no interruption
Conditional May or may not be replaced (depends on the property)
True Resource will be deleted and recreated

Executing or Deleting a Change Set

# Happy with the changes? Execute it.
$ aws cloudformation execute-change-set \
    --stack-name my-app-prod \
    --change-set-name update-instance-type

# Not happy? Delete it (no changes applied).
$ aws cloudformation delete-change-set \
    --stack-name my-app-prod \
    --change-set-name update-instance-type

Drift Detection

Drift occurs when someone manually changes a resource that CloudFormation manages. Example: an engineer adds an inbound rule to a security group via the console instead of the template.

  Template says:            Reality (after manual change):
  ┌───────────────┐        ┌───────────────┐
  │ SG: port 80   │        │ SG: port 80   │
  │     port 443  │        │     port 443  │
  │               │        │     port 22 ← │  drift!
  └───────────────┘        └───────────────┘

Running Drift Detection

# Step 1 — Start detection
$ aws cloudformation detect-stack-drift --stack-name my-app-prod
{
    "StackDriftDetectionId": "abc-123-def"
}

# Step 2 — Wait for it to complete
$ aws cloudformation describe-stack-drift-detection-status \
    --stack-drift-detection-id abc-123-def
{
    "StackDriftStatus": "DRIFTED",
    "DriftedStackResourceCount": 1
}

# Step 3 — See which resources drifted
$ aws cloudformation describe-stack-resource-drifts \
    --stack-name my-app-prod \
    --stack-resource-drift-status-filters MODIFIED
[
    {
        "LogicalResourceId": "WebSecurityGroup",
        "ResourceType": "AWS::EC2::SecurityGroup",
        "StackResourceDriftStatus": "MODIFIED",
        "PropertyDifferences": [
            {
                "PropertyPath": "/SecurityGroupIngress/2",
                "ExpectedValue": "(not present)",
                "ActualValue": "{\"IpProtocol\":\"tcp\",\"FromPort\":22,\"ToPort\":22,\"CidrIp\":\"0.0.0.0/0\"}",
                "DifferenceType": "ADD"
            }
        ]
    }
]

Drift Status Values

Status Meaning
IN_SYNC Resource matches the template
MODIFIED Resource has been changed outside CloudFormation
DELETED Resource has been deleted outside CloudFormation
NOT_CHECKED Drift detection hasn’t been run on this resource

Fixing Drift

CloudFormation doesn’t auto-fix drift. You have two options:

Option How When
Update template to match reality Add the manual change to your template, then update-stack The manual change is intentional
Re-apply the template Update the stack with the original template to revert The manual change was a mistake

Automating Drift Detection

Set up a scheduled check with EventBridge + Lambda:

  EventBridge (cron: daily at 9 AM)
       │
       ▼
  Lambda: calls detect-stack-drift
       │
       ▼
  If DRIFTED → SNS notification → Slack / email

Common Mistakes

Mistake Why It Hurts Fix
Updating stacks without change sets Surprise replacements delete data Always create a change set first
Ignoring Replacement: True Resource gets deleted and recreated Review change sets carefully for replacements
Never running drift detection Manual changes accumulate silently Schedule weekly/daily drift checks
Manual console changes to CFN-managed resources Template becomes out of sync Make ALL changes through CloudFormation

Quick Reference

Task Command
Create change set aws cloudformation create-change-set --stack-name STACK --change-set-name CS --template-body file://t.yml
Describe change set aws cloudformation describe-change-set --stack-name STACK --change-set-name CS
Execute change set aws cloudformation execute-change-set --stack-name STACK --change-set-name CS
Delete change set aws cloudformation delete-change-set --stack-name STACK --change-set-name CS
Detect drift aws cloudformation detect-stack-drift --stack-name STACK
View drift results aws cloudformation describe-stack-resource-drifts --stack-name STACK

Next → CDK — Cloud Development Kit

Updated: