AWS for DevOps — S3 for DevOps Workflows

5.3 — S3 for DevOps Workflows

S3 is the backbone of DevOps workflows on AWS — artefact storage, Terraform state backends, static website hosting, and log aggregation.

CI/CD Artefact Storage

A typical pipeline stores build outputs in S3:

  CodeBuild                    S3 Bucket
  ┌──────────┐    upload      ┌─────────────────────┐
  │ Build Job │──────────────►│ artefacts/           │
  │           │               │   v1.2.3/app.tar.gz  │
  └──────────┘               │   v1.2.4/app.tar.gz  │
                              └──────────┬────────────┘
                                         │ download
                              ┌──────────▼────────────┐
                              │ CodeDeploy / EC2       │
                              │ (pulls artefact)       │
                              └────────────────────────┘

# In your buildspec.yml, upload the artefact
$ aws s3 cp target/app.tar.gz \
    s3://my-artefacts/releases/${CODEBUILD_RESOLVED_SOURCE_VERSION}/app.tar.gz

# In the deploy phase, download it
$ aws s3 cp \
    s3://my-artefacts/releases/${COMMIT_SHA}/app.tar.gz \
    /opt/app/app.tar.gz

Terraform State Backend

Store Terraform state in S3 with DynamoDB locking:

# Create the state bucket
$ aws s3 mb s3://my-terraform-state-2026

# Enable versioning (critical — you can roll back state)
$ aws s3api put-bucket-versioning \
    --bucket my-terraform-state-2026 \
    --versioning-configuration Status=Enabled

# Enable encryption
$ aws s3api put-bucket-encryption \
    --bucket my-terraform-state-2026 \
    --server-side-encryption-configuration '{
        "Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}]
    }'

# Block public access
$ aws s3api put-public-access-block \
    --bucket my-terraform-state-2026 \
    --public-access-block-configuration \
        'BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true'

Static Website Hosting

Host a documentation site or build status dashboard:

# Enable website hosting
$ aws s3 website s3://my-docs-site/ \
    --index-document index.html \
    --error-document error.html

# Upload the site
$ aws s3 sync ./docs/build/ s3://my-docs-site/

# The site is available at:
# http://my-docs-site.s3-website.ap-south-1.amazonaws.com

For production, put CloudFront in front for HTTPS and caching.

Presigned URLs

Generate a temporary URL that allows anyone to download (or upload) without AWS credentials:

# Generate a download URL (valid for 1 hour)
$ aws s3 presign s3://my-artefacts/releases/v1.2.3/app.tar.gz \
    --expires-in 3600
https://my-artefacts.s3.amazonaws.com/releases/v1.2.3/app.tar.gz?X-Amz-...

# Generate an upload URL
$ aws s3 presign s3://my-artefacts/uploads/report.pdf \
    --expires-in 3600

Use cases: share build artefacts with external testers, upload logs from on-premise servers.

S3 Event Notifications

Trigger actions when objects are created or deleted:

$ aws s3api put-bucket-notification-configuration \
    --bucket my-artefacts \
    --notification-configuration '{
        "LambdaFunctionConfigurations": [
            {
                "LambdaFunctionArn": "arn:aws:lambda:ap-south-1:123456789012:function/process-artefact",
                "Events": ["s3:ObjectCreated:*"],
                "Filter": {
                    "Key": {
                        "FilterRules": [
                            {"Name": "prefix", "Value": "releases/"},
                            {"Name": "suffix", "Value": ".tar.gz"}
                        ]
                    }
                }
            }
        ]
    }'

This triggers a Lambda function whenever a new .tar.gz file is uploaded to the releases/ prefix.

S3 Access Logging

Log all access to a bucket for audit:

$ aws s3api put-bucket-logging \
    --bucket my-artefacts \
    --bucket-logging-status '{
        "LoggingEnabled": {
            "TargetBucket": "my-access-logs",
            "TargetPrefix": "artefacts-logs/"
        }
    }'

Quick Reference

Task	Command
Upload artefact	aws s3 cp FILE s3://BUCKET/KEY
Sync directory	aws s3 sync DIR s3://BUCKET/PREFIX/
Generate presigned URL	aws s3 presign s3://BUCKET/KEY --expires-in SECONDS
Enable website	aws s3 website s3://BUCKET/ --index-document index.html
Set notifications	aws s3api put-bucket-notification-configuration --bucket BUCKET --notification-configuration file://notif.json
Enable logging	aws s3api put-bucket-logging --bucket BUCKET --bucket-logging-status file://logging.json

---

Next → EBS Volumes & Snapshots