logo
  • Blog
  • The Cloud
  • DevOps
  • Trending
    • Blockchain
    • Bitcoin
  • How To
  • Kubernetes

AWS Lambda – Copy more than 5 EC2 Snapshots automatically between regions

Posted on June 6, 2018
Published on: June 06, 2018 | Last updated on: June 06, 2018 |
Author: Pooja Verma Pooja Verma (Git, Twitter)

 9,616 total views,  4 views today

AWS Lambda – Copy more than 5 EC2 Snapshots automatically between regions was last modified: January 17th, 2019 by Pooja Verma
Share on Facebook
Facebook
0Tweet about this on Twitter
Twitter
Share on LinkedIn
Linkedin
Email this to someone
email

AWS Lambda – Copy more than 5 EC2 Snapshots automatically between regions

https://timesofcloud.com

Copying EC2 snapshot between regions is quite an easy task now. In some scenarios, we may have to copy more than 5 snapshots automatically without any human entration.

Snapshot copy operation has a limitation of copying max 5 snapshots at one time. Hence you can not copy more than 5 snapshots at a time.

Following are the steps to automate to copy more than 5 Snapshots.

Let’s say, we have around 50 snapshots in a region, and you want to automate to copying all Snapshots to another region on AWS.

Step 1: Create a tag in all 50 Snapshots like Key=Backup, and Value=Yes

Step 2: Use below filter for getting all Snapshots yet to copy.
Filters=[{‘Name’:’tag:Backup’,’Values’:[‘Yes’]}]

Step 3: After getting all Snapshots with applied filter, run a for loop (max 5 iteration) and start copying only 5 Snapshots.

Step 4: Once the copy done, change the original Snapshot tag as follow:
Key=Backup, and Value=Yes —- To Key=Backup, and Value=Completed

Next time when this Lambda function will run, it will fetch all Snapshot except Completed on.

Step 5: To run this Lambda function, create a CloudWatch scheduler, and call this lambda function every 10 minutes.

Following is Lambda function:

import boto3
import time

region_source = 'us-east-1'
region_dest = 'ap-southeast-1'

client_source = boto3.client('ec2',region_name=region_source)
client_dest = boto3.client('ec2',region_name=region_dest)
ec2_resource = boto3.resource('ec2')

#Getting all snapshots as per specified filter
def get_snapshots():
    response = client_source.describe_snapshots(
                    Filters=[{'Name':'tag:Backup','Values':['Yes']}]
                )
    return response["Snapshots"]

#Creating a list of snapshot_id and respective tags
def get_snapshot_list(snapshots):
    snapshot_list = []
    for snapshot in snapshots:
        snapshot_id = snapshot["SnapshotId"]
        tags = snapshot["Tags"]
        snapshot_list.append((snapshot_id, tags))
    return snapshot_list

#Copying snapshot with tags
def copy_snapshot(snapshot_id, tags):
    print "Started copying.. snapshot_id: " + snapshot_id + ", from: " + region_source + ", to: " + region_dest
    
    try:
        copy_response = client_dest.copy_snapshot(
            Description='copied from us-east-1',
            SourceRegion=region_source,
            SourceSnapshotId=snapshot_id,
            DryRun=False
        )
    except Exception, e:
        raise e

    new_snapshot_id = copy_response["SnapshotId"]
    print 'new_snapshot_id : ' + new_snapshot_id
    new_snapshot = ec2_resource.Snapshot(new_snapshot_id)
    
    #Creating tags in new snapshort
    tag = new_snapshot.create_tags(
        DryRun=False,
        Tags=tags
    )
    create_tag(client_dest, new_snapshot_id, 'Source Snapshot Id', snapshot_id)
    delete_tag(client_source, snapshot_id, 'Backup', 'Yes')
    create_tag(client_source, snapshot_id, 'Backup', 'Completed')
    print "Successfully copyied.. snapshot_id: " + snapshot_id + ", from: " + region_source + ", to: " + region_dest
    return new_snapshot_id
    
    
def delete_tag(client, snapshot_id, key, value):
    print "Deleting tag - " + key ":" + value + ", snapshot_id: " + snapshot_id
    client.delete_tags(
        Resources=[snapshot_id],
        Tags=[
            {
            'Key': key,
            'Value':value
            },
        ]
    )

def create_tag(client, snapshot_id, key, value):
    print "Creating tag - " + key ":" + value + ", snapshot_id: " + snapshot_id
    client.create_tags(
        Resources=[snapshot_id],
        Tags=[
            {
                'Key': key,
                'Value':value 
            },
        ]
    )

def lambda_handler(event, context):
    snapshots = get_snapshots()
    snapshot_list = get_snapshot_list(snapshots)
    
    i = 0
    for snapshort in snapshot_list:
        if i < 5:
            snapshot_id = snapshort[0]
            tags = snapshort[1]
            new_snapshot_id = copy_snapshot(snapshot_id, tags)
            i = i + 1
        else:
            exit(0)

 9,615 total views,  3 views today

AWS Lambda – Copy more than 5 EC2 Snapshots automatically between regions was last modified: January 17th, 2019 by Pooja Verma
Share on Facebook
Facebook
0Tweet about this on Twitter
Twitter
Share on LinkedIn
Linkedin
Email this to someone
email
Posted in Amazon EC2, EC2 Lambda | 2 Comments
« AWS Lambda – Copy EC2 Snapshot automatically between regions
Kubernetes Logging Architecture »

2 thoughts on “AWS Lambda – Copy more than 5 EC2 Snapshots automatically between regions”

  1. kartik says
    November 14, 2019 at 11:32 am

    Hii, Please provide the IAM role used by the Lambda function.
    I would be a great help

    Reply
  2. Kartik says
    November 14, 2019 at 10:42 am

    Great article! But which version of the python is this and what role have to be assigned to lambda function

    Reply

Leave a comment Cancel reply

Your email address will not be published.

Latest Blogs

  • Linux: Viewing Log Messages
  • AWS CodeBuild: Getting Started
  • AWS CodeCommit: Set up Notifications
  • AWS CodeCommit: Securing The Repository and Branches
  • Managing Systemd units in Linux

Tags

Amazon EC2 AWS bash_shell bitcoin blockchain Cloud computing CodeCommit DevOps digital currency Kubernetes Linux trending ubuntu

For Improving Education

Categories

  • Amazon EC2
  • Amazon Web Services
  • AWS
  • AWS CodeCommit
  • Bash shell
  • Best Practices
  • Bitcoin
  • Blockchain
  • Chaincode
  • CLI
  • Cloud Computing
  • Cloud Security
  • CodeBuild
  • CodeCommit
  • CryptoCurrency
  • Cryptography
  • DevOps
  • Digital Currency
  • EC2 Lambda
  • Hyperledger
  • IBM Bluemix
  • IBM Garage
  • Kubernetes
  • Linux
  • Monitoring
  • OpenStack
  • Platform as a Service
  • TDD
  • Trending
  • Ubuntu
  • Virtual Server
Home | Site Map | Privacy Policy | Site Terms | Terms of use @2013, Times of Cloud.
The content is copyrighted to 'Times of Cloud' and may not be reproduced on other websites.