A Guide to Deploy Code to Warm Pool Instances with AWS CodeDeploy

Introduction

In dynamic cloud environments, it’s crucial to have efficient and automated processes to manage the lifecycle of your Amazon EC2 instances, especially when leveraging warm pools for cost efficiency.

Warm pools allow you to have pre-initialized instances that can quickly be brought online, but managing code deployment to these instances can be challenging.

In this blog post, we’ll walk you through a comprehensive approach to automating code deployment to Amazon EC2 instances within a warm pool using AWS CodeDeploy. We’ll use AWS Lambda to manage the lifecycle of these instances, ensuring that they are in the correct state during deployment and return to a hibernated state afterward. This automation simplifies the deployment process and ensures that your applications are always up-to-date with minimal manual intervention.

Objectives

Create resources using the AWS CloudFormation template.
Set up AWS Lambda function that:
Identifies Amazon EC2 instances based on their tags.
Manages the lifecycle of instances in a warm pool.
Triggers deployments using AWS CodeDeploy.
Ensure instances run during deployment and return to a stopped state after deployment.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Prerequisites

Basic knowledge of AWS services such as Amazon EC2, AWS Lambda, AWS CloudFormation, and AWS CodeDeploy.
AWS CLI and necessary permissions to create and manage AWS resources.

Step-by-Step Guide

Step 1: AWS CloudFormation Template

We will start by creating the necessary resources using AWS CloudFormation template. This template sets up:

An Amazon EventBridge rule to trigger the Lambda function when an Amazon EC2 instance changes state.
Necessary AWS IAM roles and permissions.
The AWS Lambda function itself.

AWS CloudFormation Template

{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "This CloudFormation template creates resources required for syncing new instances with the latest deployment from AWS CodeDeploy based on the CodeDeployAppname, DeploymentGroup tags on the instance",
"Resources": {
"EventRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"Description": "EventRule",
"EventPattern": {
"source": [
"aws.ec2"
],
"detail-type": [
"EC2 Instance State-change Notification"
],
"detail": {
"state": [
"running"
]
}
},
"State": "ENABLED",
"Targets": [
{
"Arn": {
"Fn::GetAtt": [
"CodeDeploySyncNewInstance",
"Arn"
]
},
"Id": "TargetFunctionV1"
}
]
}
},
"PermissionForEventsToInvokeLambda": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"FunctionName": {
"Ref": "CodeDeploySyncNewInstance"
},
"Action": "lambda:InvokeFunction",
"Principal": "events.amazonaws.com",
"SourceArn": {
"Fn::GetAtt": [
"EventRule",
"Arn"
]
}
}
},
"CodeDeployServiceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole"
],
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"codedeploy.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
}
}
},
"CodeDeployLambdaRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
"arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess"
],
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com",
"lambda.amazonaws.com"
]
],
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "Lambda-codedeploy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "StartContinuousAssessmentLambdaPolicyStmt",
"Effect": "Allow",
"Action": [
"codedeploy:Get*",
"codedeploy:UpdateDeploymentGroup",
"codedeploy:List*",
"codedeploy:CreateDeployment"
],
"Resource": [
"*"
]
}
]
}
},
{
"PolicyName": "IamPassRole",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "StartContinuousAssessmentLambdaPolicyStmt",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
{
"Fn::GetAtt": [
"CodeDeployServiceRole",
"Arn"
]
}
]
}
]
}
}
]
}
},
"CodeDeploySyncNewInstance": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Role": {
"Fn::GetAtt": [
"CodeDeployLambdaRole",
"Arn"
]
},
"Environment": {
"Variables": {
"CodeDeployRole": {
"Fn::GetAtt": [
"CodeDeployServiceRole",
"Arn"
]
}
}
},
"Code": {
"ZipFile": {
"Fn::Join": [
"\n",
[
"def handler(event, context):",
"    return 'Hello from Lambda'"
]
]
}
},
"Runtime": "python3.8",
"Timeout": 300,
"Handler": "new-instance-code-sync.lambda_handler",
"MemorySize": 512
}
}
},
"Outputs": {
"NewInstanceLambdaFunction": {
"Description": "The Lambda function that creates Resources for syncing new instances",
"Value": {
"Ref": "CodeDeploySyncNewInstance"
}
}
}
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

{

"AWSTemplateFormatVersion": "2010-09-09",

"Description": "This CloudFormation template creates resources required for syncing new instances with the latest deployment from AWS CodeDeploy based on the CodeDeployAppname, DeploymentGroup tags on the instance",

"Resources": {

"EventRule": {

"Type": "AWS::Events::Rule",

"Properties": {

"Description": "EventRule",

"EventPattern": {

"source": [

"aws.ec2"

"detail-type": [

"EC2 Instance State-change Notification"

"detail": {

"state": [

"running"

]

}

"State": "ENABLED",

"Targets": [

{

"Arn": {

"Fn::GetAtt": [

"CodeDeploySyncNewInstance",

"Arn"

]

"Id": "TargetFunctionV1"

}

]

}

"PermissionForEventsToInvokeLambda": {

"Type": "AWS::Lambda::Permission",

"Properties": {

"FunctionName": {

"Ref": "CodeDeploySyncNewInstance"

"Action": "lambda:InvokeFunction",

"Principal": "events.amazonaws.com",

"SourceArn": {

"Fn::GetAtt": [

"EventRule",

"Arn"

]

}

"CodeDeployServiceRole": {

"Type": "AWS::IAM::Role",

"Properties": {

"ManagedPolicyArns": [

"arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole"

"AssumeRolePolicyDocument": {

"Version": "2012-10-17",

"Statement": [

{

"Effect": "Allow",

"Principal": {

"Service": [

"codedeploy.amazonaws.com"

]

"Action": [

"sts:AssumeRole"

]

}

]

}

"CodeDeployLambdaRole": {

"Type": "AWS::IAM::Role",

"Properties": {

"ManagedPolicyArns": [

"arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",

"arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess"

"AssumeRolePolicyDocument": {

"Version": "2012-10-17",

"Statement": [

{

"Effect": "Allow",

"Principal": {

"Service": [

"ec2.amazonaws.com",

"lambda.amazonaws.com"

]

"Action": [

"sts:AssumeRole"

]

}

]

"Path": "/",

"Policies": [

{

"PolicyName": "Lambda-codedeploy",

"PolicyDocument": {

"Version": "2012-10-17",

"Statement": [

{

"Sid": "StartContinuousAssessmentLambdaPolicyStmt",

"Effect": "Allow",

"Action": [

"codedeploy:Get*",

"codedeploy:UpdateDeploymentGroup",

"codedeploy:List*",

"codedeploy:CreateDeployment"

"Resource": [

"*"

]

}

]

}

{

"PolicyName": "IamPassRole",

"PolicyDocument": {

"Version": "2012-10-17",

"Statement": [

{

"Sid": "StartContinuousAssessmentLambdaPolicyStmt",

"Effect": "Allow",

"Action": [

"iam:PassRole"

"Resource": [

{

"Fn::GetAtt": [

"CodeDeployServiceRole",

"Arn"

]

}

]

}

]

}

]

}

"CodeDeploySyncNewInstance": {

"Type": "AWS::Lambda::Function",

"Properties": {

"Role": {

"Fn::GetAtt": [

"CodeDeployLambdaRole",

"Arn"

]

"Environment": {

"Variables": {

"CodeDeployRole": {

"Fn::GetAtt": [

"CodeDeployServiceRole",

"Arn"

]

}

"Code": {

"ZipFile": {

"Fn::Join": [

"\n",

[

"def handler(event, context):",

" return 'Hello from Lambda'"

]

}

"Runtime": "python3.8",

"Timeout": 300,

"Handler": "new-instance-code-sync.lambda_handler",

"MemorySize": 512

}

"Outputs": {

"NewInstanceLambdaFunction": {

"Description": "The Lambda function that creates Resources for syncing new instances",

"Value": {

"Ref": "CodeDeploySyncNewInstance"

}

Explanation

This template sets up the following:

An Amazon EventBridge rule to trigger the Lambda function when an EC2 instance state changes to running.
The necessary AWS IAM roles with permissions to allow the Lambda function to interact with Amazon EC2 and AWS CodeDeploy.
The AWS Lambda function itself will handle the deployment logic.

Step 2: AWS Lambda Function for Deployment

Next, we will set up the AWS Lambda function. This function will:

Detect when an instance in the warm pool changes to the running state.
Use the instance’s tags to identify the application and deployment group.
Trigger the AWS CodeDeploy deployment.
Return the instance to the stopped state once deployment is complete.

AWS Lambda Function Code

from __future__ import print_function
from dateutil.parser import parse
from datetime import datetime
import json
import time
import os
import boto3
 
ec2client = boto3.client('ec2')
client = boto3.client('codedeploy')
codedeployrole = os.environ.get("CodeDeployRole")
 
def lambda_handler(event, context):
    print("Received event: " + json.dumps(event, indent=2, default=str))
    instance_id = event['detail']['instance-id']
    
    try:
        # Ensure the instance is running
        start_instance(instance_id)
        wait_for_instance_running(instance_id)
        
        tags = get_instance_info(ec2client, instance_id)
        
        name = tags.get('Name')
        if not name:
            raise CustomException("Name tag not found on the instance")
        
        codedeploygroup = f"{name}-deployment-group-asg"
        appname = f"{name}-application"
        
        depresponse = deployment_group_tag(appname, codedeploygroup)
        print("Deployment group response: " + json.dumps(depresponse, indent=2, default=str))
        
        ec2TagFilters = depresponse['deploymentGroupInfo'].get('ec2TagFilters', [])
        if not ec2TagFilters:
            print("Warning: No ec2TagFilters found in deployment group. Adding default tag filter based on Name.")
        
        ec2TagFilters = [tag for tag in ec2TagFilters if not (tag['Key'] == 'Name' and tag['Value'] == name)]
        ec2TagFilters.append({
            'Key': 'Name',
            'Value': name,
            'Type': 'KEY_AND_VALUE'
        })
        
        update_response = deployment_group_update(appname, codedeploygroup, ec2TagFilters)
        print("Deployment group update response: " + json.dumps(update_response, indent=2, default=str))
        
        deploymentId = depresponse['deploymentGroupInfo']['lastSuccessfulDeployment']['deploymentId']
        print("DeploymentId is " + deploymentId)
        
        depIdResponse = get_dep(deploymentId)
        bucket = depIdResponse['s3Location']['bucket']
        key = depIdResponse['s3Location']['key']
        
        syncresp = deploy_now(appname, codedeploygroup, bucket, key)
        print(syncresp)
        
        # Stop the instance after deployment
        stop_instance(instance_id)
        
        return 'Done'
    except CustomException as ce:
        print("Custom exception: " + str(ce))
        return 'Done'
    except KeyError as ke:
        print("Key error: " + str(ke))
        return 'Done'
    except Exception as e:
        print("Unhandled exception: " + str(e))
        return 'Done'
 
def start_instance(instance_id):
    ec2client.start_instances(InstanceIds=[instance_id])
    
def stop_instance(instance_id):
    ec2client.stop_instances(InstanceIds=[instance_id])
 
def wait_for_instance_running(instance_id):
    waiter = ec2client.get_waiter('instance_running')
    waiter.wait(InstanceIds=[instance_id])
 
def get_instance_info(ec2client, instance_id):
    response = ec2client.describe_tags(
        Filters=[
            {
                'Name': 'resource-id',
                'Values': [instance_id]
            }
        ]
    )
    
    tagdict = {}
    for tag in response['Tags']:
        tagdict[tag['Key']] = tag['Value']
    
    return tagdict
 
def deployment_group_tag(appname, codedeploygroup):
    response = client.get_deployment_group(
        applicationName=appname,
        deploymentGroupName=codedeploygroup
    )
    return response
 
def deployment_group_update(appname, codedeploygroup, ec2TagFilters):
    response = client.update_deployment_group(
        applicationName=appname,
        currentDeploymentGroupName=codedeploygroup,
        ec2TagFilters=ec2TagFilters,
        serviceRoleArn=codedeployrole
    )
    return response
 
def get_dep(deploymentId):
    response = client.get_deployment(deploymentId=deploymentId)
    return response['deploymentInfo']['revision']
 
def deploy_now(appname, codedeploygroup, bucket, key):
    response = client.create_deployment(
        applicationName=appname,
        deploymentGroupName=codedeploygroup,
        deploymentConfigName='CodeDeployDefault.OneAtATime',
        description='Test',
        ignoreApplicationStopFailures=True,
        updateOutdatedInstancesOnly=True,
        fileExistsBehavior='OVERWRITE'
    )
    return response
 
class CustomException(Exception):
    """Raise for my specific kind of exception"""

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

from __future__ import print_function

from dateutil.parser import parse

from datetime import datetime

import json

import time

import os

import boto3

ec2client = boto3.client('ec2')

client = boto3.client('codedeploy')

codedeployrole = os.environ.get("CodeDeployRole")

def lambda_handler(event, context):

print("Received event: " + json.dumps(event, indent=2, default=str))

instance_id = event['detail']['instance-id']

try:

# Ensure the instance is running

start_instance(instance_id)

wait_for_instance_running(instance_id)

tags = get_instance_info(ec2client, instance_id)

name = tags.get('Name')

if not name:

raise CustomException("Name tag not found on the instance")

codedeploygroup = f"{name}-deployment-group-asg"

appname = f"{name}-application"

depresponse = deployment_group_tag(appname, codedeploygroup)

print("Deployment group response: " + json.dumps(depresponse, indent=2, default=str))

ec2TagFilters = depresponse['deploymentGroupInfo'].get('ec2TagFilters', [])

if not ec2TagFilters:

print("Warning: No ec2TagFilters found in deployment group. Adding default tag filter based on Name.")

ec2TagFilters = [tag for tag in ec2TagFilters if not (tag['Key'] == 'Name' and tag['Value'] == name)]

ec2TagFilters.append({

'Key': 'Name',

'Value': name,

'Type': 'KEY_AND_VALUE'

})

update_response = deployment_group_update(appname, codedeploygroup, ec2TagFilters)

print("Deployment group update response: " + json.dumps(update_response, indent=2, default=str))

deploymentId = depresponse['deploymentGroupInfo']['lastSuccessfulDeployment']['deploymentId']

print("DeploymentId is " + deploymentId)

depIdResponse = get_dep(deploymentId)

bucket = depIdResponse['s3Location']['bucket']

key = depIdResponse['s3Location']['key']

syncresp = deploy_now(appname, codedeploygroup, bucket, key)

print(syncresp)

# Stop the instance after deployment

stop_instance(instance_id)

return 'Done'

except CustomException as ce:

print("Custom exception: " + str(ce))

return 'Done'

except KeyError as ke:

print("Key error: " + str(ke))

return 'Done'

except Exception as e:

print("Unhandled exception: " + str(e))

return 'Done'

def start_instance(instance_id):

ec2client.start_instances(InstanceIds=[instance_id])

def stop_instance(instance_id):

ec2client.stop_instances(InstanceIds=[instance_id])

def wait_for_instance_running(instance_id):

waiter = ec2client.get_waiter('instance_running')

waiter.wait(InstanceIds=[instance_id])

def get_instance_info(ec2client, instance_id):

response = ec2client.describe_tags(

Filters=[

{

'Name': 'resource-id',

'Values': [instance_id]

}

]

)

tagdict = {}

for tag in response['Tags']:

tagdict[tag['Key']] = tag['Value']

return tagdict

def deployment_group_tag(appname, codedeploygroup):

response = client.get_deployment_group(

applicationName=appname,

deploymentGroupName=codedeploygroup

)

return response

def deployment_group_update(appname, codedeploygroup, ec2TagFilters):

response = client.update_deployment_group(

applicationName=appname,

currentDeploymentGroupName=codedeploygroup,

ec2TagFilters=ec2TagFilters,

serviceRoleArn=codedeployrole

)

return response

def get_dep(deploymentId):

response = client.get_deployment(deploymentId=deploymentId)

return response['deploymentInfo']['revision']

def deploy_now(appname, codedeploygroup, bucket, key):

response = client.create_deployment(

applicationName=appname,

deploymentGroupName=codedeploygroup,

deploymentConfigName='CodeDeployDefault.OneAtATime',

description='Test',

ignoreApplicationStopFailures=True,

updateOutdatedInstancesOnly=True,

fileExistsBehavior='OVERWRITE'

)

return response

class CustomException(Exception):

"""Raise for my specific kind of exception"""

Explanation

AWS Lambda Handler: Main function that processes the event, extracts the instance ID, and triggers the deployment process.
Instance Management: Functions to start, stop, and wait for the instance state changes.
AWS CodeDeploy Management: Functions interacting with AWS CodeDeploy, including retrieving and updating deployment groups and triggering deployments.

Conclusion

Using this setup, you can automate code deployment to instances in a warm pool, ensuring they run during the deployment process and return to a stopped state afterward. This approach leverages the power of AWS services like AWS Lambda, AWS CloudFormation, and AWS CodeDeploy to create and scalable deployment pipeline.

You can easily manage and trigger deployments by tagging instances with appropriate metadata, ensuring your applications are always up-to-date with minimal manual intervention.

Drop a query if you have any questions regarding AWS CodeDeploy and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, AWS GenAI Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, Amazon ECS Service Delivery Partner, AWS Glue Service Delivery Partner, Amazon Redshift Service Delivery Partner, AWS Control Tower Service Delivery Partner, AWS WAF Service Delivery Partner and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. What are the benefits of using a warm pool in an Auto Scaling Group?

ANS: – Warm pools allow you to maintain a set of pre-initialized Amazon EC2 instances that can be quickly brought into service, reducing the time it takes to scale your application. By keeping instances in a stopped state, you save on costs, as you’re only billed for storage rather than compute. This approach is particularly beneficial for applications that experience sudden spikes in traffic and need to scale rapidly.

2. How does AWS CodeDeploy interact with instances in a warm pool?

ANS: – AWS CodeDeploy typically deploys code to instances as they enter an Auto Scaling Group. In warm pools, instances are brought from a stopped state to a running state before deployment. The AWS Lambda function handles this transition, ensuring that the instance is ready for deployment and returning it to a stopped state afterward. This integration allows you to maintain consistency in your deployments without manual intervention.

WRITTEN BY Deepak S

Deepak S works as a Research Intern at CloudThat. His expertise lies in AWS's services. Deepak is good at haunting new technologies and automobile enthusiasts.