Scaling Amazon EC2 Instances Dynamically Using Amazon SQS Messages

Overview

In this blog, we have provided a comprehensive overview of the steps in setting up an Auto Scaling group to respond dynamically to changes in system load using the Amazon SQS ApproximateNumberOfMessagesVisible metric. From configuring Amazon CloudWatch alarms for Amazon SQS messages to creating a Launch Template and Auto Scaling group, each step has been detailed to guide users through the implementation process effectively.

Introduction

AWS Auto Scaling efficiently oversees application capacity, guaranteeing stable performance while optimizing cost-efficiency. By seamlessly adjusting resource allocation across diverse services, AWS Auto Scaling simplifies setup processes, allowing for swift deployment in just minutes.

Moreover, it empowers organizations to adapt to evolving demands and maintain optimal performance levels without manual intervention.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Steps to create Auto Scaling group for Amazon SQS based on the Number of Messages Visible

Once the Messages are configured in the Amazon SQS service, we need to create Amazon CloudWatch Alarms for Amazon SQS Messages.

1. Creation of Amazon CloudWatch Alarm

In Amazon CloudWatch Alarm, select All alarms.
Click Create alarm.
Click on Select metric. Once the metric is clicked, scroll down to the Browse section and select Amazon SQS

step1

Click on Queue Metrics, select the Suitable Metric name for Amazon SQS Messages, and click on Select metric

step1b

Select the Metric and Conditions according to the need and click on Next.

step1c

In the Configure actions page, keep everything default and click on Next
Enter the Alarm name ScaleOut_alarm, click Next, and Click Create Alarm

step1d

2. Steps to create ScaleInAlarm

Select the ScaleOut_alarm, go to Actions, select copy option
In the specify metric conditions page, change the alarm condition to Lower < threshold after that, set the threshold value, and click Next
Enter the Alarm name, ScaleIn_alarm and click Next

step2

In the Configure actions page, keep everything default, click Next, and click On Create an Alarm.

Creation of Launch Template

Launch Templates or Launch Configurations, Launch Templates emerge as the preferred option due to their versatility and flexibility in managing multiple versions, allowing for easier tracking and maintenance of configuration changes over time. This capability enables teams to streamline deployment processes and ensure consistency across different environments. Additionally, by leveraging Launch Templates, organizations can enhance scalability and adaptability while optimizing resource utilization and minimizing operational overhead.

Creating a Launch Template for the Auto Scaling group, specifying instance characteristics and creation methods.

First, we need to create an Image of the Amazon EC2 instance.

step3

Then, select Launch template and click on Create Launch template
Enter the name for the launch template
Under Application and OS Images, select My AMIs and select the appropriate AMI.

step3b

Select the Instance type and key pair.
Under Network settings, select the subnet and security group.
Under Advanced details, select AWS IAM instance profile.
Click on Create Launch template.

4. Creation Of Auto Scaling Group.

Enter the name of the Autoscaling group.
Select the launch template created in the above step.

step4

Click Next, select VPC and Subnets under Network, and click N
Under Configuration and Advanced options, select No load balancer, remaining keep it default, and click
Under Configure group size and scaling page, set the desired capacity scaling limits and select Next

step4b

Review the configuration and create an Auto Scaling Group
Once the AutoScaling Group is created, select the Auto Scaling Group and select the option Automatic scaling.
Under Automatic scaling, we need to create two dynamic policy
click on create dynamic scaling policy.
Select Policy type has simple scaling
Enter the scaling policy name
Select the Amazon CloudWatch alarm created for both scale in scale out
Under Take the action, select the appropriate action and seconds.

Below are the Dynamic policy for CloudWatch Alarm

ScaleIn_alarm

step4c

ScaleOut_alarm

step4d

Now we can verify the queue messages and an auto-scaling group launch with a maximum of 3 instances.

step4e

We can also navigate to Amazon CloudWatch and check the status of Alarm

The ScaleOut_alarm status will be in alarm state, and the ScaleIn_alarm status will be OK.

As the ScaleOut_alarm enters the Alarm state, it is expected that approximately 10 minutes or later, we will observe the deployment of three Amazon EC2 instances within the Amazon EC2 Auto Scaling group SQS_ASG, as indicated in the Activity section.

Conclusion

Hence, following the above steps, we can establish a scalable and responsive infrastructure that adapts seamlessly to changing workload conditions. Additionally, monitoring the status of alarms through Amazon CloudWatch provides valuable insights into the health and performance of the system, enabling proactive management and optimization.

By implementing dynamic scaling based on Amazon SQS messages, organizations can achieve improved reliability, scalability, and cost-efficiency in their AWS environments, ultimately enhancing the overall user experience and driving business success.

Drop a query if you have any questions regarding Amazon SQS and we will get back to you quickly.

Empowering organizations to become ‘data driven’ enterprises with our Cloud experts.

Reduced infrastructure costs
Timely data-driven decisions

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. How do you create Amazon CloudWatch alarms for SQS messages?

ANS: – Amazon CloudWatch alarms are created by selecting the appropriate Amazon SQS metrics, defining conditions, and configuring actions based on threshold values.

2. How often are Amazon CloudWatch alarms evaluated for triggering scaling events?

ANS: – Amazon CloudWatch alarms evaluate their associated metrics regularly, typically every minute, and trigger scaling events based on the defined threshold conditions.

WRITTEN BY Abhilasha D

Abhilasha D is a Research Associate-DevOps at CloudThat. She is focused on gaining knowledge of Cloud environment and DevOps tools. She has keen interest in learning and researching on emerging technologies.