Deployment of CI/CD Pipeline for AWS CloudFormation

1. Introduction to CloudFormation

As developers work on multiple clients or for a group that supports various AWS deployments, we may often find ourselves challenged with updating AWS CloudFormation. We can build a CI/CD pipeline to automate the process of updating CloudFormation. AWS CodePipeline can initiate a workflow that builds and tests a stack and then pushes it to production. The workflow can either create or manipulate an existing stack.

Today, you will learn how to deploy a Continuous Integration and Continuous Delivery (CI/CD) Pipeline on the Amazon Web Services (AWS) Cloud and automatically test AWS CloudFormation templates from a version control system like Bitbucket.

Customized Cloud Solutions to Drive your Business Success

Cloud Migration
Devops
AIML & IoT

Know More

2. Overview of the Solution

We have implemented a fully functional CI/CD that updates the CloudFormation template. The pipeline starts automatically after applying the intended changes to the Bitbucket repository.

The following diagram illustrates the solution architecture.

3. The Three Stages of a CI/CD pipeline:

Source stage: When a commit is made into the source branch of the Bitbucket repository, it triggers the CI/CD pipeline. Here in this blog, we deploy AWS Security Group through the CloudFormation template as an example.

cft_security_group.json

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "CloudFormation Template For Setting up the Services required for SellerService",
    
    "Resources": {
        "SecurityGroup": {
            "Type": "AWS::EC2::SecurityGroup",
            "Properties": {
                "GroupName": "CFT-Test-SG",
                "GroupDescription": "security group for ",
                "VpcId": "vpc-dfa0a9b6",
                "SecurityGroupIngress": [
                    {"IpProtocol": "tcp","FromPort": "80","ToPort": "80","CidrIp":"10.0.0.0/16"},
                    {"IpProtocol": "tcp","FromPort": "22","ToPort": "22","CidrIp":"10.0.1.0/24"},
                    {"IpProtocol": "tcp","FromPort": "443","ToPort": "443","CidrIp":"10.0.0.0/16"}
                ],
                "SecurityGroupEgress": [
                    {"IpProtocol": "-1","CidrIp": "0.0.0.0/0"}
                ]
            }
        }
    }
}

{

"AWSTemplateFormatVersion": "2010-09-09",

"Description": "CloudFormation Template For Setting up the Services required for SellerService",

"Resources": {

"SecurityGroup": {

"Type": "AWS::EC2::SecurityGroup",

"Properties": {

"GroupName": "CFT-Test-SG",

"GroupDescription": "security group for ",

"VpcId": "vpc-dfa0a9b6",

"SecurityGroupIngress": [

{"IpProtocol": "tcp","FromPort": "80","ToPort": "80","CidrIp":"10.0.0.0/16"},

{"IpProtocol": "tcp","FromPort": "22","ToPort": "22","CidrIp":"10.0.1.0/24"},

{"IpProtocol": "tcp","FromPort": "443","ToPort": "443","CidrIp":"10.0.0.0/16"}

"SecurityGroupEgress": [

{"IpProtocol": "-1","CidrIp": "0.0.0.0/0"}

]

}

Build/test stage: In CodeBuild, it installs the cfn-lint and cfn-nag tools. Then, it checks the CloudFormation template using the two tools before deploying it to CloudFormation.

CloudFormation cfn-lint

Using cfn-lint enables syntax error checks on your CloudFormation Template. To check your template, you run the below command. You are assuming that our template is written in JSON format.

cfn-lint cft_security_group.json

1	cfn-lint cft_security_group.json

CloudFormation cfn-nag

The cfn-nag tool is for security checks. It examines the CloudFormation Template for any insecure infrastructure, e.g., security groups that allow access for everyone. To check your template, you run the below command. Assuming that our template is written in JSON format.

cfn_nag_scan cft_security_group.json

1	cfn_nag_scan cft_security_group.json

Buildspec.yaml

version: 0.2
phases:
  install:
    runtime-versions:
        ruby: 2.6    
    commands:
    - pip3 install awscli --upgrade --quiet
    - pip3 install cfn-lint --quiet              
    - apt-get install jq git -y -q
    - gem install cfn-nag
  build:
    commands:
    - cd ./
    - cfn-lint cft_security_group.json
    - cfn_nag_scan -i cft_security_group.json

version: 0.2

phases:

install:

runtime-versions:

ruby: 2.6

commands:

- pip3 install awscli --upgrade --quiet

- pip3 install cfn-lint --quiet

- apt-get install jq git -y -q

- gem install cfn-nag

build:

commands:

- cd ./

- cfn-lint cft_security_group.json

- cfn_nag_scan -i cft_security_group.json

Deploy stage: The deploy stage deploys the template to CloudFormation.

a. AWS CodePipeline:

AWS CodePipeline is a fully managed continuous delivery service that helps automate release pipelines for fast and reliable application and infrastructure updates. CodePipeline automates the release process’s build, test, and deploy phases every time there is a code change, based on the release model we define. It enables us to rapidly and reliably deliver features and updates. In addition, it helps us easily integrate AWS CodePipeline with third-party services such as GitHub, Jenkins, Sonarqube, or our custom plugin.

b. AWS CodeBuild:

AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages ready to deploy. In addition, CodeBuild helps to provision, manage, and scale our build servers. CodeBuild scales continuously and processes multiple builds concurrently.

c. Bitbucket:

Bitbucket Cloud is a Git-based code hosting and collaboration tool built for teams. Bitbucket provides three hosting options: Cloud, Server, and Data Center. We are using Bitbucket cloud here, which is accessed via a URL.

Thus, we have explored how to use CodePipeline, CodeBuild, and Bitbucket to update an existing CloudFormation.

4. Step-By-Step Solution Workflow:

Developers integrate changes into the main branch hosted within a bitbucket repository
CodePipeline polls the source code repository and triggers the pipeline to run when a new version is detected
CodePipeline runs a build of the latest revision in CodeBuild to validate and test the CloudFormation template
The CloudFormation template with resources is updated or created
Verify that the changes were applied successfully

5. Benefits of deploying CI/CD Pipeline on AWS

CI/CD provides benefits for software development teams like

Automating the process
Improving developer productivity
Improving code quality
Delivering updates to your customers faster

6. Takeaway

Continuous integration and delivery provide an ideal scenario for the organization’s application and Infra teams. The Infra team can push Cloud Formation templates to the repository. This template will be integrated, tested, deployed, tested again, merged with infrastructure, and go through security and quality reviews.

Get your new hires billable within 1-60 days. Experience our Capability Development Framework today.

Cloud Training
Customized Training
Experiential Learning

About CloudThat

CloudThat is the official AWS Advanced Consulting Partner, Microsoft Gold Partner, and Training partner, helping people develop knowledge on the cloud and help their businesses aim for higher goals using best in industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

If you have any queries about Continuous Integration and Continuous Delivery, CI/CD Pipeline deployment, AWS services, or anything related to consulting, feel free to drop in a comment, and we will get back to you quickly.