Continuous Integration to Automate Terraform Modules With GitHub Actions As Infrastructure-as-Code Pipelines

1. Introduction

GitHub Actions is a continuous integration and continuous delivery (CI/CD) tool offered by GitHub. It is open-source and one of the fastest-growing tools that allow you to automate your build, test, and releases. It is a fast deployment tool with well-founded security, less overhead, and makes you write workflows easily.

Let us assume that we want to create a workflow that gets triggered when an issue is raised. The workflow will contain multiple jobs in its virtual machine runner and performs one or more steps that you define or run an action. So what are these workflow, events, jobs ??

2. The Components of GitHub Actions

• Workflows: The workflow is an automated process that runs one or more jobs. You can define a workflow in a YAML file in your repository, and it will run when triggered by a specific event. We can have multiple workflows created in a repository to perform different roles.
• Events: Events are an activity that occurs in your repository that triggers the workflow, for example, pushing a commit, creating pull requests.
• Jobs: A job is a set of instructions executed on the runner. An instruction can be a script or an action that will run. Jobs will be executed in sequence or parallelly. We can also share data from one step to another since they are dependent on each other. The job dependencies can be handled with other jobs; by default, jobs are executed in parallel with no dependencies.
• Actions: It is a custom application that can be used to reduce the repetitive code in the workflow files. We can use the actions available in the GitHub marketplace and can also write our own actions.
• Runners: It is a backend server on which the workflow runs. Each workflow will be executed on a newly provisioned runner.

You can deep dive into the GitHub actions here

3. Example for Setting up a Workflow

Consider setting up a workflow that gets triggered when the code is pushed into the main branch. And the workflow jobs which automate the terraform to deploy Azure resources.

4. Prerequisites

1. You need to have a GitHub account and Actions should be enabled
2. Azure credentials
   • Azure Tenant ID
   • Azure subscription ID
   • We need an Azure Service Principle with a Contributor role
   • An azure Service principle’s ClientID and Secrets. Terraform will use this service principle to authenticate with Azure for creating resources
   • I will be using an Azure storage account as my terraform backend in this setup, so I have created a resource group with a storage account and container.
3. Terraform configuration files to create resources. Here, I am creating the azure resources, but you can deploy resources in any cloud of your choice.

5. Introduction to Terraform

Terraform is an Infrastructure as a Code tool that allows you to build and manage both cloud and on-premises resources in human-readable configuration files which you can share, reuse, and version. The configuration files are written in terraform HCL ( Hashicorp Configuration Language)

So Let’s begin the setup by creating a repository for our pipeline

1. Login in to your GitHub account and create a repository
   
2. Push your terraform code into the repository. Here is my terraform code, I have modules to create a Resource group, Azure Virtual Network, subnet, and a security group for the dev environment
   
3. Next, add the service principle’s secrets to your repository. Go to Settings on your repository. Under Secrets, click on Actions, and add a New repository secret. Provide a name to your secret, add its value, and click Add secret. Repeat this process to add all the required credentials
   
   
   
4. Now, we need to enable the GitHub Actions for our account, for that navigate to your repository, click Actions and search for terraform and click on the Configure. There are other third-party actions are also available here I’m going with Hashicorp.
   
   
5. Once you click the Configure, you will be taken to the terraform.yml file, the workflow configuration file
   
6. We need to make changes to the yaml file for our pipeline. Then, we need to set up the jobs to run the terraform code to deploy the Azure resources.
7. In the yaml file, the below code snippet shows that the workflow will trigger when a push is made on the main branch.
   name: 'Terraform' on: push: branches: - main

   1 2 3 4 5

   name: 'Terraform' on:   push:     branches:     - main

   Next, I have defined the job and given the steps that a job has to perform.

   Here terraform is the job that runs on ubuntu runner, and steps are the instructions which a job will perform. We can define multiple steps under a job. The secrets which are stored are referenced using the environment env variable, and the values are fetched from the Action’s secret during the execution.

   jobs: terraform: name: 'Terraform' runs-on: ubuntu-latest environment: dev steps: # Checkout the repository to the GitHub Actions runner - name: Checkout uses: actions/checkout@main # Install the latest version of Terraform CLI - name: Setup Terraform uses: hashicorp/setup-terraform@v1

   1 2 3 4 5 6 7 8 9 10 11 12 13

   jobs:   terraform:     name: 'Terraform'     runs-on: ubuntu-latest     environment: dev     steps:     # Checkout the repository to the GitHub Actions runner     - name: Checkout       uses: actions/checkout@main       # Install the latest version of Terraform CLI     - name: Setup Terraform       uses: hashicorp/setup-terraform@v1

   - name: Terraform Init run: terraform init working-directory: "./environments/dev" env: ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID}} ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}} ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}} ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET}}

   1 2 3 4 5 6 7 8

   - name: Terraform Init       run: terraform init       working-directory: "./environments/dev"       env:         ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID}}         ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}}         ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}}         ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET}}

8. Below is our final workflow configuration for our pipeline
   name: 'Terraform' on: push: branches: - main jobs: terraform: name: 'Terraform' runs-on: ubuntu-latest environment: dev steps: # Checkout the repository to the GitHub Actions runner - name: Checkout uses: actions/checkout@main # Install the latest version of Terraform CLI - name: Setup Terraform uses: hashicorp/setup-terraform@v1 - name: Terraform Init run: terraform init working-directory: "./environments/dev" env: ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID}} ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}} ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}} ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET}} - name: Terraform Format run: terraform fmt -check - name: Terraform Plan run: terraform plan -out terraform.plan -var-file=terraform.tfvars -lock=false working-directory: "./environments/dev" env: ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID}} ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}} ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}} ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET}} - name: Terraform Apply id: apply run: terraform apply -var-file=terraform.tfvars -auto-approve -lock=false working-directory: "./environments/dev" env: ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID}} ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}} ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}} ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET}}

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

   name: 'Terraform' on: push: branches: - main jobs: terraform: name: 'Terraform' runs-on: ubuntu-latest environment: dev steps: # Checkout the repository to the GitHub Actions runner - name: Checkout uses: actions/checkout@main     # Install the latest version of Terraform CLI     - name: Setup Terraform       uses: hashicorp/setup-terraform@v1     - name: Terraform Init       run: terraform init       working-directory: "./environments/dev"       env:         ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID}}         ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}}         ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}}         ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET}}        - name: Terraform Format       run: terraform fmt -check       - name: Terraform Plan run:      terraform plan -out terraform.plan -var-file=terraform.tfvars -lock=false       working-directory: "./environments/dev"       env:         ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID}}         ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}}         ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}}         ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET}}     - name: Terraform Apply       id: apply       run:         terraform apply -var-file=terraform.tfvars -auto-approve -lock=false       working-directory: "./environments/dev"       env:         ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID}}         ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}}         ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}}         ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET}}

   We can check the pipeline execution logs here

   
   

9. Once the pipeline is succeeded, we can verify the azure resources created

6. Conclusion

With that, this is how we can integrate terraform with GitHub actions and provision our infrastructure with a single GitHub action workflow in multiple environments. GitHub actions let you run your workflows when an event occurs in your repository. To run your workflows, GitHub offers Linux, Windows, and Mac OS virtual machines, and you can also run your workflows in your data centers or on the cloud.

About CloudThat

CloudThat is the official Microsoft Gold Partner, AWS Advanced Consulting Partner, and Training partner helping people develop knowledge on the cloud and help their businesses aim for higher goals using best in industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.