Private runners for GitLab

Overview

GitLab runners are pivotal components in software development, serving as the execution engine for tasks defined in YAML files within the GitLab environment. These runners operate as isolated virtual machines, effectively picking up assigned jobs and executing them precisely. Understanding the two types of GitLab runners—shared and specific—is essential in this context.

By providing isolated virtual environments for job execution, GitLab runners play a crucial role in optimizing development pipelines, making them an integral part of modern software development practices.

Introduction

GitLab runners serve as essential components for executing YAML files within the GitLab ecosystem. Operating as isolated virtual machines, these runners efficiently pick up assigned tasks and execute them by the specifications outlined in the YAML files. They are pivotal in automating processes and streamlining workflows within the GitLab environment. :

1. Shared Runners

Shared runners are available for all the projects and can run any job for multiple projects. A few runners will always be available for your project to run the job.

2. Specific Runners

Specific runners or private runners are restricted to particular projects and can also be available for multiple projects. You need to enable the runner for that particular project. A dedicated runner is beneficial as it will fulfill the high demand for CI activity for a particular project as a priority.

Private Runner creation

1. Go to project -> settings -> CI/CD -> Runners -> New Project Runner

2. Once you click on New Project Runner, you can set the following settings as shown below:

• Select the OS of the runner you want to use. By default, we are using Linux.
• Add tags for the runner; these will be those we have given in the ci.yaml so that it will prioritize the job of that particular tag.
• You can also select the Run untagged jobs check box to accept all the jobs.
• Under configuration, you can select the Paused option to pause the runner for taking up the job.
• Then, you can Create a runner.

3. Once the Runner is created, you will get the Runner token, as shown below. Copy that token, which will be used later in EKS.

NOTE: If you don’t copy the runner token, it won’t be available later, and you can’t register this runner in EKS.

Integrating Private Runner in EKS

1. First, we need to install the Gitlab runner in EKS using Helm.

• Add the GitLab runner Helm repo

• Then update the Helm repo by using the below command

• Now pull the GitLab runner helm chart. Once you pull it, it will create a directory with the name gitlab-runner with all yaml files and values.yaml

2. Before installing the helm chart, make the following changes in the yaml under the gitlab-runner directory.

• Scroll down and uncomment gitlabUrl and runnerToken Now give the gitlab URL and runner token we got at the runner creation time.

• Now, scroll down more to the rbac section and make create: true. It will be false by default, and uncomment the rules section as shown below:

• Now, come to the runners section by scrolling down, adding the below configuration as shown, and saving the values.yaml file:

3. Before installing the helm chart, create a namespace for gitlab runner by executing the below command:

Now install the GitLab runner:

4. You can check the status of the gitlab runner pod:

Docker configuration in GitLab

Since we are using docker to build our image, there are a few configurations

1. Go to project -> settings -> CI/CD -> variables and add the following variables and their corresponding values.

• Here DOCKER_HOST represents a connection endpoint for Docker’s remote API using the TCP protocol. It typically points to the Docker daemon running on a remote server.

2. Now, in the gitlab-ci.yaml do the following changes for the image stage as shown below

Here, we are updating the docker and docker dind to the latest version and adding tags that we have given to our runner and variable to specify the cert path.

• You must make the following changes for the deployment stage, as shown below.

Here we are giving only tags and updating the docker dind to the latest version.

Final Check

Whenever you run your job, the runner will pick the job, and you can see the temporary pod running inside the cluster, which will be terminated once the job is completed. Here, the pod named runner is the temporary pod.

Conclusion

GitLab runners enhance the efficiency and automation of software development pipelines by providing isolated virtual environments for executing tasks defined in YAML files. Their ability to seamlessly integrate with GitLab projects makes them indispensable for ensuring the smooth execution of various jobs and contributing to a more streamlined development process.

Drop a query if you have any questions regarding GitLab runners and we will get back to you quickly.

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, Microsoft Gold Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.