Singularity: Container Runtime for Research and Scientific Workloads

Introduction

A container runtime is a software component for launching and managing containers within a containerization platform, such as Docker or Kubernetes. It provides the necessary environment and resources for containers to run isolated and securely on a host machine. The container runtime interacts with the host operating system’s kernel to create lightweight, isolated environments known as containers. It manages the lifecycle of containers, including creating, starting, stopping, and deleting them. The runtime also handles containerization’s resource allocation, networking, storage, and security aspects.

Some popular container runtimes

Several container runtimes are available, each with its implementation and features. Here are some popular container runtimes:

1. Docker Engine: Docker is one of the most well-known containerization platforms, and Docker Engine is its default container runtime. It uses Linux kernel features like cgroups and namespaces for process isolation and resource management.
2. containerd: containerd is a lightweight, open-source container runtime originally part of Docker. It has since become a standalone project and serves as the core container runtime for Kubernetes.
3. CRI-O: CRI-O is a lightweight and optimized container runtime designed for Kubernetes. It follows the Kubernetes Container Runtime Interface (CRI) specification, enabling seamless integration with the Kubernetes ecosystem.
4. runc: runc is a command-line tool and a lightweight runtime for containers. It is the default runtime used by containerd and implements the Open Container Initiative (OCI) runtime specification.
5. cri-containerd: cri-containerd is another container runtime that adheres to the CRI specification. It is built on top of containerd and provides a runtime interface for Kubernetes.

Singularity

Singularity was developed to address the needs of researchers and scientists who often require running complex simulations or analysis tools on HPC clusters. Some of the key features and characteristics of Singularity include:

1. Compatibility: Singularity containers are designed to be compatible with the host operating system. This means that users can run Singularity containers directly on the host without requiring elevated privileges or modifications to the host system.
2. User-driven: Singularity enables users to create, manage, and distribute their containers without requiring administrative privileges. This allows researchers to encapsulate their scientific workflows and share them easily with collaborators.
3. Security: Singularity emphasizes security by providing a strong isolation boundary between the container and the host system. It utilizes user namespaces and other Linux kernel features to ensure that the containerized application cannot access or modify resources outside the container.
4. High-performance computing: Singularity is optimized for HPC environments, supporting technologies such as InfiniBand and MPI (Message Passing Interface). It allows researchers to take advantage of the performance capabilities of HPC clusters without sacrificing containerization benefits.

Singularity containers use the “.sif” file format, a compressed file containing the necessary files and metadata for the container. Users can create Singularity containers from scratch or convert existing Docker or OCI (Open Container Initiative) containers into Singularity format.

Steps to Install Singularity Installation on Linux

Containers are predominantly supported on Linux due to the kernel features such as namespaces that facilitate their functionality. To install Singularity on either a bare metal Linux system or a Linux virtual machine (VM), you can follow the steps outlined below:

Setting-up VM:

In the AWS Account and go to the Amazon EC2 instances section.

Choose Launch Instances

Select the OS of your choice, here AmazonLinux has chosen.

Select the instance type and keypair.

Select the Create Security Group and keep the default rules as shown below, or you can select the existing SG if you already have the same rules. After that, click on Launch Instances.

After the successful creation of the Instance, you will get the landing page and copy the public Ip from there:

Connect to the instance and follow the procedure mentioned below.

Installation of singularity:

If you already have a previous version of Singularity installed, removing it before proceeding with the installation commands is recommended. To install Singularity from a source, you must have certain development tools installed on your system. In case you encounter any missing dependencies during the installation process, you can try installing them using the following method:

Install a specific release

The commands below will install the release from the available GitHub releases to /usr/local.

Building an Image Using the Singularity:

Downloading an existing container from Docker Hub

The “build” command in Singularity allows you to fetch layers from Docker Hub and construct Singularity containers by assembling these layers. This process enables you to utilize pre-existing Docker images and convert them into Singularity containers. By leveraging the “build” command, Singularity simplifies the creation of containers by leveraging the extensive library of Docker images available on Docker Hub.

Building containers from Singularity recipe files

Singularity recipe files can generally be used as targets while building containers. For any information needed while writing Singularity recipe files, refer to the Container Recipes docs.  For example, let’s consider having the container recipe file called xingsing, as shown below, and for building a container by using it:

You can do so with the following command.

Conclusion

Singularity is a powerful container runtime for Linux that offers various benefits and use cases in research computing, scientific simulations, HPC, bioinformatics, machine learning, and more. When setting up Singularity on a Linux system, it is important to consider the following key points:

• Compatibility: Singularity is primarily designed to run on Linux, and leverages kernel features such as namespaces to support containers effectively.
• Singularity Recipe Files: Singularity recipe files allow you to define the construction process of containers, including the base image, software dependencies, environment variables, and execution commands. These recipe files can be used to build containers or customize existing ones.

Drop a query if you have any questions regarding Singularity and I will get back to you quickly.

About CloudThat

CloudThat is an official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner and Microsoft Gold Partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best in industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

To get started, go through our Consultancy page and Managed Services Package that is CloudThat’s offerings.