Achieving High Availability with MongoDB Replication in Docker on Amazon EC2

Introduction

MongoDB replication is fundamental to deploying a resilient and reliable database system. By synchronizing data across multiple servers, replication ensures no single point of failure, significantly enhancing the overall availability of the database. In a replicated MongoDB setup, data is distributed across a group of servers known as a replica set. This replica set architecture is designed to provide data redundancy and operational continuity, even during hardware or software failures.

A typical MongoDB replica set includes a primary node and one or more secondary nodes. The primary node handles all write operations, ensuring data consistency and integrity. Secondary nodes continuously replicate the data from the primary, maintaining up-to-date copies that can be used to balance read traffic or take over as the primary in case of an outage. This automatic failover mechanism is a critical feature, as it allows the database to self-heal and continue operations without manual intervention, thus minimizing downtime and ensuring that applications remain available to users.

Prerequisites

Amazon EC2 Instance: Launch an Amazon EC2 instance with a modern Linux distribution (e.g., Ubuntu), at least 4GB of RAM, and adequate CPU resources.

Docker and Docker Compose: Install Docker and Docker Compose on your Amazon EC2 instance. Follow Docker’s installation guide for Ubuntu and Docker Compose installation guide.

Directory Structure and Volumes: Create the necessary directory structure on your Amazon EC2 instance:

/newvolume/data/mongodb/data/mongodb

/newvolume/data/mongodb/mongodb-logs

/newvolume2/data/mongodb-replica/data/mongodb

/newvolume2/data/mongodb-replica/mongodb-logs

Security Groups: Ensure your Amazon EC2 instance’s security groups allow traffic on the required ports (27017 and 27018) for MongoDB communication.

Implementation

Step 1:

Create the new server with the ami which was taken for the MongoDB server, and attach the new volume to the MongoDB server, which doesn’t have the data.

In the above mentioned volume, 281 is the new volume  which doesn’t have the data.

Step 2:

Mount the volume that has  data  to the /newvolume/data/mongodb

Step 3:

For the newly created volume, firstly, we need to format the  volume before mounting it  to the  directories

command: sudo mkfs.ext4 /dev/nvme2n1
mount-command:  sudo mount /dev/nvme2n1  /newvolume2/data/mongodb-replica

Step 4:

Now bind the 2 volumes because if any data is getting added to the volume of MongoDB, it needs  to be moved to another volume as well

command: sudo mount –bind /newvolume/data/mongodb/ /newvolume2/data/mongodb-replica/

Step 5:

Update the docker-compose file below

docker-compose.yml

Step 6:

Restart the docker-compose

Command: sudo docker-compose up -d
command to login into docker-container: sudo docker exec -it fc-mongodb bash

Once login into the container, run the below command

Command: mongo

Step 7:

After connecting to the Mongo shell, run the below command
command: rs.initiate()
command:  rs.add(“fc-mongodb-replica:27017”)
command: rs.status()

Step 8:

After this, log into the secondary container
command: sudo docker exec -it fc-mongodb-replica bash
command: rs.slaveOk()
command:  rs.status()

Here in the status, we can see how many files are getting copied to the secondary container

Once all files are copied, startup2 will be changed to the secondary replica.

rs0:STARTUP2>

rs0:SECONDARY>

Benefits of MongoDB Replication

• High Availability: Automatic failover capabilities ensure that one of the secondary nodes is promoted to primary if the current primary node fails, thus maintaining the database’s availability without manual intervention.
• Data Redundancy: By keeping multiple copies of the data across different nodes, MongoDB replication protects against data loss.
• Improved Read Performance: Read operations can be distributed across multiple nodes, significantly improving read performance, especially in read-heavy applications.
• Disaster Recovery: With data replicated across different nodes, possibly in different geographic locations, MongoDB replication can be a disaster recovery solution.

Conclusion

Docker’s portability and isolation ensure that MongoDB instances can be consistently managed across various environments, from development to production. This setup not only enhances the resilience and reliability of your database by providing automatic failover capabilities but also simplifies the process of scaling your infrastructure to meet increasing demands.

Furthermore, Docker Compose streamlines the management and orchestration of your MongoDB replica set, reducing complexity and allowing for more efficient configuration and deployment. Maintaining a consistent environment across all stages of the application lifecycle minimizes the risk of configuration errors and ensures predictable performance. Overall, deploying MongoDB replication in Docker containers on Amazon EC2 offers a robust, scalable, and easily manageable solution that meets the demands of modern application development and operations.

Drop a query if you have any questions regarding MongoDB and we will get back to you quickly.

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.