A Guide to Migrate a PostgreSQL Database between Kubernetes Clusters

Introduction

Migrating a PostgreSQL database from one Kubernetes cluster to another can be a critical task, but with careful planning and execution, it can be accomplished smoothly. This blog post will explore a step-by-step guide for migrating a PostgreSQL database installed as a stateful set on Kubernetes from one cluster to another. Specifically, we will focus on taking a database dump, pushing it to Amazon S3, and restoring it to the target cluster. This migration process allows you to seamlessly transfer your PostgreSQL database while leveraging the existing PostgreSQL application in the target cluster.

In our situation, we have an application operating on Kubernetes that utilizes a Postgres database. We aim to move this application from one cluster to another. We have successfully installed the same application on the target cluster and now need to transfer the data stored in the Postgres database of the source cluster.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Pre-Migration Preparations

Setting up the target Kubernetes cluster
Ensuring compatibility between clusters
Configuring storage resources for the migrated database
Establishing network connectivity and security between clusters

Provisioning Resources in the Target Cluster

Deploying the required resources (pods, services, secrets, etc.) in the target cluster
Configuring the PostgreSQL Helm chart or manifests for the target environment
Ensuring proper security and access control for the database

Steps to migrate the Postgres database

Step 1 – Connect to the source cluster and copy the PostgreSQL service IP address.

step1

Step 2 – Run the Ubuntu pod in the same namespace where Postgres is running

$ kubectl run -it ubuntu --image=ubuntu -n dev – bash

1	$ kubectl run -it ubuntu --image=ubuntu -n dev – bash

Update the ubuntu pod to get the latest utilities.

$ apt-get update

1	$ apt-get update

Install the PostgreSQL client in the ubuntu pod

$ apt-get install postgresql-client

1	$ apt-get install postgresql-client

Step 3 – Install the AWS CLI

$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

1	$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

step3

Unzip the downloaded file

$ apt install unzip
$ unzip awscliv2.zip

1 2	$ apt install unzip $ unzip awscliv2.zip

Install the AWSaLI

$ ./aws/install

1	$ ./aws/install

Once the AWS CLI is installed, you must grant Amazon S3 access to Ubuntu. This can be done by attaching a role to the pod or utilizing IAM credentials with Amazon S3 access.

Step 4 – Take database dump

We must take the existing database dump and save it on Amazon S3. Replace the IP address for your Postgres service when you run the command below. Once this command has been run, a prompt will ask you for your database password. Once you enter the password, the database dump will begin.

Syntax:

pg_dump -U <username-of-database -h <Postgres-service-IP> -F t <database-name> <dump-name.tar>
$ pg_dump -U outline -h 10.100.69.226 -F t outline > outline-22-06-2023.tar

1 2	pg_dump -U <username-of-database -h <Postgres-service-IP> -F t <database-name> <dump-name.tar> $ pg_dump -U outline -h 10.100.69.226 -F t outline > outline-22-06-2023.tar

After executing the above command, you can see the tar file below

Verify the integrity of the dump file by extracting and examining its contents.

Step 5 – Pushing the dump to Amazon S3

Push the dump file to Amazon S3 using the AWS CLI

Syntax:

$ aws s3 cp <dump-filename> s3://<bucket-name>/directory-name/

1	$ aws s3 cp <dump-filename> s3://<bucket-name>/directory-name/

Restoring Database Dump

In the destination cluster where the database needs to migrate, please carry out steps 1 through 3

Step 6 – Download the database dump from Amazon S3

Syntax:

aws s3 cp s3:// <bucket-name>/directory-name /<filename> <file-name>

1	aws s3 cp s3:// <bucket-name>/directory-name /<filename> <file-name>

Once the file is downloaded from Amazon S3, you can see the file below in the Ubuntu pod.

Step 7 – Restore the database from the dump

Before restoring the database, we need to create the database.

Execute the following command to connect the database, and give the password in the prompt

Syntax:

$ psql -h <service-IP> -p <port> -U <DB-username>
$ psql -h 172.20.155.81 -p 5432 -U outline

1 2	$ psql -h <service-IP> -p <port> -U <DB-username> $ psql -h 172.20.155.81 -p 5432 -U outline

Execute the below command to create the database.

$ CREATE DATABASE outlinedemo;

1	$ CREATE DATABASE outlinedemo;

step7

To restore the database from the dump, follow the below

Syntax

$ pg_restore -h <service-IP> -p 5432 -U <DB-username> -d <databasename> <dump-filename>
$ pg_restore -h 172.20.155.81 -p 5432 -U outline -d outlinedemo outline-22-06-2023.tar

1 2	$ pg_restore -h <service-IP> -p 5432 -U <DB-username> -d <databasename> <dump-filename> $ pg_restore -h 172.20.155.81 -p 5432 -U outline -d outlinedemo outline-22-06-2023.tar

Verify the restored database on the target cluster to ensure data integrity and consistency.

Conclusion

Migrating a PostgreSQL database from one Kubernetes cluster to another while leveraging an existing PostgreSQL application on the target cluster is achievable with a well-defined process. By following this step-by-step guide, which includes taking a database dump, pushing it to Amazon S3, and restoring it on the target cluster, you can ensure a seamless transition of your database.

To verify data integrity, thoroughly test the restored database on the target cluster. With careful execution and attention to detail, you can successfully migrate your PostgreSQL database and use the new cluster’s infrastructure.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is an official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner and Microsoft Gold Partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best-in-industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

Drop a query if you have any questions regarding PostgreSQL database, Kubernetes, I will get back to you quickly.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. Can I migrate my PostgreSQL database between Kubernetes clusters without downtime?

ANS: – Yes, you can minimize downtime during the migration process by setting up replication between the source and destination clusters. This allows for continuous data synchronization and reduces the impact on your application.

2. What considerations should I consider while selecting the destination cluster for migration?

ANS: – When choosing the destination cluster, ensure it meets the requirements, such as sufficient resources, compatible network connectivity, and appropriate security configurations. Additionally, verify that the Kubernetes versions align to prevent compatibility issues.

3. How can I ensure the integrity and reliability of the migrated PostgreSQL database?

ANS: – Validating data integrity post-migration is essential. Execute comprehensive tests and queries to verify that all indexes, constraints, and data types remain intact and functional. Promptly address any discrepancies to maintain the integrity of your database.

WRITTEN BY Harikrishnan S

Harikrishnan Seetharaman is a Research Associate (DevOps) at CloudThat. He completed his Bachelor of Engineering degree in Electronics and Communication, and he achieved AWS solution architect-Associate certification. His area of interest is implementing a cloud-native solution for customers and helping them by proving robust and reliable solutions for their complex problems, DevOps, and SaaS. Apart from his professional interest he likes to spend time in farming and learning new DevOps tools.