How AWS DataSync Helps in Data Transfer?

AWS DataSync: An overview

AWS DataSync was used for migrating large amounts of data from on-premises to AWS and to transfer data between AWS storage devices. Now a new capability is added in AWS DataSync to migrate the data from either Google Cloud Storage or Microsoft Azure Files to AWS Storage services. This helps to exchange data between AWS and other Public Clouds used by vendors or customers, or partners to store their data. This data migration is done using end-to-end security, including data validation and encryption.

How does it work?

• Select the source Cloud provider (Google Cloud Storage or Microsoft Azure Files) from where we want to migrate the data.
• Create and deploy the AWS DataSync agent.
• Configure the endpoints, source location, and destination location through the AWS DataSync console.
• Data transfer starts between the other public location and the AWS Storage service selected.

Image Source: https://aws.amazon.com/

Simple steps for implementation

• From the VPC console, choose Endpoints on the navigation pane and create the VPC endpoint.
• In the AWS console, from the navigation pane, select Agent and then Create Agent
• In the Google Cloud console, open bucket settings and create a service account with Storage Object Viewer permissions and write down the credentials to programmatically access the bucket.
• Now, in AWS DataSync Console, choose Tasks and Create Task.

As given below, in the configuration, select Location type as Object storage, choose the agent created in step no.2, enter the domain name or IP address of object storage server, and also enter the google cloud bucket name and folder where objects to be transferred are stored.

Image Source: https://aws.amazon.com/

• Provide credentials for authentication.
• Configure destination location as S3 with required configurations as given below.

Image Source: https://aws.amazon.com/

• Complete the Task execution configuration and Data transfer configuration.
• Select the schedule to run this task (Like Hourly, Daily, Weekly, etc.)
• Select the log group to store logs and complete the steps.
• After a few minutes, the files from Google Object Storage are synced with Amazon S3, and you can access them from the console.

Key Features of AWS Sync

• Purpose-built Network Protocol.
• Automatic Infrastructure Management.
• Bandwidth Optimization and Control.
• Data Transfer Scheduling.
• Data Encryption and Validation.
• Monitoring and Auditing with Amazon CloudWatch and AWS CloudTrail.

Conclusion

The AWS DataSync can be useful when you are looking for secure data transfer of a large amount of data between AWS and other public locations like Google Cloud Storage and Microsoft Azure Files in a simplified way.

About CloudThat

CloudThat is a cloud-agnostic organization. We are AWS Advanced Consulting Partner, Microsoft Gold Partner and Training partner, Google Cloud Partner, and Authorized VMware Training Reseller, empowering organizations and professionals with cloud skills. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

FAQs

1. What is AWS DataSync?

Answer- It is an AWS service that lets you securely migrate your large amount data securely and in a simplified way from your premises or other public location to AWS storage services.

2. Where can I move the data from?

Answer- We can copy the data from Network File System (NFS) shares, Server Message Block (SMB) shares, Hadoop Distributed File System (HDFS), Google Cloud Storage, Azure Files, Amazon EFS, Amazon FSx, Amazon S3, and self-managed object storage.

WRITTEN BY Abhijit Dilip Powar