Overview of AWS Data Transfer: Cost Optimization, Online and Offline Data Transfer Services

1. Overview:

Suppose if a global application has noticed that they are gaining users in Asia, it’s better to host the application in a region close to Asia for a better user experience. If an application is used in a particular region (For example, Government or Corporate sites) that holds sensitive data, it can only be hosted/stored in that region by law. Also, creating a backup of data in different regions to avoid data loss during natural disasters.

These are some scenarios where data transfer can improve application performance and enhance user experience enormously. Amazon provides comprehensive data transfer services in different Availability Zones.

Let us take a closer look into AWS services for online and offline data transfer, optimize costs, pricing, and other useful aspects.

2. Introduction

AWS data transfer occurs each time data is moved from AWS to the Internet or between AWS instances in their respective regions or Availability Zones.

Data transfers between regions and Availability Zones are charged and billed per gigabyte. Costs depend on geographic location, source location, destination location, AWS services or resources, and the amount of data transferred. Data transfer works both Online and Offline. Its usage depends on the size of data, the time required, pricing, frequency, and bandwidth available.

3. AWS Data Transfer Over the Internet

Data-In from the Internet:  Data transfer from the Internet to AWS is free in any of the AWS Regions, Direct Connect, or Edge Locations.

Data-Out to the Internet:  Data is sent over the Internet and billed at a region-specific, and rates are categorized. These charges consider all aggregated data sent by all the following services: Amazon RDS, Amazon Redshift, Amazon SES, Amazon SimpleDB, Amazon EBS, Amazon S3, Amazon Glacier, Amazon SQS, AWS Storage Gateway, Amazon SNS, Amazon DynamoDB, and Amazon CloudWatch Logs.

4. Data Transfer Costs Within the AWS Cloud

There are different ways to move data around the AWS Clouds.

a. Inter-Region Data Transfer: Inter-region, data transfer rates, are billed at the source region rate.

b. Data Transfers between AWS Regions and Direct Connect Locations: There is no AWS data transfer cost in any AWS regions. We must follow the general rule that it is free to transfer data from the Internet to AWS, but there is a charge for transferring data from the AWS region to a Direct Connect Location.

c. Data Transfer between Edge Locations and AWS Regions: The Edge Location is a data center belonging to Amazon CloudFront CDN. These location cache data include video, API, or applications for low latency access. The transfer from the AWS area to the Edge location is free.

d. AWS Data Transfer Costs Within the Same Region: AWS resources or services interact with each other within the same region, and charges may apply. It is a type of AWS data transfer cost that can be a bit tricky to track and understand.

Example: If we want to transfer 100 GB of data between a Redshift cluster in VPC-a and an EC2 instance in VPC-b, both in the same Availability Zone, we would be charged $5 for egress from VPC-a and $5 for ingress into VPC-b.

Using the same case, if both reside in the same VPC but different Availability Zones, we would be charged the same.

5. Cost Optimization with AWS data transfer:

1. Use of Efficient Routes
2. Control the size of data
3. Avoid NAT devices in VPCs
4. Use of Amazon CloudFront
5. Track services like Amazon ElasticCache, Amazon CloudSearch, and Amazon MSK for their data transfer rates.
6. Avoid the use of Public IPs for data transfer.

6. Factors to consider before selecting a data transfer method:

1. What is the size of the data?
2. Format of data (Small or Large Files, whether the file is compressed on not and how it is stored)
3. Source location (Whether it’s near to AWS Region)
4. Privacy and Security for data

7. Services used for Online Data Transfer:

1. AWS Virtual Private Network
2. AWS Database Migration Service
3. AWS Direct Connect
4. AWS S3 Transfer Acceleration
5. AWS DataSync

8. Services used for Offline Data Transfer:

1. AWS Snowball
2. AWS Snowball Edge
3. AWS Snowmobile

9. Pricing:

a. Direct Connect: It helps us establish a dedicated connection between their AWS resources.

b. EC2 Data Transfer:

Data Transfer from EC2 to the Internet:

Data Transfer from EC2 to Other regions:

c. ElastiCache Data Transfer: There is no charge for traffic in or out of an ElastiCache, charges only come from traffic in or out of an instance associated with ElastiCache. The standard rate is $0.01 per GB.

d. RDS Data Transfer:

Data Transfer into RDS:

Data Transfer out of RDS:

e. Data Transfer within AWS Region:

f. Free Data Transfer within an AWS Region:

AWS Pricing Calculator

10. Takeaway

We have many ways to move our data in and out of the cloud across multiple availability zones and regions with few clicks. If the architecture is not appropriately modeled, the cost might skyrocket. We should keep track of the expenses with data analysis and follow best practices to keep costs under control.

11. About CloudThat

CloudThat is the official AWS Advanced Consulting Partner, Microsoft Gold Partner, and Training partner helping people develop knowledge on the cloud and help their businesses aim for higher goals using best in industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

If you have any queries about Amazon Machine Learning Services, Artificial Intelligence on AWS, or anything related to AWS services, feel free to drop in a comment. We will get back to you quickly. Visit our Consulting Page for more updates on our customer offerings, expertise, and cloud services.

12. FAQs

Q1. Is the data encrypted while being transferred and stored?

Yes. All data transferred between the source and destination is encrypted via Transport Layer Security (TLS), which replaced the Secure Sockets Layer (SSL). Data is never persisted in AWS DataSync itself.

Q2. How does Snowball Edge physically secure my data?

In addition to using a tamper-resistant enclosure, Snowball Edge uses industry-standard Trusted Platform Modules (TPM) to detect unauthorized modifications to the hardware, firmware, or software. AWS visually and cryptographically inspects every device for any signs of tampering and to verify that the TPM detected no changes.

WRITTEN BY Rahul Kumar Sharma