Login- Consulting
- Training
- Partners
- About Us
x
Public Transportation
Amazon EC2, Amazon S3, Amazon RDS, AWS DMS, Amazon Redshift, Amazon EventBridge
Achieved Seamless Data Migration and Transformation from On-Premises to AWS with Automated Reporting
Road Transport Corporation is a state-owned corporation that runs transport services in India. Many other Indian metro towns in Andhra Pradesh, Karnataka, Maharashtra, Goa, Odisha, and Chhattisgarh are also linked to the services offered by the customer. It serves about 6 million passengers every day, has three zones, and operates services through numerous depots.
Efficiently handled large data volumes
Optimized infrastructure expenses
Automated transformation queries
The customer faced scalability and data management challenges with their 650+GB of data on an FTP server, leading to concerns over efficient data management, query execution, and high storage costs. They wanted a streamlined infrastructure and a data warehousing solution to run complex queries quickly. CloudThat was enlisted to automate data processes, enhance performance, improve scalability, and establish a cost-efficient solution aligned with their business needs
• The migration process begins by retrieving files from the on-premises FTP server every month.
• Data residing on the server, comprising .zip files in .sql format totaling 650+GB is transferred to an Amazon EC2 instance. This Amazon EC2 instance pushes the zip files to Amazon S3, extracts the .sql files from them, deletes old records in Amazon RDS, and inserts the new records. Subsequently, it discards the .sql files after insertion.
• Amazon EventBridge is configured to start and stop the Amazon EC2 and Amazon RDS based on customer requirements.
• Approximately 28 tables of data are migrated from Amazon RDS to Amazon Redshift Serverless using the AWS Data Migration Service.
• The seamless integration between Amazon Redshift and AWS Data Migration Service underscores Amazon Redshift’s superiority in ensuring uninterrupted data transfer and synchronization, solidifying its position as the optimal solution.
• With AWS DMS facilitating schema conversion, data replication, and ongoing synchronization, Amazon Redshift emerges as a strategic choice for organizations seeking streamlined data migration processes.
• With the assistance of Amazon Redshift, accomplished transformation queries for extracting specific data, including aggregation and bunching, crucial for generating actionable insights.
• To optimize our workflow, we scheduled these transformation queries to run on the 1st day of each month, aligning with our monthly data refresh cycle. This automated scheduling mechanism not only ensured timely execution but also minimized manual intervention.
• PowerBI is used to create intuitive and interactive visualization dashboards for reporting purposes.
The solution enhanced the customer’s data management and scalability through AWS automation, optimizing processes such as data transfer from FTP to Amazon EC2, storage in Amazon S3, migration to Amazon Redshift, and seamless execution of transformation queries via Amazon EventBridge and AWS DMS.
SHARE
Our support doesn't end here. We have monthly newsletters, study guides, practice questions, and more to assist you in upgrading your cloud career. Subscribe to get them all!
