Streamlining Multi-Cloud Cost Management with Amazon QuickSight and FOCUS

Introduction

Managing cloud costs in multi-cloud environments has always been a challenge for organizations. Traditionally, customers need to navigate through individual billing portals of each cloud provider to gather cost-related information. Without a consolidated view of their expenditures, making informed, data-driven decisions to optimize cloud spending becomes complex.

The Evolution of Multi-Cloud Cost Visibility

In September 2023, AWS introduced the Cloud Intelligence Dashboard for Azure, a solution to standardize cost and usage visualization across AWS and Microsoft Azure. While customers welcomed this tool, they soon requested the ability to view costs across both platforms in a single, unified dashboard.

The primary hurdle was the variation in cost and usage data model between cloud providers. Addressing this, in June 2024, AWS launched Data Exports for FOCUS 1.0, based on the FinOps Open Cost and Usage Specification (FOCUS). This open-source standard enables consistent and standardized reporting of billing data across multiple cloud providers. Alongside this, AWS released the FOCUS Dashboard, a new Amazon QuickSight solution that visualizes this standardized data.

Solution Overview

The enhanced solution builds upon the existing Cloud Intelligence Dashboard for Azure. It maintains flexibility and scalability while introducing new features to handle the FOCUS cost model seamlessly.

The architecture processes data in two stages:

1. Data Transformation:
   • AWS Glue converts the data from AWS and Azure into a standardized format aligned with the FOCUS specification.
   • A new AWS Glue job processes the FOCUS cost model.
2. Data Analysis and Visualization:
   • Amazon Athena queries the standardized data, creating views for detailed, short-term data and longer-term trends.
   • These views feed Amazon QuickSight datasets, which power the FOCUS Dashboard for comprehensive visualization.

Key Features and Enhancements

1. Separate Storage for AWS and Azure Data:
   AWS and Azure data are stored separately in Amazon S3 buckets and AWS Glue Data Catalog tables to simplify management. Instead of merging data structures at the storage level, Athena consolidates the data via views. This separation makes it easier to incorporate other cloud providers into the dashboard in the future.
2. Enhanced Tagging Support:
   Managing tags across datasets became more complex with the integration multiple cloud providers. The solution now uses Athena queries to surface tags, making displaying and updating tags from both AWS and Azure simpler.
3. Multi-Instance Deployments:
   Customers can now deploy multiple solution instances within the same AWS account. For example, independent software vendors (ISVs) that manage costs for different clients can set up isolated deployments for each customer.
4. Streamlined Deployment and Updates:
   The solution supports both AWS CloudFormation and Terraform for infrastructure deployment. This allows customers to easily set up, update, and manage the solution using Infrastructure as Code (IaC).

Data Integration and Visualization Flow

The integration flow (illustrated in the architecture diagram) consists of:

• Azure Data Export: Azure FOCUS data is stored in a dedicated bucket and cataloged in AWS Glue.
• AWS Data Export: AWS data undergoes similar processing.
• Athena Views: Consolidated views join the data from AWS and Azure without modifying storage structures.
• Amazon QuickSight Dashboard: These views are transformed into datasets for visualization, providing both granular and high-level insights.

This modular design ensures scalability and allows the solution to accommodate additional cloud providers in the future with minimal reconfiguration.

Getting Started

To deploy the solution, visit the Cloud Intelligence Dashboard for Azure workshop for a comprehensive setup guide. The implementation steps include:

• Setting up Azure prerequisites.
• Uploading solution artifacts to an Amazon S3 bucket.
• Deploying templates via AWS CloudFormation or Terraform.
• Triggering an initial data copy.
• Integrating with the FOCUS Dashboard and managing configurations like schedules and tags.

The workshop also provides an interactive sample dashboard to help users explore features and functionalities.

Cost Considerations

The estimated costs for deploying this solution include:

• AWS Resources: Processing 2GB of data daily costs under $100 monthly.
• Amazon QuickSight Enterprise Subscription: Pricing is available on the QuickSight website.
• Microsoft Azure Egress Data Charges: Check Azure’s pricing page for details.

Conclusion

Integrating FOCUS into the Cloud Intelligence Dashboard for Azure marks a significant step toward simplifying multi-cloud cost management. By standardizing cost data and visualizations across AWS and Azure, this solution enables organizations to track, analyze, and optimize their cloud spending effectively.

With its scalable and modular design, the dashboard is well-positioned to support additional cloud providers in the future, offering a robust approach to multi-cloud cost visibility and optimization.

Drop a query if you have any questions regarding FOCUS and we will get back to you quickly.

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, AWS GenAI Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner,  AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, Amazon ECS Service Delivery Partner, AWS Glue Service Delivery Partner, Amazon Redshift Service Delivery Partner, AWS Control Tower Service Delivery Partner, AWS WAF Service Delivery Partner, Amazon CloudFront and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.