Easy Access of Data Using Amazon CloudWatch Cross-Account Observability

Overview

According to work culture, deploying applications using multiple AWS accounts is a best practice to set up security and billing boundaries between teams and decrease the crash of operational events. Whenever you think of a multi-account plan, you must examine separate telemetry data across several accounts. For this solution or flexibility to monitor all the components of your applications from a centralized view, we will be discussing about Amazon CloudWatch cross-account observability.

Introduction

CloudWatch cross-account observability is a new search, analysis, and agreed cross-account telemetry data stored in CloudWatch such as metrics, logs, and traces.

According to this above observability, now you can build central monitoring AWS account and connect your other accounts as sources. You can search, audit, and analyze logs across your applications to rumble into operational issues in a situation of seconds. You can find and visualize metrics from many accounts in a single place and create alarms that evaluate metrics for other accounts. You can begin with an aggregated cross-account view of your application to identify the resources exhibiting errors and drop into correlated traces, metrics, and logs to find the root source. This trouble-free cross-account data access and navigation helps us to reduce the time and effort required to troubleshoot problems.

Steps to Configure CloudWatch Cross-Account Observability

For cross-account observability, CloudWatch has the concept of monitoring and source accounts:

• A monitoring account is a central AWS account that can view and interact with observability data shared by other accounts.
• A source account-It is an individual AWS account that shares observability data and resources with one or more monitoring accounts.

You can also configure multiple monitoring accounts to your needs. CloudWatch cross-account observability is also combined with AWS Organization.

1. In the beginning, we will configure the monitoring account in the CloudWatch console. Go to Settings in the navigation pane. In the Monitoring account configuration section, then choose Configure.

2. Now choose which telemetry data can be shared with the monitoring account: Logs, Metrics, and Traces. Here, I enabled all three.

3. List the source accounts that will share data with this monitoring account, I can be using account IDs, organization IDs, or organization paths. For organizational purposes, use an organization ID to include all the accounts in the organization or an organization path to include all the accounts in a business unit. Here, I only have one source account to link, so I use the account ID, then choose to configure.

4. The monitoring account is now successfully enabled.

5. Choose Resources to link accounts to determine how to link my source accounts.

6. To link source accounts in an AWS organization, download an AWS CloudFormation template to be deployed in a CloudFormation delegated administration account.
7. On the other hand, to link individual accounts, I can either download a CloudFormation template to be deployed in each account or copy a URL that helps me use the console to set up the accounts. I copied the URL and then pasted it into another browser where I am signed in as the source account. Then, we can design which telemetry data to share (logs, metrics, or traces). The ARN (Amazon Resource Name) of the monitoring account configuration is filled out because I copy-pasted the URL in the previous step. Say suppose I don’t use the URL, I can simply copy the ARN from the monitoring account and paste it here. Then confirm the label used to identify my source account and choose Link.

8. To confirm the monitoring account permission dialog, type Confirm to complete the configuration of the source account.

Demo on Using CloudWatch Cross-Account Observability with Multi-Account Function

1. To test how things work with cross-account observability, here I deployed a simple cross-account application using two AWS Lambda functions, one in the source account (multi-account-function-a) and one in the monitoring account (multi-account-function-b). When they are triggered, the function in the source account publishes an event to an Amazon EventBridge event bus in the monitoring account. And then, an Event Bridge rule triggers the execution of the function in the monitoring account. This setup is using only two accounts. You would have your workloads running in multiple source accounts.

2. Go to the Lambda console, the two Lambda functions have Active tracing and Enhanced monitoring enabled. To collect telemetry data, here using the AWS Distro for Open Telemetry (ADOT) Lambda layer. The increased monitoring option turns on Amazon CloudWatch Lambda Insights to collect and aggregate Lambda function runtime performance metrics.

3. Then in the Lambda console prepare a test event of the source account. Then, choose Test and run the function a few times.

Here you can identify to understand what the components of my application, running in different accounts, are doing. Firstly, start with logs and then move to metrics and traces.

4. CloudWatch console of the monitoring account, choose Log groups in the Logs section of the navigation pane. Then search for and find the log groups created by the two Lambda functions running in different AWS accounts. As surely expected, each log group shows the account ID and label originating the data, then select both log groups and choose View in Logs Insights.

5. Now you can able to search and analyze logs from different AWS accounts using the CloudWatch Logs Insights query syntax.

After creating Contributor Insights rules on cross-account log groups, it allows me to have a holistic view of what security events are happening across accounts or identify the most expensive Lambda requests in a serverless application running on multiple accounts.

6. Select All metrics in the Metrics section of the navigation pane. To check the Lambda function span performance metrics collected by CloudWatch Lambda Insights, I choose Lambda Insights and then the function name. I query for multi-account and memory to see the memory metrics. Then again, I see the account IDs and labels that tell me that these metrics are coming from two different accounts. I can select the metrics I am interested in and create cross-account dashboards and alarms. With the metrics selected, select Add to dashboard in the Actions below.

7. You can also create a new dashboard and choose the Stacked area widget type, then choose to Add to dashboard.

8. Same thing you can do for the CPU and memory metrics (but using different widget types) to quickly create a cross-account dashboard where I can keep under control my multi-account setup.

9. Eventually, choose Service map from the X-Ray traces section of the navigation pane to see the flow of my multi-account application. In the service map section, the client triggers the Lambda function in the source account. Then, an event is driven to the other account to run the other Lambda function.

10. Go to the service map, select the gear icon for the function running in the source account (multi-account-function-a) and then View traces to look at the individual traces. The traces carry data from multiple AWS accounts. I can search for traces coming from a specific account using syntax.

The service map now twinges together telemetry from multiple accounts in a single place, delivering a consolidated view to monitor their cross-account applications. This helps me to spot issues quickly and reduces resolving time.

Availability and Pricing

Amazon CloudWatch cross-account observability is available today in all business AWS Regions using the AWS Management Console, AWS Command Line Interface (CLI), and AWS SDKs. AWS CloudFormation support is advancing in the next few days. There is no extra cost for logs and metrics in Cross-account observability and the first trace copy is free. See the Amazon CloudWatch pricing page for more details.

Conclusion

Amazon CloudWatch Cross-Account Observability smooths cross-account data access and navigation and help us to reduce the time and effort required for troubleshooting problems. This helps me to spot issues quickly and reduces resolving time.

About CloudThat

CloudThat is also the official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner and Microsoft gold partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best in industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

Drop a query if you have any questions regarding Amazon CloudWatch cross-account Observability and I will get back to you quickly.

To get started, go through our Consultancy page and Managed Services Package that is CloudThat’s offerings.