Efficient Centralized Monitoring of AWS WAF Logs Using Amazon Managed Grafana

Overview

Yet, as your cloud infrastructure expands to match your business growth, efficiently analyzing and monitoring logs across numerous AWS Accounts and Regions can prove challenging without a centralized visualization tool.

Introduction

In this blog post, we will guide you through setting up centralized logging for your AWS WAF web ACLs using an AWS Firewall Manager policy and real-time monitoring of these logs via Amazon Managed Grafana. You will also discover how to craft dashboards in Amazon Managed Grafana, providing a powerful visualization of AWS WAF logs. This is instrumental for enhancing threat intelligence, refining security rules, addressing false positives, and effectively managing incident responses. By the end of this tutorial, you will have the tools and knowledge to bolster your web application’s security posture and streamline your monitoring processes.

Architecture Overview

The diagram below showcases how AWS services are integrated within the proposed solution:

This configuration utilizes Firewall Manager to aggregate AWS WAF logs from multiple accounts into a central repository. This capability allows you to record all inspected traffic by AWS WAF into Amazon Simple Storage Service (Amazon S3) buckets through Amazon Kinesis Data Firehose, which is aligned with your AWS WAF policy set in AWS Firewall Manager. Following this, you’ll configure an AWS Glue crawler, establish an Amazon Athena table, and create an Athena view. These elements are essential for constructing a Managed Grafana dashboard that enables real-time visualization of the activities.

Prerequisites

1. Set up Amazon Athena Workgroups: Establish Amazon Athena workgroups by the prerequisites required for Amazon Managed Grafana integration.
2. Verify IAM Permissions: Ensure your AWS IAM user or role possesses the necessary permissions to access AWS Firewall Manager. For guidance, see the Access to AWS Firewall Manager resource.
3. Utilize AWS Firewall Manager: Fulfill all the prerequisites required to use AWS Firewall Manager effectively.
4. Create a Default Administrator Account: Set up an AWS Firewall Manager administrator account.
5. Establish and Implement WAF Policy: If not already done, create and apply a WAF policy within your AWS Firewall Manager administrator account.
6. Configure Amazon Managed Grafana Workspace: For details on configuration and setup, refer to the Amazon Managed Grafana – Getting Started guide. Additionally, for specific steps on creating the Amazon Managed Grafana workspace, refer to the Creating a Workspace guide.

Step-by-Step Guide

Step 1: Launch the AWS CloudFormation template

Launch the AWS CloudFormation template using this link.

To deploy your resources using an AWS CloudFormation template, follow these steps:

1. Log in to the AWS Management Console.
2. Go to the AWS CloudFormation console, select ‘Create Stack’ and then choose ‘With new resources’.
3. Enter a ‘Stack name’ and click ‘Next’.
4. Keep the ‘Configure stack options’ at their default settings and click ‘Next’.
5. On the final screen, under ‘Capabilities’, select the checkbox labeled ‘I acknowledge that AWS CloudFormation might create AWS IAM resources with custom names’.
6. Click ‘Submit’ to proceed.

After the successful creation of the Stack, the following resources will be deployed:

Amazon S3 Bucket, AWS Glue Crawler, AWS Glue Database, Amazon Kinesis Data Stream, Amazon Athena Query (located under the ‘Saved Queries’ tab for creating views in Athena), Corresponding AWS IAM Roles and Policies

Additionally, navigate to the ‘Outputs’ tab and copy the ‘KinesisDeliveryStreamName’.

Step 2: Configure centralized logging for AWS WAF policy

1. Log into the AWS Management Console with your Firewall Manager administrator credentials, and open the Firewall Manager console. Follow these steps:
2. Select ‘Security Policies’ in the navigation menu and choose your AWS WAF policy.
3. Click on the ‘Policy details’ tab, and in the ‘Policy rules’ section, select ‘Edit’.
4. Under ‘Logging configuration status’, choose ‘Enable Logging’.
5. Select the Kinesis Data Firehose stream named ‘aws-waf-logs-kinesis-delivery-stream’, which you created via CloudFormation in Step 1, for the logging configuration.
6. Click ‘Next’, review your settings, and then click ‘Save’ to apply the changes to the policy.

Step 3: Create a View in Amazon Athena using the saved queries created as part of the AWS CloudFormation stack

1. Navigate to Amazon Athena, access the Query editor, and select the ‘Saved queries’ tab. From there, choose the query labeled “aws_waf_centralized_logging”.

Note: The workgroup created is named “waf-logs-athena”

2. Confirm the Data source, Database, and Table names within the Query editor before executing the query. Once executed successfully, the query will generate a View titled “waflogs”

Step 4: Configure Amazon Athena Data Source in Amazon Managed Grafana

1. Open the Amazon Managed Grafana console by entering the Managed Grafana workspace URL and log in with your configured user credentials.
2. Navigate to ‘Administration’, then ‘Data sources’, and select ‘Amazon Athena’.
3. Set up the Amazon Athena data source by selecting ‘Default Region’ (us-east-1), ‘Data source’ (AWSDataCatalog), ‘Database’ (waflogsdb), ‘Workgroup’ (waf-logs-athena), and specify the ‘Output Location’ for your Athena query.
4. Click ‘Save & Test’ to ensure the data source functions correctly. You can now begin querying and visualizing the metrics from your AWS environment.

Step 5: Create an Amazon Managed Grafana dashboard

Amazon Managed Grafana enables you to craft a near real-time AWS WAF Logging dashboard. As a fully managed service, Amazon Managed Grafana simplifies creating, configuring, and sharing interactive dashboards and charts for monitoring your data. It also allows you to establish alerts and notifications based on specified conditions or thresholds, facilitating prompt issue identification and response.

Having completed all prior steps successfully, let’s proceed to create a dashboard in Amazon Managed Grafana by following these steps:

1. Download the Dashboard JSON File: Download the AWS WAF Logging dashboard JSON file from this link.
2. Open the Import Interface: Navigate to the Amazon Managed Grafana console. Click on the ‘+’ sign, go to the ‘Dashboards’ tab, and select ‘Import’.
3. Import the Dashboard: Copy the contents of the downloaded JSON file. Paste them into the ‘Import via panel JSON’ textbox within the import interface. Finally, click ‘Load’ to import the dashboard.

Conclusion

In this blog post, you have discovered how to set up centralized logging for your AWS WAF web ACLs using an AWS Firewall Manager policy and how to monitor these logs nearly in real-time with Amazon Managed Grafana. The visual tools provided will assist you in enhancing threat intelligence, strengthening security rules, diagnosing false positives, and accelerating incident response.

Drop a query if you have any questions regarding AWS WAF and we will get back to you quickly.

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.