Maximizing Web Security Through Effective AWS WAF Log Analysis

Overview

It is critical to secure web applications in the modern digital environment. AWS Web Application Firewalls (AWS WAFs) protect applications from myriad threats, including SQL injection, cross-site scripting (XSS), and other malicious activities. However, simply deploying an AWS WAF isn’t enough. To fully harness its capabilities, it’s essential to analyze AWS WAF logs effectively. This blog post delves into the significance of AWS WAF logs, how to query them for meaningful insights, and provides practical examples to help you get started.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

AWS WAF Logs

HTTP traffic to and from a web application is tracked, filtered, and blocked by a Web Application Firewall (AWS WAF). It functions by examining incoming requests and using a set of rules to identify and eliminate security risks.

Every action taken by the AWS WAF whether allowing, blocking, or counting requests is logged, providing a wealth of information about the traffic patterns and potential security threats your application faces.

Key Components of AWS WAF Logs

Understanding the structure of AWS WAF logs is crucial for effective analysis. You will come across the following important fields:

@timestamp: The time when the request was made.
clientIp: The IP address of the client making the request.
uri: The URI that the client has requested.
terminatingRuleId: The AWS WAF rule ID that set off the action.
action: The action taken by the AWS WAF (e.g., ALLOW, BLOCK, COUNT).
requestBodySize: The size of the request body.
webaclId: The ID of the Web ACL that processed the request.
@message: The raw log message may contain additional details like User-Agent.

Practical Query Examples

Let’s explore some practical queries to analyze AWS WAF logs effectively.

Identifying Blocked Requests

Objective: Retrieve the timestamp, client IP, URI, and the rule that blocked the request.

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter action = "BLOCK"
| sort @timestamp desc

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| filter action = "BLOCK"

| sort @timestamp desc

Explanation:

fields: Selects the timestamp, client IP, URI, and terminating rule ID.
filter: Filters logs where the action is “BLOCK”.
sort: Orders the results by timestamp in descending order.

Counting Blocked Requests by Client IP

Objective: Count the number of blocked requests per client IP.

fields @timestamp, @message, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| stats count() as requestCount by ClientIP
| filter action == "BLOCK"
| sort @timestamp desc
| limit 10000

fields @timestamp, @message, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| stats count() as requestCount by ClientIP

| filter action == "BLOCK"

| sort @timestamp desc

| limit 10000

Explanation:

stats count() as requestCount by ClientIP: Aggregates the number of blocked requests per client IP.
limit 10000: Limits the results to the top 10,000 entries.

Analyzing Requests from a Specific IP

Objective: Examine blocked requests from a specific IP address (e.g., 10.0.0.0).

fields @timestamp, @message, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter httpRequest.clientIp == "10.0.0.0"
| filter action == "BLOCK"
| sort @timestamp desc
| limit 10000

fields @timestamp, @message, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| filter httpRequest.clientIp == "10.0.0.0"

| filter action == "BLOCK"

| sort @timestamp desc

| limit 10000

Explanation:

filter httpRequest.clientIp == “10.0.0.0”: Filters logs to only include requests from the specified IP.
filter action == “BLOCK”: Further filters to only blocked actions.

Filtering by Specific AWS WAF Rule

Objective: Retrieve blocked requests that were blocked by the rate-limit rule.

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter action = "BLOCK"
| filter terminatingRuleId == "rate-limit"
| sort @timestamp desc

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| filter action = "BLOCK"

| filter terminatingRuleId == "rate-limit"

| sort @timestamp desc

Explanation:

filter terminatingRuleId == “rate-limit”: Targets logs where the specific rule was triggered.

Monitoring Specific URI Patterns

Objective: Analyze blocked requests for URIs matching the pattern /xyz/ within a specific Web ACL.

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter webaclId = "arn of AWS WAF"
| filter action = "BLOCK"
| filter httpRequest.uri like /xyz/
| sort @timestamp desc

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| filter webaclId = "arn of AWS WAF"

| filter action = "BLOCK"

| filter httpRequest.uri like /xyz/

| sort @timestamp desc

Explanation:

filter webaclId = “…”: Targets a specific Web ACL by its ARN.
filter httpRequest.uri like /xyz/: Filters URIs matching the pattern /xyz/.
sort RequestCount desc: Sorts the results by the number of requests in descending order.

Analyzing Allowed or Counted Requests by URI

Objective: Determine how many accepted or tallied requests are for each URI.

fields @timestamp, httpRequest.uri as URI
| filter action = "ALLOW" or action = "COUNT"
| stats count(*) as requestCount by httpRequest.uri
| sort requestCount desc
| limit 100

fields @timestamp, httpRequest.uri as URI

| filter action = "ALLOW" or action = "COUNT"

| stats count(*) as requestCount by httpRequest.uri

| sort requestCount desc

| limit 100

Explanation:

filter action = “ALLOW” or action = “COUNT”: Includes only allowed or counted actions.
stats count(*) as requestCount by httpRequest.uri: Aggregates the number of such requests per URI.

Analyzing Request Body Sizes

Objective: Categorize and count requests based on the size of the request body.

Requests with Body Size ≤ 8,912 bytes:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter requestBodySize <= 8912 
| sort @timestamp desc
| stats count() as TotalRequestCount

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| filter requestBodySize <= 8912

| sort @timestamp desc

| stats count() as TotalRequestCount

Requests with Body Size > 8,192 and ≤ 65,536 bytes:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter requestBodySize > 8192 and requestBodySize <= 65536
| sort @timestamp desc
| stats count() as TotalRequestCount

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| filter requestBodySize > 8192 and requestBodySize <= 65536

| sort @timestamp desc

| stats count() as TotalRequestCount

Requests with Body Size > 65,536 bytes:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter requestBodySize > 65536 
| sort @timestamp desc
| stats count() as TotalRequestCount

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| filter requestBodySize > 65536

| sort @timestamp desc

| stats count() as TotalRequestCount

Detailed Count by URI, Rule, and Action for Large Requests:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter requestBodySize > 65536 
| sort @timestamp desc
| stats count() as RequestCount by URI, rule, action

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule

| filter requestBodySize > 65536

| sort @timestamp desc

| stats count() as RequestCount by URI, rule, action

Explanation:

filter requestBodySize: Filters logs based on the size of the request body.
stats count() as TotalRequestCount: Counts the number of requests in each size category.
stats count() as RequestCount by URI, rule, action: Provides a detailed count for large requests, broken down by URI, rule, and action.

Conclusion

AWS WAF logs are a treasure trove of information that, when analyzed effectively, can significantly enhance your web application’s security and performance. By crafting precise queries, as demonstrated in this blog, you can uncover valuable insights into traffic patterns, potential threats, and the effectiveness of your security rules. Remember to integrate these practices into your regular security routines to maintain a defense against evolving cyber threats.

Drop a query if you have any questions regarding AWS WAF and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, AWS GenAI Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, Amazon ECS Service Delivery Partner, AWS Glue Service Delivery Partner, Amazon Redshift Service Delivery Partner, AWS Control Tower Service Delivery Partner, AWS WAF Service Delivery Partner and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. What is an AWS WAF log?

ANS: – An AWS WAF (Web Application Firewall) log records the actions taken by the firewall, such as allowing, blocking, or counting HTTP requests. These logs provide insight into potential threats and traffic patterns targeting your application.

2. Why is it important to analyze AWS WAF logs?

ANS: – Analyzing AWS WAF logs helps detect malicious activities, optimize application performance, comply with regulations, and respond to security incidents efficiently.

3. How can I query AWS WAF logs for insights?

ANS: – You can use Amazon CloudWatch Logs Insights to craft queries, such as identifying blocked requests, counting requests by IP, and analyzing traffic patterns based on AWS WAF rules.

WRITTEN BY Noopur Shrivastava

Noopur Shrivastava works as a Research Associate at CloudThat. She is focused on gaining knowledge of the Cloud environment. Noopur loves learning about new technology and trying out different approaches to problem-solving.