A Tutorial on Configuring Amazon VPC Flow Logs with Amazon CloudWatch Log Groups

Overview

One can manage a dynamic AWS environment with multiple Amazon VPCs hosting critical applications. Security is a top priority, and you want to proactively detect and respond to any abnormal network activity in real time. An attacker gains unauthorized access to one of your Amazon EC2 instances within a VPC and attempts to exfiltrate sensitive data. To identify and respond to this security incident, you leverage Amazon VPC Flow Logs with Amazon CloudWatch Log Groups in real time.

Upon detecting a potential security incident, one can create Amazon VPC Flow Logs in real-time for the affected Amazon VPC. Simultaneously, we can create a real-time Amazon CloudWatch Log Group dedicated to collecting logs from the Flow Logs of the affected Amazon VPC. As the Flow Logs stream into Amazon CloudWatch Logs in real-time, you use Amazon CloudWatch Logs Insights to analyze the network traffic patterns. We can set up real-time metric filters and alarms within Amazon CloudWatch to detect abnormal patterns in network traffic. For example, you might set an alarm to trigger when a significant increase in outbound traffic from a specific Amazon EC2 instance occurs.

Introduction

On the other hand, Amazon CloudWatch, a key component of Amazon Web Services (AWS), specializes in resource monitoring. It facilitates real-time monitoring of various AWS resources, including Amazon EC2 instances, Amazon Elastic Block Store (EBS) volumes, Amazon Elastic Load Balancers, and Amazon RDS database instances. Amazon CloudWatch comes pre-configured to deliver metrics related to request counts, latency, and CPU usage, providing a comprehensive view of the health and performance of your AWS resources.

Prerequisites

Before we begin the real-time setup, make sure you have the following prerequisites in place:

1. AWS Account: Access to an AWS account with sufficient permissions for configuring Amazon VPC Flow Logs and Amazon CloudWatch Logs.
2. Amazon VPC: An existing VPC in your AWS environment.
3. AWS IAM Role: A role with the necessary Amazon VPC Flow Logs and Amazon CloudWatch Logs permissions. This role should include the AmazonVPCFullAccess and CloudWatchLogsFullAccess policies.

Step-by-Step Guide

Navigate to the AWS console’s VPC Dashboard.

1. On the VPC Dashboard, locate and select “Flow Logs.”

2. Within the Flow Logs section, click on “Create Flow Logs.”

Note: If you haven’t set up a Destination log group or AWS IAM Role, follow the steps to create them from scratch.

• To create a Destination log group, go to the Amazon CloudWatch dashboard in your Amazon VPC Console.
• Click on “Logs” and provide a name for the log group. Then, click on “Create Log.”

3. Return to the Amazon VPC Flow Logs dashboard and enter the name of the log group you just created.

4. Proceed to create an AWS IAM role that grants permission for Amazon VPC to write to the log group.

• Navigate to the AWS IAM dashboard.

• Click on “Create a Role.”
• Choose the role type as “EC2 Role.”

• Continue to the “Next” step to set up tag names.
• Move to “Next:Tags,” enter the role name, and click “Create a Role.”

• Give any random name for the role and click the createrole button.

5. Once the AWS IAM role is created, attach the necessary inline policy for publishing flow logs to CloudWatch Logs.

6. Locate the role you created and attach the inline policy.

7. Verify that your role is configured with a trust relationship, allowing the Flow Logs service to assume the role. This permission enables Amazon EC2 instances to write into the specified Log group.

8. To review and modify the trust relationship:

• Access the Roles Dashboard within the AWS IAM console.

• Click on “Trust Relationship” for the relevant role, edit the trust relationship, and past the JSON mentioned below:

9. Return to your Amazon VPC Flow Logs dashboard, then choose the role you previously created. Click on “Create Flow Logs.”

10. Once the flow logs are created, wait for a brief period. Afterward, revisit the Amazon CloudWatch dashboard. Navigate to log groups, and you will observe the traffic data within the Log stream.

Conclusion

Integrating Amazon Virtual Private Cloud (VPC) Flow Logs with Amazon CloudWatch Log Groups on AWS provides a powerful solution for monitoring network traffic and gaining insights into the operational aspects of your Amazon VPC. Following the steps outlined in this guide, a real-time monitoring infrastructure that captures and analyzes IP traffic going to and from your network interfaces can be set up. The seamless integration with Amazon CloudWatch Log Groups facilitates centralized log storage and analysis.

Through this setup, you not only enhance the security posture of your AWS environment but also gain the ability to detect and respond to abnormal network activity proactively. Leveraging Amazon CloudWatch Alarms and Metrics Filters ensures you receive immediate notifications, enabling swift incident response and mitigation. The real-time nature of this configuration allows for dynamic scalability, accommodating the ever-changing nature of AWS resources.

As you continue to explore and optimize your cloud architecture, the combination of Amazon VPC Flow Logs and Amazon CloudWatch Log Groups proves to be a valuable tool for maintaining a secure, well-monitored, and efficiently managed network infrastructure on AWS.

Drop a query if you have any questions regarding Amazon VPC and Amazon CloudWatch, and we will get back to you quickly.

About CloudThat

CloudThat is an official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, Microsoft Gold Partner, and many more, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best-in-industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.