Enhancing Efficiency and Security through Containerization and Centralized Monitoring in AWS

Solutions

• Deployed highly available, scalable, and fault-tolerant microservices on an Amazon ECS cluster, utilizing Autoscaling and Application Load Balancers.
• Configured custom security groups with specific naming conventions as virtual firewalls for all AWS workloads, effectively managing incoming and outgoing traffic.
• Deployed MongoDB databases as containers on the Amazon ECS cluster running on Amazon EC2 instances and transitioned MySQL databases from public subnets to private subnets on RDS.
• Set up the EFK (Elasticsearch, Fluent-bit, Kibana) stack on Amazon EC2 instances as a centralized logging and monitoring solution.
• Collect application and container logs using Fluent-bit and Elastic APM-Agent and visualize them on Kibana dashboards.
• Configured Private Hosted Zones with records for Elasticsearch, APM, and Kibana to ensure data generated by applications remains within the Amazon VPC and to avoid data transfer charges.
• Deployed Fluent-bit as a Sidecar Container alongside the main application to collect and transfer container logs to Elasticsearch.
• Enabled AWS CloudTrail logs across all regions and stored them in a central Amazon S3 bucket, with log file integrity validation.
• Modified the existing Launch Template to include additional subnets for high availability, and wrote a custom userdata script to clear dangling images.
• Leveraged Amazon ECS Service with Placement Strategies to distribute containers evenly among instances in different Availability Zones.