AWS re:Invent 2022 : Introduction to New EC2 Instances: Hpc7g, C7gn, Inf2

Overview

Amazon has just launched three new instances, namely: For HPC applications on Amazon EC2, Hpc7g instances with new AWS Graviton3E processors offer the greatest price/performance.

For executing the most significant deep learning models at scale, Inf2 instances powered by new AWS Inferentia2 processors offer the lowest latency at the lowest cost on Amazon EC2.

Nitro V5

Before discussing the Nitro V5, we must look at a nitro system. Nitro is effectively a collection of different components using purpose-built hardware and software. It is a hypervisor built for high security and performance.

The Nitro System is an effective collection of building blocks that can be assembled in many ways, giving us the flexibility to design and rapidly deliver EC2 instance types with an ever-broadening selection of compute, storage, memory, and networking options.

Previously there have been four generations of Nitro with improved Bandwidth and Packet rates for each version. The Nitro V5 is a high-performance chip with 2x Transistors, 50% Faster DRAM Speed, and 2x more PCIe Bandwidth.

Some key features of Nitro V5:

• Nitro v5 has a 60% higher PPS.
• 30% lower latency.
• It has 40% better performance per Watt.

Graviton

For your cloud applications operating in Amazon EC2, Graviton processors are the ones that are intended to provide the greatest price performance.

The newest members of the AWS Graviton processor family are Graviton3 processors. They offer faster floating-point performance and improved computing performance. For ML applications, Amazon Graviton3 processors outperform AWS Graviton2 processors by up to three times.

The current Graviton line’s new Graviton3E chip version offers considerable performance gains, including a 35% increase in workloads that heavily rely on vector instructions.

Inferentia

A very low-cost machine learning inference chip called Amazon Inferentia is designed to give high throughput, low latency inference capabilities. As well as models that employ the ONNX format, Amazon Inferentia supports the deep learning frameworks TensorFlow, Apache, and PyTorch.

The second-generation Amazon Inferentia2 accelerator offers superior performance and capabilities than the first-generation AWS Inferentia. Compared to Inferentia, Inferentia2 offers up to 4x higher throughput and 10x lower latency. Large language models and vision transformers are examples of the more complicated models for which they are optimized.

Hpc7g Instances

The new AWS Graviton3E processors power HPC7G instances. These instances are optimized for demanding network-intensive applications that require increased networking capacity, better packet rate performance, and lower latency, such as data analytics and closely-coupled cluster computing tasks and network virtual appliances (firewalls, virtual routers, load balancers).

For high performance computing workloads like computational fluid dynamics, weather simulations, genomics, and molecular dynamics on Amazon EC2, HPC7g instances powered by new AWS Graviton3E processors provide the best price performance.

It can supply 200 Gbps of dedicated network bandwidth tailored for traffic within instances in the same VPC. The hpc7g instances will come in various sizes and have up to 128 GiB memory and 64 vCPUs. It can deliver twice as good performance as the current C6gn, which has Graviton2 processor power.

We can use Hpc7g instances with AWS Parallel Cluster, an open-source cluster management tool, to provision Hpc7g instances alongside other instances with the same HPC cluster, which provides more flexibility to run different workloads types.

Use Cases:

AWS HPC7g instances are compute-optimized instances designed for high-performance computing (HPC) workloads requiring significant CPU and memory resources.

Some use cases for AWS HPC7g instances include:

• Molecular dynamics simulations: Used for molecular dynamics simulations that require a large number of processors and a high memory-to-processor ratio to process large amounts of data.
• Computational fluid dynamics: Computational fluid dynamics simulations require large computational resources to simulate complex fluid flows.

Weather forecasting: Running complex weather forecasting models that require high levels of parallel processing and large amounts of memory.

C7gn Instance

The highest network bandwidth and packet-processing speed among Amazon EC2 network-optimized instances are provided by C7gn instances, which include new AWS Nitro Cards powered by new, fifth generation Nitro chips with network acceleration. These instances also consume less power.

For more consistent performance with reduced CPU use, the Nitro Cards offload and accelerate I/O for functions from the host CPU to specialized hardware, delivering nearly all of an Amazon EC2 instance’s resources to customer workloads.

Compared with current-generation Amazon EC2 instances, C7gn instances can provide up to two times the network bandwidth, up to 50% higher packet processing per second, and lower network latency thanks to new AWS Nitro Cards.

Nitro Cards, which reduces power usage for client workloads. It allows the customer to scale both performance and throughput and provides low network latency to optimize the cost of the most demanding network intense workloads in the instance.

Use Cases:

These instances are optimized for compute intensive workloads.

Some use cases for C7gn instances include:

• High-Performance Computing (HPC): Molecular Dynamics Simulations, Weather Modeling, and Financial Modeling.
• Machine Learning: Training and getting the inference in deep learning models.
• Data Analytics: Data Warehousing, Data Mining, and Data Processing, particularly when working with large datasets.
• Video Encoding: Transcoding video files to different formats and resolutions.

Inf2 Instances

Inf2 instances are explicitly designed for deep learning (DL) inference. Your most demanding DL applications are created to give high performance at the lowest cost in Amazon EC2. Inference-optimized Amazon EC2 instance allows distributed inference. Inf2 offers the greatest performance for deep learning models with more than 100 billion parameters by distributing huge models across several CPUs.

Inf2 instances can be incorporated to run inference applications such as NLU, image generation, language translation, fraud detection, and many others.

With AWS Neuron, the unified software development kit (SDK) for ML inference, we can begin employing Inf2 instances. Customers may deploy their current models to Inf2 instances with minimal code modification because of AWS Neuron’s integration with well-known ML frameworks like PyTorch and TensorFlow. Because of the need for quick inter-chip communication when partitioning big models across multiple chips, Inf2 instances enable Neuron Link, AWS’s high-speed intra-instance interconnect, which provides 192 GB/s of ring connection. Comparing Inf1 instances of the current generation, Inf2 instances offer up to 4 times the throughput and 10 times reduced latency.

Here is a list of the various sub-types of Inf2:

The benefits of the Inf2 instance are listed below:

1. There is native support for Machine Learning Algorithms.
2. There is an increased performance while having a significantly lower inference price.
3. We can efficiently deploy a 175B parameter model for inference in multiple accelerators on single Inf2 instances.

Use Cases:

Inf2 instances are optimized to accelerate inferencing workloads, which are the production stage of machine learning models that involve making predictions on new data.

Some use cases for AWS Inf2 instances include:

• Natural Language Processing (NLP): Sentiment Analysis, Chatbots.
• Computer Vision: Object Detection, Image Classification, and Facial Recognition.
• Fraud Detection: Inf2 instances can help accelerate fraud detection and prevention in financial institutions by quickly identifying and flagging suspicious transactions.
• Recommendation Systems: Inf2 instances can accelerate recommendation systems, such as those used by e-commerce websites to suggest products to customers based on their past purchases and browsing history.
• Speech Recognition: Inf2 instances can accelerate speech recognition workloads, such as those used by virtual assistants and voice-controlled devices.

Inf2 instances can be utilized for machine learning workloads that require high inferencing capabilities.

Conclusion

Here through this article, we have tried to have an intuition over the newly launched Compute instances Hpc7g and C7gn, and Inf2, and we can see that these instances come up with a motive of high performance at a lower price, though the Hpc7g and C7gn are not much talked about as for now. In the next set of articles, I will try to come up with detailed information with the demo.

About CloudThat

CloudThat is also the official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner and Microsoft gold partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best in industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

Drop a query if you have any questions regarding Amazon EC2 Instance and I will get back to you quickly.

To get started, go through our Consultancy page and Managed Services Package that is CloudThat’s offerings.