AWS Neuron adds support for Llama-2 70b model and PyTorch 2.0
AWS Neuron is the SDK for Amazon EC2 Inferentia and Trainium based instances purpose-built for generative AI. Today, with Neuron 2.15 release, we are announcing support for Llama-2 70b model training as well as PyTorch 2.0 support.
AWS Neuron is the SDK for Amazon EC2 Inferentia and Trainium based instances purpose-built for generative AI. Today, with Neuron 2.15 release, we are announcing support for Llama-2 70b model training as well as PyTorch 2.0 support.
Neuron integrates with popular ML frameworks like PyTorch and TensorFlow, so you can get started with minimal code changes and without vendor-specific solutions. Neuron includes a compiler, runtime, profiling tools, and libraries to support high performance training of generative AI models on Trn1 instances and inference on Inf2 instances. This release adds Llama-2 70b model training support with Neuron Distributed library and adds Beta support for PyTorch 2.0.
You can use AWS Neuron SDK to train and deploy models on Trn1 and Inf2 instances, which are available in the following AWS Regions as On-Demand Instances, Reserved Instances, and Spot Instances, or as part of a Savings Plan: US East (N. Virginia), US West (Oregon), and US East (Ohio).
For a full list of new features and enhancements in Neuron 2.15, visit Neuron Release Notes. To get started with Neuron, see: