AWS Neuron 2.31.0 now available with NKI 0.5.0 and UltraServer Operator
AWS Neuron 2.31.0 is now available, introducing NKI 0.5.0 with new MX FP8 scale dtype support, tensor indirection on compute operations for fewer instructions in indexed access patterns, and new NkiTensor view APIs for zero-cost tensor layout transformations. This release also introduces the Neuron UltraServer Operator for Amazon EKS in public beta, automating UltraServer discovery, workload allocation, and resource claim generation for Trainium UltraServer workloads on Amazon EKS.
The Neuron Compiler includes a redesigned code generation backend now enabled by default on Trn2 and Trn3, delivering improved performance. The Neuron Runtime adds contiguous shared scratchpad support, simplifying device configuration by eliminating the need to manually set scratchpad page sizes. The NKI Library adds 14 new experimental kernels covering MoE training collectives, deformable attention, DeepSeek MLA projection, and ring attention, alongside PyTorch reference implementations. Neuron Explorer adds System Trace Viewer source code linking and updated default grouping for improved workload debugging.
To get started, see the following resources: