Amazon SageMaker HyperPod now supports continuous provisioning for enhanced cluster operations

Amazon SageMaker HyperPod now offers continuous provisioning, a new capability that enables greater flexibility and efficiency for enterprise customers running large-scale AI/ML workloads. AI/ML customers need to start training quickly, scale seamlessly, perform maintenance without disrupting operations, and have granular visibility into cluster operations. Customers also require the ability to efficiently manage dynamic inference workloads where capacity needs change frequently, making operational agility critical for successful AI initiatives.

With continuous provisioning, SageMaker HyperPod automatically provisions remaining capacity in the background while training jobs can begin immediately on available instances. HyperPod will retry in the background when it encounters node provisioning failures and ensure clusters reliably reach their desired scale without requiring any manual intervention. This helps customers reduce time-to-training and maximizes resource utilization across dynamic workloads. You can now perform concurrent operations such as scaling nodes independently, applying patches, or adjusting different instance groups simultaneously, thus increasing efficiency. The enhanced event-driven architecture provides comprehensive real-time visibility through the new Events APIs, offering complete operational history to enable faster troubleshooting and better decision-making. These capabilities enable customers to achieve improved operational agility, better resource utilization, and enhanced visibility into cluster operations, allowing AI/ML teams to focus on innovation rather than infrastructure management.

This feature is currently available for SageMaker HyperPod clusters using the EKS orchestrator. You can enable continuous provisioning by setting the NodeProvisioningMode parameter to "Continuous" when creating new HyperPod clusters using the CreateCluster API.

This feature is available in all AWS Regions where Amazon SageMaker HyperPod is supported. To learn more about continuous provisioning, see the Amazon SageMaker HyperPod User Guide.