AWS Step Functions expands data source options and improves observability for Distributed Map

AWS Step Functions now supports additional data sources and new observability metrics for Distributed Map. AWS Step Functions is a visual workflow service capable of orchestrating over 14,000+ API actions from over 220 AWS services to build distributed applications and data processing workloads. Distributed Map is a task state of Step Functions that runs the same process for multiple entries in a data set.

With this update, Distributed Map now supports additional data inputs, so you can orchestrate large-scale analytics and ETL workflows. You can now process AWS Athena data manifest and Parquet files directly, iterate over S3 objects under a specified prefix using S3ListObjectsV2, read from JSON objects and natively extract array data from JSON object from Amazon S3 or state input, eliminating the need for custom pre-processing. You also now get visibility into your Distributed Map usage with the following metrics, such as Approximate Open Map Runs Count, Open Map Run Limit, and Approximate Map Runs Backlog Size.

New input sources and improved observability for Distributed Map are available in all commercial AWS Regions where AWS Step Functions is available. To get started, you can use the Distributed Map mode today in the AWS Step Functions console. To learn more, visit the Step Functions developer guide.