AWS Step Functions expands data source and output options for Distributed Map

AWS Step Functions now supports additional data sources and output options for Distributed Map, enabling more flexible large-scale parallel processing workflows. Distributed map can now process data from JSON Lines (JSONL) and a broader range of delimited file formats stored in Amazon S3. Additionally, distributed map offers new output transformations for greater control over result formatting.

AWS Step Functions is a visual workflow service capable of orchestrating over 14,000+ API actions from over 220 AWS services to build distributed applications and data processing workloads. With this update, you can more easily iterate over and process diverse datasets using Step Functions distributed map. In addition to existing JSON and comma separated value (CSV) file support, distributed map now supports JSONL, and new delimited file formats, such as semicolon-delimited files and tab-delimited files. This expands processing capabilities to a wider range of data formats without custom pre-processing. The new flexible output transformations give you more control over how results are formatted, enabling easier aggregation of results and simplifying integration with downstream systems.

To get started, you can use the distributed map mode today in the AWS Step Functions console or read this blog post. To learn more, visit the Step Functions developer guide.