Amazon SageMaker now supports data lineage in IAM-based domains

Amazon SageMaker Unified Studio now supports OpenLineage compatible data lineage in IAM-based domains, capturing lineage events from Apache Spark jobs run on Amazon EMR, AWS Glue, SageMaker Visual ETL, and notebooks. This capability is already available in IAM Identity Center-based domains. The interactive lineage graph provides an aggregate visual representation of how data moves from source to consumption, with configurable graph depth, event timestamp mode for detailed column-level lineage, and a dataset-only view for simplified visualization. For both IAM-based and IAM Identity Center-based domains, you can programmatically publish, query, and manage data lineage from OpenLineage compatible applications. You can now also remove published events using the DeleteLineageEvent API.

This feature is available in all AWS Regions where Amazon SageMaker Unified Studio is available. To get started, visit the Amazon SageMaker Unified Studio documentation and API reference.