AWS Glue now enables continuous logging for Spark ETL jobs
AWS Glue now provides continuous logs to track real-time progress of executing Apache Spark stages in ETL jobs . You can access different log streams for Apache Spark driver and executors in Amazon CloudWatch and filter out highly verbose Apache Spark log messages making it easier to monitor and debug your ETL jobs.
AWS Glue now provides continuous logs to track real-time progress of executing Apache Spark stages in ETL jobs . You can access different log streams for Apache Spark driver and executors in Amazon CloudWatch and filter out highly verbose Apache Spark log messages making it easier to monitor and debug your ETL jobs.
To learn more about this feature, please visit our documentation.