Amazon CloudWatch Logs launches the ability to transform and enrich logs
Amazon CloudWatch Logs announces log transformation and enrichment to improve log analytics at scale with consistent, and context-rich format. Customers can add structure to their logs using pre-configured templates for common AWS services such as AWS Web Application Firewall (WAF), Route53, or build custom transformers with native parsers such as Grok. Customers can also rename existing attributes and add additional metadata to their logs such as accountId, and region.
Logs emitted from various sources vary widely in format and attribute names, which makes analysis across sources cumbersome. With today’s launch, customers can simplify their log analytics experience by transforming all their logs into a standardized JSON structure. Transformed logs can be leveraged to accelerate analytics experience using field indexes, discovered fields in CloudWatch Logs Insights, provide flexibility in alarming using metric filters and forwarding via subscription filters. Customers can manage log transformations natively within CloudWatch without needing to setup complex pipelines.
Log transformation and enrichment capability is available in all AWS Commercial Regions, and included with existing Standard log class ingestion price. Logs Store (Archival) costs will be based on log size after transformation, which may exceed the original log volume. With a few clicks in the Amazon CloudWatch Console, customers can configure transformers at log group level. Alternatively, customers can setup transformers at account, or log group level using AWS Command Line Interface (AWS CLI), AWS CloudFormation, AWS Cloud Development Kit (AWS CDK), and AWS SDKs. Read the documentation to learn more about this capability.