Announcing AWS Lambda Kafka event source mapping integration in Amazon MSK Console

AWS announces Lambda’s Kafka event source mapping (ESM) integration in the Amazon MSK Console, streamlining the process of connecting MSK topics to Lambda functions. This capability allows you to simply provide your topic and target function in the MSK Console while the integration handles ESM configuration automatically, enabling you to trigger Lambda functions from MSK topics without switching consoles.

Customers use MSK as an event source for Lambda functions to build responsive event-driven Kafka applications. Previously, configuring MSK as an event source required navigating between MSK and Lambda consoles to provide parameters like cluster details, authentication method, and network configuration. The new integrated experience brings Lambda ESM configuration directly into the MSK Console with a simplified interface requiring only target function and topic name as mandatory fields. The integration handles ESM creation with optimized defaults for authentication and event polling configurations, and can automatically generate the required Lambda execution role permissions for MSK cluster access. To optimize latency and throughput, and to remove the need for networking setup, the integration uses Provisioned Mode for ESM as the recommended default. These improvements streamline MSK integration with Lambda and reduce configuration errors, enabling you to quickly get started with your MSK and Lambda applications.

This feature is generally available in all AWS Commercial Regions where both Amazon MSK and AWS Lambda are available, except Asia Pacific (Thailand), Asia Pacific (Malaysia), Israel (Tel Aviv), Asia Pacific (Taipei), and Canada West (Calgary).

You can configure Lambda’s Kafka event source mapping from the MSK Console by navigating to your MSK cluster and providing the topic, Lambda function, and optional fields under the Lambda integration tab. Standard Lambda pricing and MSK pricing applies. To learn more, read Lambda developer guide and MSK developer guide.