Event-driven applications often need to process data in real-time. When you use AWS Lambda to process records from Apache Kafka topics, you frequently encounter two typical requirements: you need to p...

In this post, we introduce an alternative architecture to synchronize mainframe data to the cloud using Amazon Managed Streaming for Apache Kafka (Amazon MSK) for greater flexibility and scalability. ...

Amazon MSK now offers AI Agent Skills that give AI coding assistants expert, up-to-date guidance for operating Amazon MSK. The skills provide expert guidance for common operational tasks such as troubleshooting, sizing, configuring, monitoring, and migration from external Kafka clusters. Teams can leverage these skills to keep their clusters healthy and performant, and to migrate their external Kafka workloads to MSK Express to take advantage of up to 3 times more throughput per broker, scale up to 20 times faster, and reduced recovery time by 90 percent as compared to Standard brokers running Apache Kafka. The skills turn tasks that once required specialized knowledge into a guided experience developers can complete quickly, on their own. You can use the MSK skills with your existing AI coding agent - Kiro, Claude Code, or Cursor. To get started, configure the Agent Toolkit for AWS using the AWS CLI, then ask your coding agent a question, such as "What broker type and size should I use for my MSK cluster?" or "Is my Kafka cluster compatible with MSK Express?"

In this post, we address common challenges associated with manual MSK topic configuration management and present a robust Terraform-based solution. This solution supports both provisioned and serverle...

Amazon MSK Connect is now available in ten additional AWS Regions: Asia Pacific (Jakarta), Asia Pacific (Hong Kong), Asia Pacific (Osaka), Asia Pacific (Melbourne), Europe (Milan), Europe (Zurich), Middle East (Bahrain), Middle East (UAE), Africa (Cape Town), and Israel (Tel Aviv). MSK Connect enables you to run fully managed Kafka Connect clusters with Amazon Managed Streaming for Apache Kafka (Amazon MSK). With a few clicks, MSK Connect allows you to easily deploy, monitor, and scale connectors that move data in and out of Apache Kafka and Amazon MSK clusters from external systems such as databases, file systems, and search indices. MSK Connect eliminates the need to provision and maintain cluster infrastructure. Connectors scale automatically in response to increases in usage and you pay only for the resources you use. With full compatibility with Kafka Connect, it is easy to migrate workloads without code changes. MSK Connect will support both Amazon MSK-managed and self-managed Apache Kafka clusters. You can get started with MSK Connect from the Amazon MSK console or the Amazon CLI. Visit the AWS Regions page for all the regions where Amazon MSK is available. To get started visit, the MSK Connect product page, pricing page, and the Amazon MSK Developer Guide.

Amazon MSK now offers AI Agent Skills that give AI coding assistants expert, up-to-date guidance for operating Amazon MSK. The skills provide expert guidance for common operational tasks such as troubleshooting, sizing, configuring, monitoring, and migration from external Kafka clusters. Teams can leverage these skills to keep their clusters healthy and performant, and to migrate their external Kafka workloads to MSK Express to take advantage of up to 3 times more throughput per broker, scale up to 20 times faster, and reduced recovery time by 90 percent as compared to Standard brokers running Apache Kafka. The skills turn tasks that once required specialized knowledge into a guided experience developers can complete quickly, on their own. You can use the MSK skills with your existing AI coding agent - Kiro, Claude Code, or Cursor. To get started, configure the https://docs.aws.amazon.com/agent-toolkit/latest/userguide/what-is-agent-toolkit.html using the https://docs.aws.amazon.com/agent-toolkit/latest/userguide/aws-cli.html, then ask your coding agent a question, such as "What broker type and size should I use for my MSK cluster?" or "Is my Kafka cluster compatible with MSK Express?"

Amazon MSK (Managed Streaming for Apache Kafka) now supports new Managed Streaming for Apache Flink blueprints to generate vector-embeddings using Amazon Bedrock, making it easier to build real-time AI applications powered by up-to-date, contextual data. This blueprint simplifies the process of incorporating the latest data from your Amazon MSK streaming pipelines into your generative AI models, eliminating the need to write custom code to integrate real-time data streams, vector databases, and large language models. With just a few clicks, customers can configure the blueprint to continuously generate vector embeddings using Bedrock's embedding models, then index those embeddings in Amazon OpenSearch for their Amazon MSK data streams. This allows customers to combine the context from real-time data with Bedrock's powerful large language models to generate accurate, up-to-date AI responses without writing custom code. Customers can also choose to improve the efficiency of data retrieval using built-in support for data chunking techniques from LangChain, an open-source library, supporting high-quality inputs for model ingestion. The blueprint manages the data integration and processing between MSK, the chosen embedding model, and the Open Search vector store, allowing customers to focus on building their AI applications rather than managing the underlying integration. Real-time vector embedding blueprint is generally available in the US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Paris), Europe (London), Europe (Ireland) and South America (Sao Paulo) AWS Regions. Visit the Amazon MSK documentation for the list of additional Regions, which will be supported over the next few weeks. To learn more about how to use the blueprint to generate real-time vector embeddings from your Amazon MSK data, visit the AWS blog.

Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 4.1, introducing Queues as a preview feature, a new Streams Rebalance Protocol in early access, and Eligible Leader Replicas (ELR). Along with these features, Apache Kafka version 4.1 includes various bug fixes and improvements. For more details, please refer to the https://downloads.apache.org/kafka/4.1.0/RELEASE_NOTES.html A key highlight of Kafka 4.1 is the introduction of Queues as a preview feature. Customers can use multiple consumers to process messages from the same topic partitions, improving parallelism and throughput for workloads that need point-to-point message delivery. The new Streams Rebalance Protocol builds upon Kafka 4.0's consumer rebalance protocol, extending broker coordination capabilities to Kafka Streams for optimized task assignments and rebalancing. Additionally, ELR is now enabled by default to strengthen availability. To start using Apache Kafka 4.1 on Amazon MSK, simply select version 4.1.x when creating a new cluster via the AWS Management Console, AWS CLI, or AWS SDKs. You can also upgrade existing MSK provisioned clusters with an in-place rolling update. Amazon MSK orchestrates broker restarts to maintain availability and protect your data during the upgrade. Kafka version 4.1 support is available today across all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where Amazon MSK is offered. To learn how to get started, see the https://docs.aws.amazon.com/msk/latest/developerguide/getting-started.html.

https://docs.aws.amazon.com/glue/latest/dg/schema-registry.html (GSR) has now expanded the programming language support for GSR Client library to include C# support along with existing Java support. C# applications integrating with Apache Kafka or https://aws.amazon.com/msk/, https://aws.amazon.com/kinesis/data-streams/, and Apache Flink or https://aws.amazon.com/managed-service-apache-flink/ can now interact with AWS Glue Schema Registry to maintain data quality and schema compatibility in streaming data applications. AWS Glue Schema Registry, a serverless feature of https://aws.amazon.com/glue/, enables you to validate and control the evolution of streaming data using registered schemas at no additional charge. Schemas define the structure and format of data records produced by applications. Using AWS Glue Schema Registry, you can centrally manage and enforce schema definitions across your data ecosystem. This ensures consistency of schemas across applications and enables seamless data integration between producers and consumers. Through centralized schema validation, teams can maintain data quality standards and evolve their schemas in a controlled manner.   C# support is available across all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where Glue Schema Registry is available. Visit the Glue Schema Registry https://docs.aws.amazon.com/glue/latest/dg/schema-registry-gs-serde-csharp.html , and https://www.nuget.org/packages/AWS.Glue.SchemaRegistry to get started with C# integration.  

Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 3.8 on Express Brokers, introducing new features, bug fixes, and performance improvements for Kafka workloads running on Express Brokers. The update brings enhancements in data compression capabilities. You can now configure compression levels for lz4, zstd, and gzip formats, allowing precise control over the balance between compression efficiency and resource usage. The transition to version 3.8 is straightforward - you can either create new clusters or upgrade your existing ones. For a detailed understanding of the specific improvements and bug fixes included in this release, see the Apache Kafka release notes for version 3.8.

AWS Lambda launches enhanced error handling capabilities for Amazon Managed Streaming for Apache Kafka (MSK) and self-managed Apache Kafka (SMK) event sources. These capabilities allow customers to build custom retry configurations, optimize retries of failed messages, and send failed events to a Kafka topic as an on-failure destination, enabling customers to build resilient Kafka workloads with robust error handling strategies. Customers use Kafka event source mappings (ESM) with their Lambda functions to build their mission-critical Kafka applications. Kafka ESM offers robust error handling of failed events by retrying events with exponential backoff, and retaining failed events in on-failure destinations like Amazon SQS, Amazon S3, Amazon SNS. However, customers need customized error handling to meet stringent business and performance requirements. With this launch, developers can now exercise precise control over failed event processing and leverage Kafka topics as an additional on-failure destination when using Provisioned mode for Kafka ESM. Customers can now define specific retry limits and time boundaries for retry, automatically discarding failed records beyond these limits to customer-specified destination. They can now also set automatic retries of failed records in the batch and enhance their function code to report individual failed messages, optimizing the retry process. This feature is available in all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where AWS Lambda’s Provisioned mode for Kafka ESM is available. To enable these capabilities, provide configuration parameters for your Kafka ESM in the ESM API, AWS Console, and AWS CLI. To learn more, read the https://docs.aws.amazon.com/lambda/latest/dg/with-kafka-esm.html and https://aws.amazon.com/lambda/pricing/. 

Effective today, all new Amazon MSK Provisioned clusters with Express brokers will support Intelligent Rebalancing at no additional cost. This new capability makes it effortless for customers to execute automatic partition balancing operations when scaling their Kafka clusters up or down. Intelligent Rebalancing maximizes the capacity utilization of MSK Express-based clusters by optimally rebalancing Kafka resources on them for better performance, eliminating the need for customers to manage partitions themselves or via third-party tools. Intelligent Rebalancing performs these operations up to 180 times faster compared to Standard brokers. MSK Express brokers are designed to deliver up to three times more throughput per-broker, scale up to 20 times faster, and reduce recovery time by 90 percent as compared to Standard brokers running Apache Kafka. With Intelligent Rebalancing, MSK Express-based clusters are continuously monitored for resource imbalance or overload based on intelligent Amazon MSK defaults to maximize cluster performance. When required, brokers are efficiently scaled, without affecting cluster availability for clients to produce and consume data. Customers can now take full advantage of the scaling and performance benefits of MSK Provisioned clusters for Express brokers while simplifying cluster management operations. Intelligent Rebalancing is being rolled out for all new MSK Provisioned clusters with Express brokers in all AWS Regions, where Express brokers are available. Intelligent Rebalancing does not require any additional configuration or setup to get started. To learn more, see the https://docs.aws.amazon.com/msk/latest/developerguide/intelligent-rebalancing.html.

Amazon MSK Connect is now available in the Asia Pacific (Malaysia) Region. MSK Connect enables you to run fully managed Kafka Connect clusters with Amazon Managed Streaming for Apache Kafka (Amazon MSK). With a few clicks, MSK Connect allows you to easily deploy, monitor, and scale connectors that move data in and out of Apache Kafka and Amazon MSK clusters from external systems such as databases, file systems, and search indices. MSK Connect eliminates the need to provision and maintain cluster infrastructure. Connectors scale automatically in response to increases in usage and you pay only for the resources you use. With full compatibility with Kafka Connect, it is easy to migrate workloads without code changes. MSK Connect will support both Amazon MSK-managed and self-managed Apache Kafka clusters. You can get started with MSK Connect from the Amazon MSK console or the Amazon CLI. Visit the AWS Regions page for all the regions where Amazon MSK is available. To get started visit, the MSK Connect product page, pricing page, and the Amazon MSK Developer Guide.

Effective today, all new Amazon MSK Provisioned clusters with Express brokers will support Intelligent Rebalancing at no additional cost. This new capability makes it effortless for customers to execute automatic partition balancing operations when scaling their Kafka clusters up or down. Intelligent Rebalancing maximizes the capacity utilization of MSK Express-based clusters by optimally rebalancing Kafka resources on them for better performance, eliminating the need for customers to manage partitions themselves or via third-party tools. Intelligent Rebalancing performs these operations up to 180 times faster compared to Standard brokers. MSK Express brokers are designed to deliver up to three times more throughput per-broker, scale up to 20 times faster, and reduce recovery time by 90 percent as compared to Standard brokers running Apache Kafka. With Intelligent Rebalancing, MSK Express-based clusters are continuously monitored for resource imbalance or overload based on intelligent Amazon MSK defaults to maximize cluster performance. When required, brokers are efficiently scaled, without affecting cluster availability for clients to produce and consume data. Customers can now take full advantage of the scaling and performance benefits of MSK Provisioned clusters for Express brokers while simplifying cluster management operations. Intelligent Rebalancing is being rolled out for all new MSK Provisioned clusters with Express brokers in all AWS Regions, where Express brokers are available. Intelligent Rebalancing does not require any additional configuration or setup to get started. To learn more, see the Amazon MSK Developer Guide.

AWS Glue Schema Registry (GSR) has now expanded the programming language support for GSR Client library to include C# support along with existing Java support. C# applications integrating with Apache Kafka or Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Kinesis Data Streams, and Apache Flink or Amazon Managed Service for Apache Flink can now interact with AWS Glue Schema Registry to maintain data quality and schema compatibility in streaming data applications. AWS Glue Schema Registry, a serverless feature of AWS Glue, enables you to validate and control the evolution of streaming data using registered schemas at no additional charge. Schemas define the structure and format of data records produced by applications. Using AWS Glue Schema Registry, you can centrally manage and enforce schema definitions across your data ecosystem. This ensures consistency of schemas across applications and enables seamless data integration between producers and consumers. Through centralized schema validation, teams can maintain data quality standards and evolve their schemas in a controlled manner.   C# support is available across all AWS regions where Glue Schema Registry is available. Visit the Glue Schema Registry developer guide, and SDK to get started with C# integration.

You can now use Amazon MSK Replicator to replicate streaming data across Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters in Asia Pacific (Hyderabad) and Asia Pacific (Malaysia) Regions. MSK Replicator is a feature of Amazon MSK that enables you to reliably replicate data across Amazon MSK clusters in different or the same AWS Region(s) in a few clicks. With MSK Replicator, you can easily build regionally resilient streaming applications for increased availability and business continuity. MSK Replicator provides automatic asynchronous replication across MSK clusters, eliminating the need to write custom code, manage infrastructure, or setup cross-region networking. MSK Replicator automatically scales the underlying resources so that you can replicate data on-demand without having to monitor or scale capacity. MSK Replicator also replicates the necessary Kafka metadata including topic configurations, Access Control Lists (ACLs), and consumer group offsets. If an unexpected event occurs in a region, you can failover to the other AWS Region and seamlessly resume processing. With this launch, MSK Replicator is now available in the following AWS Regions: US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Europe (Stockholm), South America (Sao Paulo), China (Beijing), China (Ningxia), Asia Pacific (Hyderabad), and Asia Pacific (Malaysia). You can get started with MSK Replicator from the Amazon MSK console or the Amazon CLI. To learn more, visit the MSK Replicator product page, pricing page, and documentation.

https://aws.amazon.com/msk/ (Amazon MSK) now supports Apache Kafka version 3.8 on Express Brokers, introducing new features, bug fixes, and performance improvements for Kafka workloads running on Express Brokers. The update brings enhancements in data compression capabilities. You can now configure compression levels for lz4, zstd, and gzip formats, allowing precise control over the balance between compression efficiency and resource usage. The transition to version 3.8 is straightforward - you can either create new clusters or upgrade your existing ones. For a detailed understanding of the specific improvements and bug fixes included in this release, see the https://downloads.apache.org/kafka/3.8.0/RELEASE_NOTES.html.

Amazon MSK Replicator now supports mutual TLS (mTLS) authentication for data replication from external Apache Kafka clusters - including on-premises, self-managed on AWS, or other cloud providers - to https://aws.amazon.com/msk/features/express-brokers-for-amazon-msk/. With this capability, external Apache Kafka clusters configured with mTLS authentication can now use MSK Replicator to migrate workloads to MSK Express brokers, support disaster recovery by using MSK Express-based clusters as a failover or backup target, and enable data distribution across hybrid and multi-cloud environments. MSK Replicator is a feature of Amazon MSK that automates data replication between Kafka clusters, eliminating the need to manage custom replication infrastructure or configure open-source tools. Previously, MSK Replicator supported SASL/SCRAM authentication only for connecting to external Apache Kafka clusters. With this launch, you can now also use mTLS authentication with MSK Replicator to replicate data from external Kafka clusters to Express brokers on Amazon MSK. Unlike self-managed replication tools, MSK Replicator lets you retain your original Kafka topic names during replication while automatically avoiding infinite replication loops. It also synchronizes consumer group offsets bidirectionally, enabling you to move producers and consumers across clusters independently, in any order, without coordination constraints or the risk of data loss. This new capability is supported in all AWS Regions where MSK Express brokers are available. Visit the MSK Replicator https://docs.aws.amazon.com/msk/latest/developerguide/msk-replicator.html, https://aws.amazon.com/msk/features/msk-replicator/, https://aws.amazon.com/msk/pricing/, and this https://aws.amazon.com/blogs/big-data/migrate-third-party-and-self-managed-apache-kafka-clusters-to-amazon-msk-express-brokers-with-amazon-msk-replicator/ to learn more.

Amazon MSK Connect is now available in ten additional AWS Regions: Asia Pacific (Jakarta), Asia Pacific (Hong Kong), Asia Pacific (Osaka), Asia Pacific (Melbourne), Europe (Milan), Europe (Zurich), Middle East (Bahrain), Middle East (UAE), Africa (Cape Town), and Israel (Tel Aviv). MSK Connect enables you to run fully managed Kafka Connect clusters with Amazon Managed Streaming for Apache Kafka (Amazon MSK). With a few clicks, MSK Connect allows you to easily deploy, monitor, and scale connectors that move data in and out of Apache Kafka and Amazon MSK clusters from external systems such as databases, file systems, and search indices. MSK Connect eliminates the need to provision and maintain cluster infrastructure. Connectors scale automatically in response to increases in usage and you pay only for the resources you use. With full compatibility with Kafka Connect, it is easy to migrate workloads without code changes. MSK Connect will support both Amazon MSK-managed and self-managed Apache Kafka clusters. You can get started with MSK Connect from the Amazon MSK console or the Amazon CLI. Visit the https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ for all the regions where Amazon MSK is available. To get started visit, the MSK Connect https://aws.amazon.com/msk/features/msk-connect/, https://aws.amazon.com/msk/pricing/, and the https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect.html.

https://aws.amazon.com/msk/ (Amazon MSK) has added support for Express brokers in eight additional AWS Regions: AWS GovCloud (US-West), AWS GovCloud (US-East), Asia Pacific (Jakarta), Asia Pacific (Melbourne), Asia Pacific (Osaka), Europe (Zurich), Israel (Tel Aviv), and Asia Pacific (Hong Kong). Express brokers are a new broker type for Amazon MSK Provisioned designed to deliver up to 3x more throughput per broker, scale up to 20x faster, and reduce recovery time by 90% as compared to standard Apache Kafka brokers. Express brokers come pre-configured with Kafka best practices by default, support all Kafka APIs, and provide the same low-latency performance that Amazon MSK customers expect, so they can continue using existing client applications without any changes. You can now create an MSK cluster with Express brokers in these AWS Regions from the Amazon MSK https://us-east-2.signin.aws.amazon.com/oauth?client_id=arn%3Aaws%3Asignin%3A%3A%3Aconsole%2Fcanvas&code_challenge=Rx9g3IIBch8VAmdpe2UX_lsWypVnS0XbubpHyQf1ncU&code_challenge_method=SHA-256&response_type=code&redirect_uri=https%3A%2F%2Fconsole.aws.amazon.com%2Fconsole%2Fhome%3FhashArgs%3D%2523%26isauthcode%3Dtrue%26nc2%3Dh_ct%26src%3Dheader-signin%26state%3DhashArgsFromTB_us-east-2_9a482948c806810f. To get started visit, the MSK https://aws.amazon.com/msk/features/, https://aws.amazon.com/msk/pricing/, and the https://docs.aws.amazon.com/msk/latest/developerguide/what-is-msk.html.

Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 4.1, introducing Queues as a preview feature, a new Streams Rebalance Protocol in early access, and Eligible Leader Replicas (ELR). Along with these features, Apache Kafka version 4.1 includes various bug fixes and improvements. For more details, please refer to the Apache Kafka release notes for version 4.1. A key highlight of Kafka 4.1 is the introduction of Queues as a preview feature. Customers can use multiple consumers to process messages from the same topic partitions, improving parallelism and throughput for workloads that need point-to-point message delivery. The new Streams Rebalance Protocol builds upon Kafka 4.0's consumer rebalance protocol, extending broker coordination capabilities to Kafka Streams for optimized task assignments and rebalancing. Additionally, ELR is now enabled by default to strengthen availability. To start using Apache Kafka 4.1 on Amazon MSK, simply select version 4.1.x when creating a new cluster via the AWS Management Console, AWS CLI, or AWS SDKs. You can also upgrade existing MSK provisioned clusters with an in-place rolling update. Amazon MSK orchestrates broker restarts to maintain availability and protect your data during the upgrade. Kafka version 4.1 support is available today across all AWS regions where Amazon MSK is offered. To learn how to get started, see the Amazon MSK Developer Guide.

You can now connect your Apache Kafka applications to https://aws.amazon.com/msk/features/msk-serverless/ in the South America (São Paulo) https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/. Amazon MSK is a fully managed service for Apache Kafka and Kafka Connect that makes it easier for you to build and run applications that use Apache Kafka as a data store. Amazon MSK Serverless is a cluster type for Amazon MSK that allows you to run Apache Kafka without having to manage and scale cluster capacity. MSK Serverless automatically provisions and scales compute and storage resources, so you can use Apache Kafka on demand. With these launches, Amazon MSK Serverless is now generally available in Asia Pacific (Sydney), Asia Pacific (Singapore), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Seoul), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (Stockholm), Europe (Paris), Europe (London), South America (São Paulo), US East (N. Virginia), US East (Ohio), and US West (Oregon) AWS regions. To learn more and get started, see our https://aws.amazon.com/msk/features/msk-serverless/.

AWS Lambda launches enhanced error handling capabilities for Amazon Managed Streaming for Apache Kafka (MSK) and self-managed Apache Kafka (SMK) event sources. These capabilities allow customers to build custom retry configurations, optimize retries of failed messages, and send failed events to a Kafka topic as an on-failure destination, enabling customers to build resilient Kafka workloads with robust error handling strategies. Customers use Kafka event source mappings (ESM) with their Lambda functions to build their mission-critical Kafka applications. Kafka ESM offers robust error handling of failed events by retrying events with exponential backoff, and retaining failed events in on-failure destinations like Amazon SQS, Amazon S3, Amazon SNS. However, customers need customized error handling to meet stringent business and performance requirements. With this launch, developers can now exercise precise control over failed event processing and leverage Kafka topics as an additional on-failure destination when using Provisioned mode for Kafka ESM. Customers can now define specific retry limits and time boundaries for retry, automatically discarding failed records beyond these limits to customer-specified destination. They can now also set automatic retries of failed records in the batch and enhance their function code to report individual failed messages, optimizing the retry process. This feature is available in all AWS Commercial Regions where AWS Lambda’s Provisioned mode for Kafka ESM is available. To enable these capabilities, provide configuration parameters for your Kafka ESM in the ESM API, AWS Console, and AWS CLI. To learn more, read the Lambda ESM documentation and AWS Lambda pricing.

Amazon MSK Connect is now available in the Asia Pacific (Malaysia) Region. MSK Connect enables you to run fully managed Kafka Connect clusters with Amazon Managed Streaming for Apache Kafka (Amazon MSK). With a few clicks, MSK Connect allows you to easily deploy, monitor, and scale connectors that move data in and out of Apache Kafka and Amazon MSK clusters from external systems such as databases, file systems, and search indices. MSK Connect eliminates the need to provision and maintain cluster infrastructure. Connectors scale automatically in response to increases in usage and you pay only for the resources you use. With full compatibility with Kafka Connect, it is easy to migrate workloads without code changes. MSK Connect will support both Amazon MSK-managed and self-managed Apache Kafka clusters. You can get started with MSK Connect from the Amazon MSK console or the Amazon CLI. Visit the https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ for all the regions where Amazon MSK is available. To get started visit, the MSK Connect https://aws.amazon.com/msk/features/msk-connect/, https://aws.amazon.com/msk/pricing/, and the https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect.html.

Amazon Managed Streaming for Apache Kafka (Amazon MSK) is now available in Asia Pacific (New Zealand) region. Customers can create Amazon MSK Provisioned clusters in this region starting today. Amazon MSK is a fully managed service for Apache Kafka and Kafka Connect that makes it easier for you to build and run applications that use Apache Kafka as a data store. Amazon MSK is fully compatible with Apache Kafka, which enables you to more quickly migrate your existing Apache Kafka workloads to Amazon MSK with confidence or build new ones from scratch. With Amazon MSK, you spend more time building innovative streaming applications and less time managing Kafka clusters. Amazon MSK offers two types of Apache Kafka provisioned broker - Standard brokers and Express brokers. Standard brokers offer the most flexibility to configure your cluster’s performance. You can configure availability, durability, throughput, and latency. You also control the storage configurations on your cluster and are responsible for managing storage provisioning and utilization. Express brokers are a new broker type for Amazon MSK Provisioned designed to deliver up to 3x more throughput per broker, scale up to 20x faster, up to 5x more partitions per broker, and reduce recovery time by 90% as compared to Standard brokers. Express brokers come pre-configured with Kafka best practices by default, support all Kafka APIs, and provide the same low-latency performance that Amazon MSK customers expect, so they can continue using existing client applications without any changes. You can now create an MSK provisioned cluster with Standard or Express brokers in Asia Pacific (New Zealand) Region through the Amazon MSK console or the Amazon CLI. To get started, see the Amazon MSK Developer Guide.