| Kafka 3.0 Released With KRaft |
| Written by Kay Ewbank | |||
| Friday, 24 September 2021 | |||
|
Apache has released Kafka 3.0 with new features, breaking API changes, and improvements to KRaft—Apache Kafka’s built-in consensus mechanism that will replace Apache ZooKeeper. Apache Kafka is a distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications.
It was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers. One major improvement to Apache 3.0 is the more mature version of KRaft, the internal Raft implementation that allows you to run Kafka brokers without Apache ZooKeeper. KRaft is not yet recommended for production, but the team says they have made many improvements to the KRaft metadata and APIs. A new major feature in 3.0 is the ability for KRaft Controllers and KRaft Brokers to generate, replicate, and load snapshots for the metadata topic partition that is used by the Kafka Cluster to store and replicate metadata information about the cluster like Broker configuration, topic partition assignment, and leadership, Kafka Connect is another area to have been improved. Connect is a Kafka component that acts as a data hub for data integration between databases, key-value stores, search indexes, and file systems. The new version handles task restarts more smoothly. Until now, if you wanted to restart the whole connector, you had to make individual calls to restart the Connector instance and the Task instances. In 3.0, KIP-745 gives the ability to the users to restart either all or only the failed of a connector’s Connector and Task instances with a single call. Kafka Streams has improvements in timestamp-based synchronization. The way Streams tasks fetch records has been enhanced, and this is supported by a new method in the Kafka Consumer API to deal locally with the consumer lag of a specific partition. Another change of note is to MirrorMaker.This is a stand-alone tool for copying data between two Apache Kafka clusters, and it now has more flexible configuration options. Kafka 3 is available now.
More InformationRelated ArticlesKafka Replaces Zookeeper With Quorum Apache Kafka 2.7 Updates Broker Kafka 2.5 Adds New Metrics And Improves Security Kafka Graphs Framework Extends Kafka Streams
To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin. 
Comments
or email your comment to: comments@i-programmer.info |
