Kafka Adds KRaft-Based Authorizer
Written by Kay Ewbank   
Thursday, 19 May 2022

Apache Kafka, the distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications, has been updated with improvements including a KRaft-based authorizer and a proposal for marking KRaft mode as production ready in Apache Kafka 3.3.

Kafka was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers.

kafka

KRaft is Kafka Raft, the consensus protocol that has been added to remove Apache Kafka's dependency on ZooKeeper for metadata management. Until now, users making use of KRaft mode have still had to have an Apache ZooKeeper instance if they want to use AclAuthorizer. The new built-in Authorizer for KRaft mode doesn't depend on ZooKeeper, so avoids this.

This release also sees moves to using KRaft as production-ready. KRaft mode is popular as it addresses many urgent scalability and performance issues in Kafka, as well as eliminating the need to run an Apache ZooKeeper cluster alongside every Kafka cluster. KRaft entered preview status in Kafka 3.0, and this release opens the discussion on it becoming production-ready for new clusters in the upcoming Kafka 3.3 release. If this is agreed, ZooKeeper mode will be deprecated in Kafka 3.4 and removed entirely in Kafka 4.0.

Another improvement means the Kafka controller can send a hint to the partition leader to recover the partition in situations where the transaction state has been left inconsistent following an unclean election. Another change means that Kafka Streams can distribute its standby replicas over distinct “racks”.

Kafka 3.2 is available now. 

kafka

More Information

Kafka Website

Related Articles

Kafka 3.1 Adds OIDC Support

Kafka 3.0 Released With KRaft  

Kafka Replaces Zookeeper With Quorum

Apache Kafka 2.7 Updates Broker

Kafka 2.5 Adds New Metrics And Improves Security

Kafka 2 Adds Support For ACLs

Kafka Graphs Framework Extends Kafka Streams

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Suspended For Claiming AI Is Sentient
15/06/2022

Can a large language model be sentient? This is a news item that has been doing the rounds and just about everyone has some comment to make about it.  But who better to comment than a man accused [ ... ]



200 Years Ago Charles Babbage Proposed His Difference Engine
14/06/2022

Two hundred years ago today, on June 14, 1822 Charles Babbage presented a one-page note to the Royal Astronomical Society in London setting out his plans to build a Difference Engine that wo [ ... ]


More News

pythondata

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Thursday, 19 May 2022 )