Apache Kafka is now available in version 1.0 as it is now stable enough for business use. The new release of the distributed streaming engine is more tolerant of disk failures, supports Java 9, and has better metrics.

The Apache Kafka distributed data streaming engine has finally hit version 1.0, a milestone that indicates the platform is finally ready for mainstream business. Kafka is a message broker that was originally developed at LinkedIn. It was then donated to Apache and made open source. Its benefits are that it is lightweight, scalable, and fast. Unlike traditional message brokers that just handle the interchange of messages and data streams between languages, Kafka provides low-latency, high-throughput, fault-tolerant publish and subscribe pipelines and is able to process streams of events. It can publish and subscribe to streams of data like a messaging system, process streams of data efficiently and in real time and store streams of data safely in a distributed, replicated cluster. 

The improvements to the new version include performance improvements with exactly-once semantics, significantly faster TLS and CRC32C implementations with Java 9 support, significantly faster controlled shutdown, and better JBOD support.

The achievement of exactly-once semantics means that if a problem occurs somewhere in the distributed system, perhaps a broker crashing or a network problem, and the producer retries sending the message, that message will be delivered exactly once to the end consumer. This is the ideal outcome, but is difficult to achieve because it requires the messaging system itself and the application producing and consuming the messages to cooperate.

Another improvement in the new version is to the Streams API. Kafka Streams is a client library for building applications and microservices, where the input and output data are stored in Kafka clusters. The builder API for Streams has been improved to hide internal methods; and a new API has been added to expose the state of active tasks at runtime. There's also a new cogroup API that can be used to deal with partitioned aggregates with fewer StateStores and fewer moving parts in your code.

Another improvement is the ability of Kafka to tolerate disk failures better. Until now, JBOD (Just a Bunch Of Disks) storage configurations have not been recommended, so requiring RAID instead. With this release, Kafka will handle disk failures better so that a single disk failure in a JBOD broker will not bring the entire broker down. Instead, the broker will continue serving any log files that remain on functioning disks.
 

More Information

Kafka Website

Related Articles

Comparing Kafka To RabbitMQ

Apache Kafka Adds New Streams API

GoKa Stream Processing For Kafka

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info