Kafka 1 Becomes More Tolerant
Kafka 1 Becomes More Tolerant
Written by Kay Ewbank   
Tuesday, 07 November 2017

 Apache Kafka is now available in version 1.0 as it is now stable enough for business use. The new release of the distributed streaming engine is more tolerant of disk failures, supports Java 9, and has better metrics.


kakfalogo

The Apache Kafka distributed data streaming engine has finally hit version 1.0, a milestone that indicates the platform is finally ready for mainstream business. Kafka is a message broker that was originally developed at LinkedIn. It was then donated to Apache and made open source. Its benefits are that it is lightweight, scalable, and fast. Unlike traditional message brokers that just handle the interchange of messages and data streams between languages, Kafka provides low-latency, high-throughput, fault-tolerant publish and subscribe pipelines and is able to process streams of events. It can publish and subscribe to streams of data like a messaging system, process streams of data efficiently and in real time and store streams of data safely in a distributed, replicated cluster. 

The improvements to the new version include performance improvements with exactly-once semantics, significantly faster TLS and CRC32C implementations with Java 9 support, significantly faster controlled shutdown, and better JBOD support.

The achievement of exactly-once semantics means that if a problem occurs somewhere in the distributed system, perhaps a broker crashing or a network problem, and the producer retries sending the message, that message will be delivered exactly once to the end consumer. This is the ideal outcome, but is difficult to achieve because it requires the messaging system itself and the application producing and consuming the messages to cooperate.

Another improvement in the new version is to the Streams API. Kafka Streams is a client library for building applications and microservices, where the input and output data are stored in Kafka clusters. The builder API for Streams has been improved to hide internal methods; and a new API has been added to expose the state of active tasks at runtime. There's also a new cogroup API that can be used to deal with partitioned aggregates with fewer StateStores and fewer moving parts in your code.

Another improvement is the ability of Kafka to tolerate disk failures better. Until now, JBOD (Just a Bunch Of Disks) storage configurations have not been recommended, so requiring RAID instead. With this release, Kafka will handle disk failures better so that a single disk failure in a JBOD broker will not bring the entire broker down. Instead, the broker will continue serving any log files that remain on functioning disks.
 

 kakfalogo

 

More Information

Kafka Website

Related Articles

Comparing Kafka To RabbitMQ

Apache Kafka Adds New Streams API

GoKa Stream Processing For Kafka

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.

 

Banner


Angular 5 - Smaller, Faster, Easier To Use
06/11/2017

There's a new major release of Angular that the developers claim is smaller, faster, and easier to use. New features of Angular  5, codenamed pentagonal-donut, include automatic use of the build  [ ... ]



Prepare to Run a Code Club with Future Learn
21/11/2017

The Raspberry Pi Foundation has just opened the doors on a new free, short online course on the Future Learn platform. The aim of 'Prepare to Run a Code Club' is to assist new volunteers who want [ ... ]


More News

 

 
 

 

blog comments powered by Disqus

 
 

   
Banner
Banner
RSS feed of news items only
I Programmer News
Copyright © 2017 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.