Apache Storm,  a distributed real-time analytics system for Hadoop that can be used for processing large volumes of rapidly changing data, has been released on Microsoft Azure HDInsight.

General availability of Storm comes as part of the latest update to HDInsight, alongside the news that HDInsight is now available on Linux.

The release has been described by Oliver Chiu on the Azure blog as being due to:

‘a natural decision to give customers the choice of running their big data Hadoop workloads on both Windows and Linux’,

presumably partially driven by the fact that 20 percent of Azure Virtual Machines run on Linux.

Apache Storm was made available in preview last October as clusters on HDInsight, and Microsoft is now announcing the general availability of Storm as a deployment option with the aim of making Hadoop easier to use. The release also includes support for Hadoop 2.6, new Virtual Machine sizes, cluster scaling, and built-in integration to Azure DocumentDB.

In this Channel 9 video in the Data Exposed series Asad Khan gives the background to the Azure Storm service and its latest features:

The most interesting aspect for developers is the fact that Storm is now available for both .NET and Java, and integration with Visual Studio allows you to work in VS on Storm apps.

This includes the ability to use Storm spouts and bolts written in other languages. Details of how to do this are included in and article Develop C# topologies for Apache Storm on HDInsight using Visual Studio that is part of the Microsoft Azure documentation. 

Microsoft is making use of the Hortonworks availability of HDP 2.2 to release the next version of HDInsight built on Hadoop 2.6, Hive and Pig. This version of HDInsight includes work done on Stinger.next to speed up Hadoop queries. The new version also includes cluster scaling in Azure HDInsight, so you can change the number of nodes of a running HDInsight cluster without having to delete and recreate a new cluster.  

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info