Spark Announcements
Written by Kay Ewbank   
Tuesday, 23 February 2016

Details of the next version of Apache Spark, a free community edition of the Databricks cloud-based big data platform and Dashboards, a reporting front end, have all been announced.

The announcements were made at the recent Spark Summit East by Matei Zaharia, Databricks CTO and Spark creator.

Apache Spark is an open source data processing engine, and Zaharia said in the conference keynote that the next major version, Spark 2.0, is coming in April or May this year. The new version will see speed improvements of five to ten times; a new structured streaming real-time engine on SQL/dataframes; and unification of datasets and dataframes.

The new streaming engine is built on the Spark SQL engine, and also supports interactive and batch queries that aggregate data in a stream, then serve using JDBC. 



Alongside the announcement of Spark 2, Databricks has also announced the beta release of Databricks Community Edition, a free version of the cloud-based big data platform. Databricks was the founder of Apache Spark, and the largest contributor to Spark development. This service will provide users with access to a micro-cluster as well as a cluster manager and notebook environment. The idea is that developers can use the environment to learn Spark without the need to set up and run their own cluster environment.The initial beta rollout is invitation only. Wider access is planned over the next few months, with general availability planned for late Q2 2016.



The third announcement is Databricks Dashboards, a visual reporting application for Apache Spark clusters that can be used to provide reports and interative queries. Dashboards is actually an alternative view of a Databrick notebook aimed at end users who want to see different views of their data. Once a dashboard has been built, it can be shared with other users via its URL. The dashboard can be created with drop-down menus that can be used to choose or input parameters to alter the data being retrieved. The users needn't have any Spark knowledge, not any access to critical code. The dashboards can be updated automatically as underlying data changes.  sparklogo

More Information

Spark Site

Spark Summit East


Related Articles

Apache Releases Spark 1.6

Spark 1.4 Released

MOOC On Apache Spark 

Learning Spark (book review) 


To be informed about new articles on I Programmer, sign up for our weekly newsletter,subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin




Google Updates Android Logo

Google has announced  a 3D version of its Android logo, with the twin aim of better representing the Android community and being 'a lot of fun, too'.

Python Extends Its Lead In IEEE Spectrum Rankings

This is the 10th year for the annual Top Programming Languages exercise. Python coming top isn't news. It has now done so eight years in a row. What is important is that it is even further in the lead [ ... ]

More News


Summer SALE Kindle 9.99 Paperback $10 off!!




or email your comment to:

Last Updated ( Tuesday, 23 February 2016 )