Apache Kylin 2.1 has been released with new features including support for RDBMS and the ability to edit metadata model's JSON directly on the web.

Kylin is an open source analytics solution. It was originally developed at eBay before becoming an Apache project. It has a SQL interface, and can be used to carry out OLAP multidimensional analysis on Hadoop supporting extremely large datasets.

The Kylin OLAP Engine is made up of a metadata engine, a query engine, a job engine and a storage engine. It also includes a REST Server to service client requests. The query engine is based on Apache Calcite.

Apache Kylin v2.0.0 introduced a new cubing engine based on Apache Spark that can be selected to replace the original MapReduce engine. By default, Kylin uses MapReduce Cube Engine built on the Hadoop MapReduce framework to aggregate source data. Initial tests showed that the Spark engine could cut the build time to 50% in most cases. Release 2 also added support for the snowflake data model and for TPC-H queries. 

Apache Kylin lets you query massive data sets in three steps:

Kylin currently offers integration capability with BI Tools inlcuding Tableau, PowerBI and Excel. Integration with Microstrategy is coming soon.

The new release is a major release after 2.0, with more than 100 bug fixes and enhancements.

The addition of support for RDBMS data sources means that data in formats such as Oracle, SQL Server and MySQL can be used within the analyses. Tools such as Apache Sqoop can also be used to export the data from RDBMS to HDFS to make it easier for Kylin to get the data and then build that into cubes.

The second improvement adds the ability to edit metadata JSON. In the previous version, when a model's metadata was broken, the only way to fix the metadata was to use bin/metastore.sh. This new release lets the administrator edit the model's JSON directly in the web.

Other improvements include better handling of subqueries, and the routing of unsupported queries back to source.

More Information

Kylin Home Page 

Related Articles

Apache Spark 2.0 Released 

Apache Kafka Adds New Streams API

Apache Spark With Structured Streaming

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info