Apache Kylin Adds RDBMS Support
Written by Kay Ewbank   
Wednesday, 06 September 2017

 Apache Kylin 2.1 has been released with new features including support for RDBMS and the ability to edit metadata model's JSON directly on the web.

Kylin is an open source analytics solution. It was originally developed at eBay before becoming an Apache project. It has a SQL interface, and can be used to carry out OLAP multidimensional analysis on Hadoop supporting extremely large datasets.


kylin

 

The Kylin OLAP Engine is made up of a metadata engine, a query engine, a job engine and a storage engine. It also includes a REST Server to service client requests. The query engine is based on Apache Calcite.

Apache Kylin v2.0.0 introduced a new cubing engine based on Apache Spark that can be selected to replace the original MapReduce engine. By default, Kylin uses MapReduce Cube Engine built on the Hadoop MapReduce framework to aggregate source data. Initial tests showed that the Spark engine could cut the build time to 50% in most cases. Release 2 also added support for the snowflake data model and for TPC-H queries. 

Apache Kylin lets you query massive data sets in three steps:

  1. Identify a Star Schema on Hadoop.
  2. Build a Cube from the identified tables.
  3. Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or RESTful API.

Kylin currently offers integration capability with BI Tools inlcuding Tableau, PowerBI and Excel. Integration with Microstrategy is coming soon.

The new release is a major release after 2.0, with more than 100 bug fixes and enhancements.

The addition of support for RDBMS data sources means that data in formats such as Oracle, SQL Server and MySQL can be used within the analyses. Tools such as Apache Sqoop can also be used to export the data from RDBMS to HDFS to make it easier for Kylin to get the data and then build that into cubes.

The second improvement adds the ability to edit metadata JSON. In the previous version, when a model's metadata was broken, the only way to fix the metadata was to use bin/metastore.sh. This new release lets the administrator edit the model's JSON directly in the web.

Other improvements include better handling of subqueries, and the routing of unsupported queries back to source.

 

kylin

More Information

Kylin Home Page 

Related Articles

Apache Spark 2.0 Released 

Apache Kafka Adds New Streams API

Apache Spark With Structured Streaming

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

 

Banner

Apache Releases Tomcat 11
07/11/2024

Apache has announced the release of Tomcat 11, as well as marking the 25th anniversary of the first commit to the Apache Tomcat source code repository since becoming an ASF project.


+ Full Story
Edera Releases Open Source Container Benchmark And Scanner
07/11/2024

Edera has released Am I Isolated, an open source container security benchmark that probes users runtime environments and tests for container isolation.


+ Full Story
More News

 

espbook

 

Comments




or email your comment to: comments@i-programmer.info
Last Updated ( Wednesday, 06 September 2017 )