Apache Kyuubi 1.8 has been released with improvements including a new experimental Web UI and a new chat engine with OpenAI ChatGPT backend.

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses. It provides a pure SQL gateway through a Thrift JDBC/ODBC interface for end-users to manipulate large-scale data with pre-programmed and extensible Spark SQL engines.

Kyuubi's key features start with its provision of end-to-end multi-tenancy for resource acquiring and data/metadata access with a unified authentication/authorization layer. It supports load balancing via ZooKeeper, which provides an enterprise-grade high availability, as well as an unlimited high client concurrency, and can support multiple disparate workloads with one single platform, one copy of data, and one SQL interface.

The Kyuubi team is aiming to make Kyuubi an "out-of-the-box" tool for data warehouses and lakehouses to minimize the barriers and costs for end-users to use Spark, Flink, and other computing engines at the client side. At the server-side, Kyuubi server and engine's multi-tenant architecture provides the administrators a way to achieve computing resource isolation, data security, high availability, and high client concurrency.

Kyuubi comes with several APIs that provide connectivity and interoperation between supported clients and a Kyuubi server., including an API offering a

HiveServer2-compatible interface that allows end users to use a thrift client, a Java Database Connectivity(JDBC) interface over thrift, or an Open Database Connectivity (ODBC) interface over a JDBC-to-ODBC bridge to communicate with Kyuubi.

Kyuubi also comes with RESTful APIs and a MySQL protocol that enables MySQL connectors such as Connector/J to communicate with Kyuubi.

Alongside the new experimental Web UI and chat engine, the latest version comes with major improvements to the Spark Batch API to optimize it for large-scale job submission. The Spark AuthZ plugin has also been improved to add better support for Iceberg and DSv2 operators.

The Flink engine now includes YARN Application Mode support, and there's Atlas hook support in the Spark Lineage plugin. Finally, the Kyuubi Helm Chart and Spark on K8s have been improved.

Kyuubi 1.8 is available now.

More Information

Kyuubi Website

Kyuubi On GitHub

Related Articles

Apache Iceberg Improves Spark Support

Apache Spark Now Understands English

Running Spark on Kubernetes: A Quick Guide

Apache Flink ML 2.0 Released

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info