The new release of Apache Drill has added the ability to run Drill as a YARN application, along with support for HTTP Kerberos authentication using SPNEGO, and SQL syntax highlighting of queries.

Apache Drill is a distributed SQL query engine that works with most non-relational datastores, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A single query can join data from multiple datastores. Drill can be used by analysts, data scientists and developers to explore and analyze non-relational data without having to make the data more regimented for analysis. Drill processes the data in-situ without requiring users to define schemas or transform data.

Drill is the only columnar execution engine that supports complex and schema-free data, and the only execution engine that performs data-driven query compilation and schema discovery during query execution. Drill was designed to provide good query performance with the flexibility offered by the JSON document model.

The main change to the new version is the ability to run Drill as a YARN application if you want Drill to work alongside other applications, such as Hadoop and Spark, in a YARN-managed cluster. When used like this, YARN handles the work of resource assignment to applications in the cluster so you don't have to deal with your own resource allocation. YARN automatically deploys the Drill software onto each Drill node and manages the Drill cluster. When used like this, Drill can become a long-running application, and you can monitor the Drill-on-YARN cluster using the Application Master web UI.

The next improvement is the addition of SPNEGO support.  SPENGO is Simple and Protected GSSAPI Negotiation Mechanism, and it is an alternative to Kerebos. You can use SPNEGO to extend Kerberos authentication to Web applications through HTTP.

SQL Syntax support has also been improved. The query syntax now appears highlighted in the Drill Web Console, and auto-complete is supported in all SQL editors, including the Edit Query tab within an existing profile to rerun the query. If you're working in an appropriate browser such as Chrome, you can type Ctrl+Space for a drop-down list and then use arrow keys for navigating through options. There's also an auto-complete feature that specifies Drill keywords and functions, and the ability to write SQL from templates using snippets.

More Information

Apache Drill

Drill On YARN

Related Articles

MapR Releases Docker Container For Local Development

Apache Drill Reaches 0.6

Perform Data Queries Faster With Drill

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info