Apache Drill Adds YARN Support
Written by Kay Ewbank   
Tuesday, 17 April 2018

The new release of Apache Drill has added the ability to run Drill as a YARN application, along with support for HTTP Kerberos authentication using SPNEGO, and SQL syntax highlighting of queries.

Apache Drill is a distributed SQL query engine that works with most non-relational datastores, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A single query can join data from multiple datastores. Drill can be used by analysts, data scientists and developers to explore and analyze non-relational data without having to make the data more regimented for analysis. Drill processes the data in-situ without requiring users to define schemas or transform data.



Drill is the only columnar execution engine that supports complex and schema-free data, and the only execution engine that performs data-driven query compilation and schema discovery during query execution. Drill was designed to provide good query performance with the flexibility offered by the JSON document model.

The main change to the new version is the ability to run Drill as a YARN application if you want Drill to work alongside other applications, such as Hadoop and Spark, in a YARN-managed cluster. When used like this, YARN handles the work of resource assignment to applications in the cluster so you don't have to deal with your own resource allocation. YARN automatically deploys the Drill software onto each Drill node and manages the Drill cluster. When used like this, Drill can become a long-running application, and you can monitor the Drill-on-YARN cluster using the Application Master web UI.

The next improvement is the addition of SPNEGO support.  SPENGO is Simple and Protected GSSAPI Negotiation Mechanism, and it is an alternative to Kerebos. You can use SPNEGO to extend Kerberos authentication to Web applications through HTTP.

SQL Syntax support has also been improved. The query syntax now appears highlighted in the Drill Web Console, and auto-complete is supported in all SQL editors, including the Edit Query tab within an existing profile to rerun the query. If you're working in an appropriate browser such as Chrome, you can type Ctrl+Space for a drop-down list and then use arrow keys for navigating through options. There's also an auto-complete feature that specifies Drill keywords and functions, and the ability to write SQL from templates using snippets.




More Information

Apache Drill

Drill On YARN

Related Articles

MapR Releases Docker Container For Local Development

Apache Drill Reaches 0.6

Perform Data Queries Faster With Drill

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.


.NET MAUI Adds New Layouts

 .NET 6 Preview 7 has improved .NET MAUI with new layouts, which the team says is a significant change for performance and reliability. .NET MAUI is the .NET Multi-platform App UI, a framework fo [ ... ]

TPM Security Easy To Defeat

The headline says it all, but not in sufficient depth for you to understand exactly what the situation is. There is a hardware hack that means that you can compromise a machine that has been set up in [ ... ]

More News





or email your comment to: comments@i-programmer.info