Amazon has announced PartiQL, a SQL compatible query language designed to let you query data no matter what database it's stored in. A reference implementation of the language has been open sourced.

Amazon says that so long as a query engine supports PartiQL, you can process structured data from relational databases (both transactional and analytical), semi-structured and nested data in open data formats (such as an Amazon S3 data lake), and schema-less data in NoSQL or document databases that allow different attributes for different rows.

The open source implementation supports users parsing PartiQL queries into abstract syntax trees that their applications can analyze or process and supports interpreting PartiQL queries directly.

Amazon is already being used by a number of Amazon systems including Amazon S3 Select, Amazon Glacier Select, Amazon Redshift Spectrum, and Amazon Quantum Ledger Database. Couchbase also plans to add support for PartiQL to Couchbase Server.

Amazon has lots of semi-structured data, most often in the Ion format, and the company needed a way to use this non-relational data in AWS relational database services like Redshift. To do this, they developed PartiQL. Because this provides a consistent way to query data across a variety of formats, it means you can move your data across data sources, without having to change your queries. It's backwards-compatible with SQL, and provides extensions for multi-valued, nested, and schema-less data.

PartiQL doesn't need a predefined schema for a dataset, and can be used by both database engines that assume the presence of a schema and  schema-less engines.

It offers format independence, with queries written identically across underlying data in JSON, Parquet, ORC, CSV, Ion, or other formats. It also offers data store independence.

PartiQL is available from the PartiQL website or on GitHub.

More Information

PartiQL Website

PartiQL On GitHub

Related Articles

Amazon Open Sources Python Library for AWS Glue

Athena Query Alterer Open Sourced

Databricks Delta Adds Faster Parquet Import

New AWS Services

AWS Improvements For Developers

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info