Apache Arrow 15 has been released. Improvements include new format strings in the C data interface, and Flight SQL now being stable in Arrow Flight.

Apache Arrow is a development platform for in-memory analytics. It has technologies that enable big data systems to process and move data quickly. Arrow is language independent, can be used for flat and hierarchical data, and the data store is organized for efficient analytic operations. It also provides computational libraries. Languages currently supported are C, C++, C#, Go, Java, JavaScript, Julia, MATLAB, Python, R, Ruby, and Rust.

The improvements to the new release start with the addition of new format strings for the C Data Interface. The additional format strings have been added for ListView, LargeListView, BinaryView and StringView array types.

Flight SQL is now considered stable for Arrow Flight RPC. Flight RPC is an RPC framework for high-performance data services based on Arrow data, and is built on top of gRPC and the IPC format. RPC (Remote Procedure Call) is a request–response protocol. One of the biggest features that sets apart Flight from other data transport frameworks is parallel transfers, allowing data to be streamed to or from a cluster of servers simultaneously. This enables developers to more easily create scalable data services that can serve a growing client base. The Flight SQL specification was also clarified in this release to specify how the result set schema of a prepared statement is affected by bound parameters. Another improvement means the JDBC Arrow Flight SQL driver now supports mTLS authentication and bind parameters. It can also reuse credentials across metadata and data connections.

C++ in Arrow has faster scalar encoding and decoding, and its Parquet support adds the ability for row group filtering for nested paths among other improvements.

The Java improvements include better handling of 256-bit decimals in the JDBC adapter, and the addition of a bill-of-materials package.

The developers say they expect a breaking change for Java in the next release, Apache 16. This will be due to the addition of support for Java 9 modules, which will require changing the JVM flags used to launch your application. Arrow 15.0.0 is not affected.

Other minor improvements have been added for JavaScript, Python, R, Go, C# and Ruby.

Apache Arrow 15 is available now.  

More Information

Apache Arrow Website

Related Articles

Apache Arrow 5 Improves Asynchronous Scanner

Apache Arrow 4 Adds New C++ Compute Functions

Apache Arrow Improves C++ Support

Apache Arrow 2 Improves C++ and Rust Support

Apache Arrow Reaches 1.0

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info