A new low-level programming language, Terra, has been designed to work with Lua to generate code for high performance computing.

Lua is a lightweight embeddable scripting language that combines simple procedural syntax with data description constructs based on associative arrays and extensible semantics.

Terra is a simple, statically-typed, compiled language with manual memory management, and as such is similar to C, but has been designed to interoperate with Lua. Terra functions are first-class Lua values created using the terra keyword. When needed they are JIT-compiled to machine code.

The details about Terra have been published by Zach DeVito and a team at Stanford and Purdue University. The team says that using Lua as a linguistic host for a new low-level language, they have developed a system that means you can write high-level code in Lua and high-performance code in Terra.

The team suggests it can be used as a scripting language with high-performance extensions, as an embedded JIT-compiler for building languages, or as a standalone low-level language.

When used as a scripting language, the advantage of a low-level of abstraction is a predictable control of performance when you need it. The description says

“Terra programs use the same LLVM backend that Apple uses for its C compilers. This means that Terra code performs similarly to equivalent C code. For instance, our translations of the nbody and fannhakunen programs from the programming language shootout1 perform within 5% of the speed of their C equivalents when compiled with Clang, LLVM’s C frontend. Terra also includes built-in support for SIMD operations, and other low-level features like non-temporal writes and prefetches. You can use Lua to organize and configure your application, and then call into Terra code when you need controllable performance.”

In the case of usage as an embedded JIT-compiler for building languages, the team uses techniques from multi-stage programming to make it possible to meta-program Terra using Lua. Terra expressions, types, functions, and expressions are all first-class Lua values, so you can generate arbitrary programs at runtime.

This means you can compile domain-specific languages (DSLs) written in Lua into high-performance Terra code, and since Terra is built on the Lua ecosystem, it is easy to embed Terra-Lua programs in other software as a library, so you can add a JIT-compiler into your existing software. You can use it to add a JIT-compiled DSL to your application, or to auto-tune high-performance code dynamically.

Because Terra was designed so that it can run independently from Lua, it can be used as a stand-alone low-level language in cases where your final program doesn’t need Lua. You simply save Terra code into a .o file or executable. The team says you can think of it as a replacement for C++ template metaprogramming or C preprocessor X-Macros with better syntax.

The Stanford team has released a paper, Terra: A Multi-stage Language for High Performance Computing, where they describe how Terra and Lua could be used to generate high-performance code. In the paper, the team says that they’ve re-implemented existing multi-language systems entirely in Terra. Their Terra-based auto-tuner for BLAS routines performs within 20% of ATLAS, and the DSL for stencil computations runs 2.3x faster than hand-written C.

You can read more about how Terra can be used for generative programming and language compilation on the Terra site and can download it from GitHub.