Apache Flink has released Stateful Functions 2, the first version of its event-driven database that includes the feature of stateful functions, small piece of codes that are invoked through a message.  

Stateful Functions (StateFun) combine support for state and composition with FaaS implementations like AWS Lambda and resource orchestration frameworks like Kubernetes.

Apache Flink is an open source platform for distributed stream and batch data processing, with a streaming dataflow engine for data distribution and distributed computations over data streams. Flink has a number of APIs, including a DataSet API for static data embedded in Java, Scala, and Python; a DataStream API for unbounded streams embedded in Java and Scala; the Table API with a SQL-like expression language embedded in Java and Scala; and the streaming SQL API that enables SQL queries to be executed on streaming and batch tables, with a syntax is based on Apache Calcite.

The new release has been made possible by Stateful Functions joining Apache Flink at the beginning of the year. Stateful Functions was created as a library running on Flink for creating general purpose event-driven applications.

Developers could use the library to implement functions to receive and send messages, and maintain state in persistent variables. Flink provided the runtime with efficient exactly-once state and messaging. The Flink team describes Stateful Functions 1.0 as a FaaS-inspired mix between stream processing and actor programming — on steroids.

The new release, Stateful Functions 2.0, physically decouples the functions from Flink and the JVM and instead invokes them through simple services. That makes it possible to execute functions on a FaaS platform, a Kubernetes deployment or behind a micro service. Flink invokes the functions through a service endpoint via HTTP or gRPC based on incoming events, and supplies state access. Because state access is part of the function invocation, the functions behave like stateless applications. 

The functions can be implemented in any programming language that can handle HTTP requests or bring up a gRPC server. The StateFun project includes a very slim SDK for Python, taking requests and dispatching them to annotated functions. The Flink team plans to provide similar SDKs for other languages, such as Go, JavaScript or Rust. Users do not need to write any Flink code; data ingresses/egresses and function endpoints can be defined in a compact YAML spec.

StateFun is available on GitHub.

More Information

StateFun On GitHub

Flink website

Related Articles

Apache Flink 1.9 Adds New Query Engine

Apache Flink 1.5.0 Adds Support For Broadcast State

Flink Gets Event-time Streaming

Flink Reaches Top Level Status

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info