Memphis.dev has introduced a message broker designed to support the development of real-time applications. Memphis Cloud is based on the Memphis open source project with added security and features for stateless stream processing at scale.

Memphis has three main components; the Memphis broker which acts as the data storage layer. That is the component that stores and controls the ingested messages and their entire lifecycle management. The second element is a metadata store that is responsible for storing the platform metadata only, such as general information, monitoring, GUI state, and pointers to dead-letter messages. The metadata store uses Postgres. The third component is a REST gateway that is responsible for exposing Memphis management and data ingestion through REST requests.

Memphis.dev is developed from this. It is described as an intelligent, frictionless message broker that accelerates development of real-time applications. It can be used to build modern queue-based applications that require large volumes of streamed and enriched data. The resilient, distributed architecture is cloud-native and can run on any Kubernetes, on any cloud without zookeeper, bookeeper, or JVM.

The additions to Memphis Cloud include augmentation of Kafka clusters, built-in schema management, enforcement, and transformation; multi-tenancy for traffic isolation; use-based billing and true multi-cloud capabilities. The augmentation of Kafka clusters comes from the fact that while Kafka users can set up inter-cluster data flows with Kafka's MirrorMaker, Memphis cloud users can create more Memphis clusters and form a supercluster that replicates data in an async manner between the clusters. Memphis Cloud can also run alongside Kafka, ensuring messages are delivered in order and without any loss.

Memphis Cloud also includes a module called Schemaverse that is responsible for built-in schema management, enforcement, and transformation. Schemaverse provides a schema store and schema management layer on top of Memphis broker. It provides runtime-level rendering of existing producers and consumers, and the ability to modify schemas without rebooting producers. Schemaverse has a number of SDKs: including Go, Python, and Node.js (Nest / Typescript). Supported protocols include Protobuf, Avro, JSON and GraphQL.

Memphis says Memphis Cloud is a true multi-cloud product with the ability to create primary instances on GCP, and a replica on AWS, and it is also multi-tenancy for users of SaaS platforms who want to isolate traffic between their customers. The serverless nature of the platform is pay-as-you-use, with the ability to scale to trillions of events without any concern to the infrastructure. The developers say all of this means that Memphis Cloud is poised to disrupt the dominance of the combination of Kafka and Flink. 

Yaniv Ben Hemo, co-founder and CEO, Memphis, said: 

"The world is asynchronous and built out of events. Message brokers are the engine behind their flow in the modern software architecture, and the modern message broker should be much more intelligent with much less friction. Memphis.dev takes five minutes on average for a user to get to production and start building queue-based applications and distributed streaming pipelines."

Memphis Cloud is available now.

More Information

Memphis Dev Website

Memphis On GitHub

Related Articles

Kafka KRaft Now In Production

Apache Flink ML 2.0 Released

Apache Flink 1.9 Adds New Query Engine

Kafka Adds KRaft-Based Authorizer  

Kafka 3.1 Adds OIDC Support

Kafka 3.0 Released With KRaft  

Kafka Replaces Zookeeper With Quorum

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info