|DataStax Astra DB gets Change Data Capture|
|Written by Nikos Vaggalis|
|Monday, 25 April 2022|
DataStax adds CDC to its Astra DB database-as-a-service platform to deliver database changes in real time via event streams, making real-time data available for use across data lakes, data warehouses and other applications.
CDC is a way to capture changes made in the database and forwarding them in real-time to external applications (such as Kafka) through connectors such as the ones offered by Debezium, the open source distributed platform that turns your existing databases into event streams.
There are many ways to implemented CDC like row versioning, pubsub, triggers and log monitoring, with the log-based one being the most popular and automated. As for a painstakingly manual trigger-based approach make sure to check "Connecting To The Outside World with Perl and Database Events".
The use cases of CDC include real-time analytics, database replication or customized solutions like Connecting To The Outside World with Perl and Database Events, which uses Ingres as the underlying DBMS and, through trigger-based CDC, it transforms SQL data to hl7 which then posts to a web service:
At different points in time, hospital clerks collect the details of the patient's visit and register them to the system through a GUI application/data entry form.This data will then end up as rows in several tables in the database.The Ingres RDBMS will handle the database part, while Perl will handle the application part being in charge of gluing the database to the outside world by extracting and transforming this data to a HL7 message and sending it over to a Web service endpoint.
Astra DB's CDC too is powered by the Astra Streaming technology, which is built on the Apache Pulsar distributed pub/sub-messaging system.
Using a simple configuration based approach, you can enable CDC on one or more Astra DB tables and publish the changes to an event topic in Astra Streaming. From there, your real-time applications can subscribe to change events using client libraries in Java, Golang, Python, or Node.js. Additional endpoints support direct subscription via websocket interface or using a standard JMS client. And with that you cover a wide range of use case scenarios:
Security monitoring: Gain visibility into anomalous behavior that may indicate a security breach with CDC’s consumable stream of event data.
To start with CDC quickly you should go through the following steps:
That's the boilerplate procedure.From there on you can use the Apache Pulsar CLI tools to produce and consume messages.
or email your comment to: email@example.com
|Last Updated ( Monday, 25 April 2022 )|