A newer version of this documentation is available.
View Latest (v2.0)

Gluesync SQL to NoSQL

Targeting Apache Kafka

Core principles

Gluesync offers support for streaming data through Apache Kafka to let you focus on the development of your modern application stack and leave all the complexity of handling data sourcing from relational and NoSQL databases to it.

Data could be modeled accordingly to your needs using Gluesync’s data modeling features and then sent immediately to a designated Kafka topic where it has written permissions to stream changes.

Supported auth protocols and encryption mechanisms

Gluesync supports the SASL SSL authentication mechanism against Kafka with both SCRAM-256 and SCRAM-512 algorithms, as well as non-authenticated Kafka clusters.

Topics management

In order to push changes feed to a Kafka topic this has to be created first and the Gluesync user authorized to write and publish messages to it.

We also suggest you set the automatic creation of topics to let Gluesync manage the creation of every needed topic.

Architectural overview

Here in the following diagram represents an architectural overview of the environment you are going to have after having deployed Gluesync.

a diagram illustrating the architectural overview of Gluesync for Apache Kafka

Messages format & keys

Gluesync encapsulates the message in JSON format into the defined Kafka topics, the message structure looks like the following represented by the screen below.

Gluesync Kafka message example

The message format is similar to other industry products like, for example, Debezium. This has been selected as a message format to guarantee a seamless experience for users switching over from other platforms in favor of Gluesync.

For delete operations the document body will be sent empty while the key field will still be valorized with the document key belonging to the delete operation. Like is shown in the image here below.
Gluesync Kafka message example

Q&A

I am using a different authentication mechanism, is it supported? We’re open to supporting a wider spectrum of different authentication mechanisms and features. Please drop us a line writing at this email address: Contact MOLO17.

Having further questions? For more information regarding Gluesync please reach us via email by pressing here: Contact MOLO17.