MySQL CDC with Gluesync: Setup & Configuration

Source data from MySQL

Prerequisites

To have Gluesync working on your MySQL instance you will need to have:

Valid user credentials with permission to read from source tables.

Setup via Web UI

Hostname / IP Address: DNS record or IP Address of your server;
Port: Optional, defaults to 3306;
Database name: Name of your database;
Username: Username with read/write access to the tables;
Password: Password belonging to the given username;
Max connections count: Maximum number of connections the pool can instantiate.

Specific configuration

This agent has no specific configuration properties.

Setup via Rest APIs

Here following an example of calling the CoreHub’s Rest API via curl to setup the connection for this Agent.

Connect the agent

curl --location --request PUT 'http://core-hub-ip-address/pipelines/{pipelineId}/agents/{agentId}/config/credentials' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
        "hostCredentials": {
        "connectionName": "myAgentNickName",
        "host": "host-address",
        "port": 3306,
        "databaseName": "db_name",
        "username": "",
        "password": "",
        "maxConnectionsCount": 100,
        "enableTls": true,
        "certificatePath": "/myPath/cert.pem"
      }
}'

Setting up for CDC

Follow the below steps to set up Bin logs redaction on your MySQL instance to let Gluesync capture its transaction logs.

MySQL Bin log

Edit the MySQL config file, usually located at /etc/my.cnf or /etc/mysql/my.cnf then under the [mysqld] section place or edit the following parameter:

[mysqld]
server_id=300
log_bin=mysql-bin
binlog_format= row
gtid_mode=OFF

Once done create a new user that will be then given to Gluesync to connect and grab the transaction logs. In the example below we’re going to create a user called gluesync:

CREATE USER 'gluesync'@'host' IDENTIFIED BY 'password';
GRANT REPLICATION SLAVE ON *.* TO 'gluesync'@'%' WITH GRANT OPTION;

Then, don’t forget to restart your MySQL instance by issuing (command may vary depending on your OS):

sudo service mysql restart

Working with Before & After images

Before & after images are a feature that allows Gluesync to track the changes that have occurred in your database and comparing them with their previous values. This enables Gluesync to apply only the changes that have occurred, saving bandwidth and processing time.

This feature is automatically enabled and available for use in this agent.