Aerospike CDC with Gluesync: ESP & XDR Setup Guide

Source data from Aerospike

Prerequisites

To have Gluesync working on your Aerospike instance as a source connector you will need to have:

valid user credentials with permission to read, and write to the source database;
ESP connector installed in your system (bare metal, VM, Docker…);
Gluesync Aerospike’s agent port 8017 must be exposed for external connectivity with the Aerospike ESP connector.

The snapshot feature is not currently supported with this initial release of the Aerospike CDC agent for Gluesync.

Setup via Web UI

Hostname / IP Address: DNS record of your Aerospike Server or IP Address of one of the nodes (automatic discovery of all other nodes is then applied).
Port: Optional, defaults to 3000.
Database name: Name of your target Namespace.
Username: Username with read & write access role to target Namespace.
Password: Password belonging to the given username.
Disable auth: (optional) Defaults to false, by setting that parameter to true you will disable the authentication mechanism (as per the Aerospike docs) when you’re connecting to a development instance of Aerospike.
additionalHosts: (optional, defaults to null) Tells Aerospike source connector to make use of a cluster map of additional hosts available at the bootstrap time by feeding those to the Aerospike client SDK. If you don’t provide additional hosts at the bootstrap time they will be then retrieved anyway when the connection to Hostname / IP Address is established. This comes to help in case the Hostname / IP Address is down at the bootstrap time;
Enable Tls: (optional, defaults to false) Enable or disable the usage of TLS encryption;
tlsName: (optional, defaults to null) TLS certificate name used for secure connections. Set null or remove the key entry if TLS is disabled;
trustStorePath: (optional) The path to the trust store certificate;
trustStorePassword: (optional) If a certificate path is specified, then this value is passed as a trust store password;
keyStorePath: (optional) The path to the keystore certificate;
keyStorePassword: (optional) If a certificate path is specified, then this value is passed as a keystore password.

Custom host credentials

This agent has no specific configuration properties.

Specific configuration

This agent has no specific configuration properties.

Setup via Rest APIs

Here following an example of calling the CoreHub’s Rest API via curl to setup the connection for this Agent.

Connect the agent

curl --location --request PUT 'http://core-hub-ip-address:1717/pipelines/{pipelineId}/agents/{agentId}/config/credentials' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
        "hostCredentials": {
        "connectionName": "myAgentNickName",
        "host": "host-address",
        "port": 3000,
        "username": "",
        "password": "",
        "disableAuth": false,
        "enableTls": true,
        "additionalHosts": ["192.168.10.X", "192.168.10.Y", "192.168.10.Z"],
        "certificatePath": "/myPath/cert.pem",
        "trustStorePath": "/trusStorePath/xyz"
}'

Setup of ESP & XDR

Setting up gluesyncDC under Aerospike XDR

First of all, we need to set up an XDR "datacenter" endpoint for Gluesync under your aerospike.conf file. This XDR endpoint will be responsible for streaming changes made at the selected namespace level right into the ESP connector.

To set this XDR endpoint you can refer to the following example:

{
  ...

  xdr {

    # For naming convention purposes we keep gluesyncDC
    dc gluesyncDC {
      # Explicitly tell the XDR that this endpoint is a streaming connector and not an Aerospike node
      connector true

      # 192.168.1.201 is your IP / DNS address of the Aerospike ESP server
      # Port 8017 is the default we suggest using, you can change it as you wish
      node-address-port 192.168.1.201 8017

      # Substitute test with your namespace, add as many of these objects as necessary
      namespace test {

          # We explicitly avoid replicating Gluesync's set to limit overhead
          ignore-set GLUESYNC
      }
    }
  }

  ...
}

Where:

gluesyncDC: is the name of the XDR endpoint we suggest you take as per our naming convention;
connector: true, will explicitly tell the XDR that this endpoint is a streaming connector and not an Aerospike node;
node-address-port: as per our example, 192.168.1.201 is the IP Address assigned to the ESP server, change it accordingly to your deployment configuration. DNS is fine either. Port 8017 is the default port we are explicitly declaring to let XDR establish a connection with the ESP server; the port can change as per your requirement but it has to match on the ESP’s end as well;
namespace: is your namespace name;
ignore-set: please keep the ignore-set GLUESYNC configuration as this avoids the replication of Gluesync’s managed set across XDR to reduce overhead.

Please consider reading Aerospike XDR documentation for any additional configuration by following this link.

After you’ve completed this configuration step you will need to apply the configuration, to do so you can either restart your Aerospike server or you can dynamically apply the same configuration without having to restart the server by following this Aerospike documentation link.

Repeat this step on each of your nodes belonging to the Aerospike Cluster since every node owns its partition, you will otherwise get just a fraction of the data.

Setting up ESP connector yaml

Once you have XDR set your Aerospike cluster will try to establish a connection with the given ESP endpoint. To let it be able to achieve we require deploying a working ESP connector by issuing it an aerospike-esp-outbound.yml file that will look like the following provided in this example below.

service:
  port:
    - 8017
  manage:
    port: 8902
    # address & port that can be used to check ESP metrics endpoint (optionally)
    # http://localhost:8902/manage/rest/v1/metrics
  protocol: TCP

# tells ESP to perform console logging (optional)
logging:
  enable-console-logging: true

# converts output XDR message format in JSON (see https://aerospike.com/docs/connect/esp/configuring/format)
format:
  mode: json

# Destinations
destinations:
  gluesyncDC:
    urls:
    # endpoint that the ESP uses to connect to Gluesync Aerospike source connector, port 1717 is its web server's default port
      - http://YOUR_GLUESYNC_ADDRESS_HERE:8017/esp
    protocol: HTTP_1_1
    max-requests-queued-per-endpoint: 10240
    connection-ttl: 15000
    max-connections-per-endpoint: 100
    call-timeout: 10000
    connect-timeout: 2000
    health-check:
      call-timeout: 10000
    headers:
      authority-header-omit-default-port: true
      send-digest-header: true
    http-success-status-codes:
      - 200
      - 202-206
      - 208
      - 400
      - 405

record-ordering:
  enable: true
  lut-cache-ttl-seconds: 30

port-based-config:
# port 8017 is our default port, if you've declared another port at the aerospike.conf XDR level, please change it accordingly (this change has to be propagated at agent's port level too)
  8017:
    routing:
      mode: static-multi-destination
      # This has to match the same declared DC endpoint that we have put in the aerospike.conf file
      destinations: gluesyncDC

To learn more about the Aerospike ESP connector please refer to the following link.

Generating valid TLS certificates for your Aerospike database

To create certificates for connection to the Aerospike Server you need:

CA Certificate: example.ca.crt
Client Certificate: example.client.crt
Client Private Key: example.client.key

First, install the CA certificate into a new Java TrustStore with the following command:

keytool -importcert -storetype jks -alias example.ca -keystore example.ca.jks -file example.ca.crt -storepass changeit

Then use the following command to create a single chain certificate file named example.client.chain.crt:

cat example.ca.crt example.client.crt example.client.key > example.client.chain.crt

Finally, create a Keystore in PKCS #12 format to pass to Gluesync with the following command:

openssl pkcs12 -export -in example.client.chain.crt -out example.client.chain.p12 -password pass:"changeit" -name example.client -noiter -nomaciter

In the Aerospike-specific configuration section it will now be possible to use certificates as follows:

  "aerospike": {
    ...
    "trustStore": "/opt/app/config/example.ca.jks",
    "keyStorePath": "/opt/app/config/example.client.chain.p12",
    "keyStorePassword": "changeit",
    "enableTls": true,
    "tlsName": "TLS_NAME",
    ...
  }

Gluesync needs to create its set within the selected namespace to store some information as document checksums. The set name is GLUESYNC. Don’t delete or edit this collection or its content since it is automatically being managed by Gluesync.

For further details, please refer to the official Aerospike documentation at this link: aerospike.com/docs/server/operations/configure/network/tls/mtls_java#java-application