Static Aerospike agent

Target Aerospike

Prerequisites

To have Gluesync working on your Aerospike instance you will need to have:

  • Valid user credentials with permission to write to the target Namespace.

Please note that Aerospike has 15 char limit per each field name, meaning that every field you have which is longer than 15 chars requires to be renamed via Gluesync field renaming feature available from the entity UI create/edit step. If a field longer than 15 chars limit is found by Gluesync an error will be throw before allowing the user to proceed further.

Setup via Web UI

  • Hostname / IP Address: DNS record of your Aerospike Server or IP Address of one of the nodes (automatic discovery of all other nodes is then applied).

  • Port: Optional, defaults to 3000.

  • Database name: Name of your target Namespace.

  • Username: Username with read & write access role to target Namespace.

  • Password: Password belonging to the given username.

  • Disable auth: (optional) Defaults to false, by setting that parameter to true you will disable the authentication mechanism (as per the Aerospike docs) when you’re connecting to a development instance of Aerospike.

  • additionalHosts: (optional, defaults to null) Tells Aerospike source connector to make use of a cluster map of additional hosts available at the bootstrap time by feeding those to the Aerospike client SDK. If you don’t provide additional hosts at the bootstrap time they will be then retrieved anyway when the connection to Hostname / IP Address is established. This comes to help in case the Hostname / IP Address is down at the bootstrap time;

  • Enable Tls: (optional, defaults to false) Enable or disable the usage of TLS encryption;

  • tlsName: (optional, defaults to null) TLS certificate name used for secure connections. Set null or remove the key entry if TLS is disabled;

  • trustStorePath: (optional) The path to the trust store certificate;

  • trustStorePassword: (optional) If a certificate path is specified, then this value is passed as a trust store password;

  • keyStorePath: (optional) The path to the keystore certificate;

  • keyStorePassword: (optional) If a certificate path is specified, then this value is passed as a keystore password.

Custom host credentials

This agent has no specific configuration properties.

Specific configuration

  • writePageSize: (optional, defaults to 250) Write page size;

  • writePageMultiplier: (optional, defaults to 0) Write page multiplier (multiplier / message kb);

  • numberOfLoops: (optional, defaults to 2) Number of (netty/NIO) loops to use;

  • useBatchWrites: (optional, defaults to false) Use batch writes (experimental);

  • enableClientSideCompression: (optional, defaults to false) Tells the Couchbase SDK to enable client-side compression before sending data to server.

Setup via Rest APIs

Here following an example of calling the CoreHub’s Rest API via curl to setup the connection for this Agent.

Connect the agent

curl --location --request PUT 'http://core-hub-ip-address:1717/pipelines/{pipelineId}/agents/{agentId}/config/credentials' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
        "hostCredentials": {
        "connectionName": "myAgentNickName",
        "host": "host-address",
        "port": 3000,
        "username": "",
        "password": "",
        "disableAuth": false,
        "enableTls": true,
        "additionalHosts": ["192.168.10.X", "192.168.10.Y", "192.168.10.Z"],
        "certificatePath": "/myPath/cert.pem",
        "trustStorePath": "/trusStorePath/xyz"
}'

Setup specific configuration

The following example shows how to apply the agent-specific configurations via Rest API.

curl --location --request PUT 'http://core-hub-ip-address:1717/pipelines/{pipelineId}/agents/{agentId}/config/specific' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
      "configuration": {
            "writePageSize": 250,
            "writePageMultiplier": 0,
            "numberOfLoops": 2,
            "useBatchWrites": false,
            "enableClientSideCompression": true
      }
}'

Aerospike bin names and entity names

Aerospike identifies the bin object with a key name that uniquely identifies an object stored in the database.

Since each bin key has a maximum length of 14 characters Gluesync will throw an exception in case you accidentally have placed a longer entity name inside the sourceEntities list.

Generating valid TLS certificates for your Aerospike database

To create certificates for connection to the Aerospike Server you need:

  • CA Certificate: example.ca.crt

  • Client Certificate: example.client.crt

  • Client Private Key: example.client.key

First, install the CA certificate into a new Java TrustStore with the following command:

keytool -importcert -storetype jks -alias example.ca -keystore example.ca.jks -file example.ca.crt -storepass changeit

Then use the following command to create a single chain certificate file named example.client.chain.crt:

cat example.ca.crt example.client.crt example.client.key > example.client.chain.crt

Finally, create a keystore in PKCS #12 format to pass to Gluesync with the following command:

openssl pkcs12 -export -in example.client.chain.crt -out example.client.chain.p12 -password pass:"changeit" -name example.client -noiter -nomaciter

For further details, please refer to the official Aerospike documentation at this link: aerospike.com/docs/server/operations/configure/network/tls/mtls_java#java-application