Static Couchbase agent

Target Couchbase

Prerequisites

To have Gluesync working on your Couchbase Server instance you will need to have:

  • valid user credentials with permission to read, and write to the target bucket;

  • Following ports opened on your Couchbase Server cluster:

  • 8091-8096 (REST API);

  • 18091 (TLS REST API), in case you use TLS;

  • 11210:11210 (Bucket API), in case you use TLS;

  • 21100:21100 (Management API), in case you use TLS;

If you’re considering running Gluesync on a Couchbase Capella DBaaS, please consider also looking at the specific section Deploying in a Capella DBaaS cluster here below on that page.

Setup via Web UI

  • Hostname / IP Address: DNS SVR record of your Couchbase Server cluster or IP Address of one of the nodes (automatic discovery of all other nodes is then applied). For Capella users, put the given hostname like cb.XYZ.cloud.couchbase.com;

  • Port: Optional, defaults to 8091, use 18091 for Couchbase Capella DBaaS and secured connections;

  • Database name: Name of your target bucket;

  • Username: Username with read & write access role to target bucket;

  • Password: Password belonging to the given username;

  • Enable Tls: (optional, defaults to false) Enable or disable the usage of TLS encryption;

  • Tls certificates: (optional) File browser to let you upload your certificates;

Custom host credentials

This agent has no specific configuration properties.

Specific configuration

The following example shows how to apply the agent-specific configurations via Rest API.

  • Write page size: (optional, defaults to 250) Write page size;

  • Write page multiplier: (optional, defaults to 0) Write page multiplier (multiplier / message kb);

  • Use collections: (optional, defaults to true) Tells Gluesync to create and use a Scopes & Collections from Couchbase Server per each of your source tables instead of using the default collection. The default scope is _default. This only applies to Couchbase version 7.0 and above;

  • Timeout seconds: (optional, defaults to 60 seconds) Number of seconds to set as timeout for operations involving communication with the server;

  • Enable client-side compression: (optional, defaults to false) Tells the Couchbase SDK to enable client-side compression before sending data to server.

Setup via Rest APIs

Here following an example of calling the CoreHub’s Rest API via curl to setup the connection for this Agent.

Connect the agent

curl --location --request PUT 'http://core-hub-ip-address:1717/pipelines/{pipelineId}/agents/{agentId}/config/credentials' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
        "hostCredentials": {
        "connectionName": "myAgentNickName",
        "host": "host-address",
        "port": 8091,
        "databaseName": "myBucket",
        "username": "",
        "password": "",
        "enableTls": true,
        "certificatePath": "/myPath/cert.pem",
}'

Setup specific configuration

The following example shows how to apply the agent-specific configurations via Rest API.

curl --location --request PUT 'http://core-hub-ip-address:1717/pipelines/{pipelineId}/agents/{agentId}/config/specific' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
  "configuration": {
      "writePageSize": 250,
      "writePageMultiplier": 0,
      "useCollections": true,
      "enableClientSideCompression": true
    }
}'

How operations are performed

Every operation that this agent does is done by leveraging KV (key-value) operations to write to Couchbase except delete operations for DataModeling entities.

Here is a summary table.

FullTable entity: - insert/update/delete → KV Mapping entity: - insert/update/delete → KV Query entity: - insert/update/delete → KV DataModeling entity: - insert/update→ KV - delete → N1QL

Deploying in a Capella DBaaS cluster

Gluesync has been certified by the vendor to work and perform live replications together with Couchbase Capella clusters. To establish a connection to a Capella Cluster you just need to have:

  • Valid credentials for an admin user able to connect to the destination bucket;

  • Valid .pem certificate file;

  • The hostname of your Capella instance, for example: cb.instancereferencename.cloud.couchbase.com

Looking for using Scopes & Collections?

From Couchbase Server version 7.0 and above it is possible to start using Scopes & Collection feature. For more details about that feature, you can have a look at this link from the official Couchbase documentation page.

Gluesync supports both Scopes & Collections in that way, when useCollections is set to true:

  • automatically creates a new Collection per each one of your source tables if none with the same name is found;

  • make use of existing collections where the Collection name matches your source table;

  • if your source schema is not set it defaults to using _default scope.

when useCollections is set to false:

  • document automatically defaults to default collection and _default scope when running under Couchbase 7.0 or above;

  • each document automatically belongs to the target designed bucket when running under Couchbase 6.6.X or older.

Troubleshooting

Here is the list of common errors.

  • ERROR TIMEOUT: BUCKET_OPEN_IN_PROGRESS: You might either have specified the wrong server port for your Couchbase connection or this is blocked by the firewall.