Search docs.molo17.com
    Configuration guide Connectors

    Set up a Couchbase target and shape the documents

    Couchbase stores JSON documents, not rows. Gluesync turns each source row into one document, writes it into a scope and a collection, and names it with a document key. This guide creates a bucket and a Couchbase user with only the roles Gluesync needs, copies a PostgreSQL table with a snapshot, follows inserts, updates and deletes with CDC, shows what the documents and their keys look like, changes the scope, collection and key, and lists the errors you are most likely to meet.

    In this guide

    You will create a bucket and a user with least roles, connect the Couchbase target, map a table to a collection, run a snapshot and follow changes with CDC, and read the documents with N1QL. You will then set a custom scope, collection and document key for a table with a composite primary key, switch off scopes and collections to see the type field, and read the speed settings of the start dialog. A table at the end lists the errors that were reproduced for this guide.

    Quick information

    Goal

    Replicate relational tables to Couchbase as JSON documents, with the scope, collection and document key you choose

    Audience

    Data engineers and Couchbase administrators

    Level

    Intermediate

    Time required

    About 40 minutes

    Area

    Couchbase → RBAC users; Control Plane → Pipelines → Couchbase target → Base settings and Advanced settings; entity → Edit settings → Target

    Applies to

    Gluesync 2.3

    Before you begin

    Make sure that:

    1. You have a Couchbase Server 7.0 or later (Enterprise or Community) or a Capella cluster, with the Data and Query services, and you can run couchbase-cli or the Web Console as an administrator. This guide was tested with Couchbase Server Enterprise 7.6. Couchbase is a target in this guide: you need another database as the source;

    2. Core Hub can reach the cluster on port 8091 (management) and 11210 (data), and on 8093 (query) if you turn off scopes and collections. Use 18091, 11207 and 18093 with TLS;

    3. You have a source that Core Hub can read with CDC. This guide uses PostgreSQL 16, set up as in Set up a PostgreSQL WAL source;

    4. You can sign in to the Control Plane with a role that can create pipelines;

    5. You have two source tables with a primary key. This guide uses public.orders with 200,000 rows (id bigint primary key, customer varchar(50), amount numeric(12,2), status varchar(20), created_at timestamptz, ship_date date, paid boolean, tags text[], note text) and public.order_lines with 300 rows and the composite primary key (order_id, line_no).

    The statements and the messages quoted in this guide were run and captured against Couchbase Server Enterprise 7.6 and the Control Plane that ships with Gluesync 2.3. The Couchbase target reference lists every option and the REST API setup.

    Guide steps

    Create the bucket and the user, and grant the roles

    CLI

    Gluesync does not need the admin role. It needs to read and write documents, and to create scopes and collections. Connect to a node as an administrator and run the commands below. Replace the passwords.

    1. Create the bucket. A bucket is the namespace of every document of the pipeline:

      couchbase-cli bucket-create -c localhost:8091 -u Administrator -p '<admin password>' \
  --bucket sales --bucket-type couchbase --bucket-ramsize 512 --storage-backend couchstore

    2. Create the user with three roles on that bucket. scope_admin lets Gluesync create scopes and collections, data_reader and data_writer let it read and write documents:

      couchbase-cli user-manage -c localhost:8091 -u Administrator -p '<admin password>' \
  --set --rbac-username gsuser --rbac-password '<a strong password>' \
  --roles 'scope_admin[sales],data_reader[sales],data_writer[sales]' --auth-domain local
    Expected result

    Both commands answer SUCCESS. In the Web Console, Security lists gsuser with the three roles on sales.

    scope_admin is required even if every scope and collection already exists. When you connect, Gluesync creates a scope named gluesync-scope in the bucket, with the collections gluesync-collection and metadata-collection. It uses them for its own bookkeeping, so do not delete them. Step 2 shows what happens without the role.
    Create the pipeline and the Couchbase target connection

    UI

    1. In the Control Plane, select Create your first pipeline, or the + button of the pipeline list if you already have pipelines. The Agents setup wizard opens;

    2. In 1. Setup source, choose the PostgreSQL CDC agent tile and fill Base settings with the host, port, database and user of your source. Select Test and continue, then Next;

    3. In 2. Setup target(s), choose the Couchbase tile and fill Base settings:

      • Host name / Address: a node of your cluster. Gluesync discovers the other nodes;

      • Port: 8091;

      • Bucket name: sales;

      • Username and Password: the user of step 1;

    4. Select Test and continue. The target appears with gsuser and the address. Select Next, enter a Pipeline name and select Finish.

    Setup target page of the Couchbase agent with host
    Figure 1. The Couchbase target connection. The key list under Advanced connection properties offers the timeouts, the number of key-value connections, DNS SRV and client-side compression.

    For TLS, upload your certificate under Security and use port 18091. For Capella, use the host name of the cluster and port 18091. The Couchbase target reference describes Capella.

    Expected result

    The target card shows a green check. In Couchbase, the bucket now has the scope gluesync-scope. The pipeline appears in the HEALTHY group of the pipeline list.

    If the test fails, the connection is saved anyway and the notification Failed to save shows the cause. Open the menu of the target card, select Edit, correct the values and test again. The troubleshooting table lists the messages.

    Add the entity and read the mapping

    UI

    1. In the pipeline, select Entities. The Add entities page opens with the Object browser of the source. Expand the schema public and select the check box of orders;

    2. Gluesync maps all nine columns at once. Select the mapping icon of the entity in Selected entities to open Field mapping. The Target Scope is public, the schema of the source table, and the Target Collection is orders, the name of the table;

    3. Read the Target type column. The key column id is marked as a key and is not editable. Select Confirm, then Confirm and continue.

    Field mapping dialog with source columns of public.orders on the left
    Figure 2. The field mapping of a Couchbase target. The scope is the source schema, the collection is the source table, and the primary key is marked as the key.
    Expected result

    The entity public.orders is in the Hold group of the pipeline. Nothing is written to Couchbase yet: Gluesync creates the scope and the collection when it starts the entity.

    Every entity needs a key, because the key names the document. A table without a primary key is rejected, unless you choose the key columns with Check keys. Read Choose keys for tables without a primary key.

    Run the snapshot

    UI

    1. In the Hold group, select the play icon of public.orders. The dialog Sync changes (CDC) opens;

    2. Leave Start from on Full copy and Data already in the target on Keep and merge. The snapshot overwrites a document with the same key and leaves the others alone;

    3. Select Start entity.

    Start dialog with Standard
    Figure 3. The start dialog of an entity, with the speed details open. The Writing section says that the Couchbase target has no bulk load, and sets the number of writers.
    Expected result

    The entity moves to the Active group and Last rows grows. On the test table of 200,000 rows on a local Docker host, the snapshot read and the snapshot write both finished in about five seconds.

    Read the documents in Couchbase

    SQL

    Gluesync created the scope public and the collection orders for you. Query them as an administrator. The scope name public is a reserved word in N1QL, so write every name between backticks. Create a primary index first, because COUNT needs an index:

    CREATE PRIMARY INDEX IF NOT EXISTS ON `sales`.`public`.`orders`;
SELECT COUNT(*) AS c FROM `sales`.`public`.`orders`;
SELECT META(o).id AS doc_key, o.* FROM `sales`.`public`.`orders` o USE KEYS ["public::orders::1"];
    Expected result

    The count is 200000. The document key is public::orders::1, the scope, the collection and the primary key, joined by ::. The document is:

    {"amount": 771.12, "created_at": "2026-11-09T08:20:59.470876Z", "customer": "customer-1", "id": 1,
 "note": "note 1", "paid": false, "ship_date": "2026-11-10", "status": "paid", "tags": ["a", "b1"]}

    The mapping from relational to JSON is direct, one column for one field:

    Source column JSON field

    numeric(12,2)

    A JSON number: 771.12

    timestamptz

    An ISO 8601 string in UTC: "2026-11-09T08:20:59.470876Z"

    date

    A string: "2026-11-10"

    boolean

    true or false

    text[]

    A JSON array: ["a","b1"]

    NULL

    A field with the value null: "note": null

    The rows stay flat: Gluesync does not nest the rows of a child table in the document of its parent. To build nested documents, use a data modeling entity, described in the reference.

    Follow inserts, updates and deletes with CDC

    SQL

    Change the source table:

    INSERT INTO public.orders VALUES (200001, 'new customer', 99.90, 'new', now(), current_date, false, ARRAY['x'], 'inserted by cdc');
UPDATE public.orders SET status = 'shipped', amount = 123.45 WHERE id = 1;
DELETE FROM public.orders WHERE id = 2;

    Wait a few seconds and read the three documents:

    SELECT META(o).id, o.* FROM `sales`.`public`.`orders` o
USE KEYS ["public::orders::200001", "public::orders::1", "public::orders::2"];
    Expected result

    The query returns two documents. public::orders::200001 holds the new row, and public::orders::1 shows "status": "shipped" and "amount": 123.45. public::orders::2 is not returned: Gluesync removed the document.

    Every write is a key-value operation on one document: an insert, an upsert or a remove. Gluesync keeps the writes of one document in order, and runs the writes of different documents in parallel. A transient failure such as a timeout is retried on that write alone.

    Set the scope, the collection and the document key

    UI

    The second table, order_lines, has a composite key. This step writes it into a scope called shop, and names its documents with a prefix.

    1. In Add entities, check order_lines, then select the Settings (gear) icon of the entity. Select Target and then couchbase;

    2. Under Target Scope, clear the value, type shop and select Add "shop". The value shows the badge NEW. Changing the scope clears the Target Collection, so enter order_lines again in the same way;

    3. Under Document key, the Preview shows the default key, public :: order_lines :: order_id :: line_no. Turn on Customize document key, and set:

      • Prefix: line;

      • Separator: -. The list offers ::, -, _, / and None;

      • Fields: both key columns, in the order of Key order. Move a field with the arrows;

      • Suffix: empty;

    4. Close the dialog, select Confirm and continue, and start the entity as in step 4.

    Settings of public.order_lines with the Target tab open
    Figure 4. A custom document key with the prefix line, the separator - and both key fields. The Preview shows the result.

    Check the result in Couchbase:

    CREATE PRIMARY INDEX IF NOT EXISTS ON `sales`.`shop`.`order_lines`;
SELECT META(o).id, o.* FROM `sales`.`shop`.`order_lines` o USE KEYS ["line-1-1", "line-100-3"];
    Expected result

    The scope shop and the collection order_lines exist, and Gluesync created them for you. The key is line-1-1, and the document is {"line_no": 1, "order_id": 1, "qty": 1, "sku": "SKU-1"}. The collection holds 300 documents.

    Choose the key before the first snapshot. A different key writes different documents, and the documents already written under the old key stay in the collection. The key must be unique for each row of the table: two rows with the same key overwrite each other.

    The Entity tab of the same dialog has TTL (In seconds. Defaults to 0s, meaning no TTL). A value above 0 gives every document of that entity an expiry.

    Switch off scopes and collections

    UI

    By default, Gluesync writes each table in its own collection. For a cluster that must keep every document in the default collection, turn this off in the target. In the connection form, open Advanced settings, select Add property, choose the key Use Scope and Collections and turn the value off. In a pipeline that already exists, open the Edit menu of the target.

    With the option off, Gluesync needs more from the user, because it finds the tables by reading the type field of the documents in _default._default:

    1. Add the role query_select on the default collection:

      couchbase-cli user-manage -c localhost:8091 -u Administrator -p '<admin password>' \
  --set --rbac-username gsuser \
  --roles 'scope_admin[sales],data_reader[sales],data_writer[sales],query_select[sales:_default:_default]' \
  --auth-domain local

    2. Create a primary index on the default collection, or grant query_use_sequential_scans. Without an index the object browser fails with the error of the troubleshooting table:

      CREATE PRIMARY INDEX ON `sales`.`_default`.`_default`;
    Expected result

    After a snapshot of order_lines, the documents are in _default._default. Each one carries an extra field "type": "order_lines" with the name of the collection, and the default key is public::order_lines::100::1. The advanced property Custom 'type' field name renames the field.

    Tune the write speed

    UI

    The Couchbase target writes one document at a time on parallel lanes. It has no bulk load: the start dialog says This target has no bulk load. The speed depends on the number of writers.

    1. Open the start dialog of an entity and select Faster, or Custom and See or change the details;

    2. Under Writing, set Writers during the copy from 1 to 10. The Standard speed uses two writers and the Faster speed uses four. Each writer drives 16 writes at the same time, so two writers keep 32 writes in flight;

    3. Under Reading, turn on Read in parallel and choose a column with evenly spread values, such as an ID, to read the table in ranges.

    Expected result

    On the test table of 200,000 rows on a local Docker host, the snapshot write took about 4.8 seconds with the Standard speed and about 3.8 seconds with four writers and four parallel reads. On a remote cluster, the network and the number of nodes decide the speed. Raise the writers step by step and watch the load on the Data service.

    The connection settings under Advanced connection properties tune the client:

    Property Default When to change it

    Key-Value Timeout

    30 seconds

    Raise it for a slow or remote cluster. A write that exceeds it is retried

    Key-Value Durable Timeout

    30 seconds

    The timeout of durable writes

    Key-Value Connections Number

    2

    More connections for a very high write rate

    Enable Client-Side Compression

    Off

    Turn it on to save network bandwidth on a slow link

    Connection Timeout, Management Timeout, Query Timeout

    30, 75, 75 seconds

    Raise them for a cluster far from Core Hub

    Gluesync does not ask for a durability level on the key-value writes of a snapshot or of CDC. Only the transactions of data modeling entities use the durability level majority. The properties Write page size and Write page multiplier in the Entity tab are deprecated and ignored.

    Read Tune replication throughput for the settings that apply to every target.

    Verify the result

    You have set up the Couchbase target if:

    • The user has scope_admin, data_reader and data_writer on the bucket, and query_select plus an index if you turned off scopes and collections;

    • The scope gluesync-scope exists in the bucket and you did not delete it;

    • The count of documents in the collection equals the source count after the snapshot;

    • A CDC insert, update and delete reach the collection, and the deleted document is gone;

    • You know the document key of each entity, and it is unique for each row;

    • You chose the scope, the collection and the key before the first snapshot.

    Troubleshooting

    Problem Possible cause What to do

    Failed to save with User authentication failed; check username and password

    Wrong user or password. The values are saved anyway. The notification starts with WaitUntilReady timed out in stage WAIT_FOR_CONFIG and takes about 15 seconds

    Correct the values in the Edit menu of the target and test again

    Failed to save with Either the bucket with name "<name>" is not present, the user does not have the right privileges to access it, or the bucket is hibernated

    Wrong Bucket name, or the user has no role on that bucket

    Create the bucket, or grant the roles of step 1 on it

    Failed to save with Unexpected HTTP status 403 Forbidden and User needs the following permissions cluster.bucket[<name>].collections!write

    The user has no scope_admin role. Gluesync cannot create gluesync-scope

    Grant scope_admin[<bucket>]

    The entity moves to the Error group with UpsertRequest failed with unexpected status code NO_ACCESS and Not authorized for command

    The user lost data_writer, or never had it

    Grant data_writer[<bucket>]. The entity recovers on its own once the role is back, and the pending change is written

    Tables Fetch and Schemas Fetch fail with User does not have credentials to run SELECT queries on default:<bucket>._default._default

    Use Scope and Collections is off and the user cannot query the default collection

    Grant query_select[<bucket>:_default:_default]

    Tables Fetch fails with User does not have credentials to use sequential scans

    Use Scope and Collections is off and the default collection has no index

    CREATE PRIMARY INDEX ON <bucket>._default._default;, or grant query_use_sequential_scans

    A N1QL query on <bucket>.public.<collection> fails with a syntax error near public

    public is a reserved word

    Write the scope between backticks: +`bucket.public.orders+`

    The count is lower than the source

    Two rows have the same document key, and the second overwrote the first

    Include every key column in the Document key, or choose other fields

    Documents of an old key remain after you changed the key

    A key change writes new documents and does not remove the old ones

    Delete the old documents in Couchbase, or empty the collection and run a new snapshot

    Related guides

    Contact MOLO17 Support if you need assistance with a Couchbase target.