Search docs.molo17.com
    Configuration guide Connectors

    Set up Apache CouchDB as a source and a target

    Apache CouchDB works in Gluesync as a source and as a target, with the same agent. This guide takes you from two CouchDB servers to a running pipeline that copies a database with a snapshot and then follows its changes. You prepare a least-privilege user on each server, create the connections, map the key, start the entity and check the documents on the other side. The last part covers the settings that decide the write speed and the errors you are most likely to meet.

    In this guide

    You will learn how to create a CouchDB user that can read the source with no administrator rights, create a target database with the number of shards you choose, connect CouchDB as a source and as a target in the Control Plane, set _id as the key, start a snapshot with live changes, check inserts, updates and deletes with curl, change the shards and the write settings of the target, avoid the field names that CouchDB refuses, and fix the 401 and 403 answers of the server. A table of data type notes and a symptom, cause and fix table close the guide.

    Quick information

    Goal

    Replicate a CouchDB database to a second CouchDB with a snapshot and CDC, using users that have only the rights Gluesync needs

    Audience

    Data engineers and database administrators who own a CouchDB source or target

    Level

    Intermediate

    Time required

    About 45 minutes

    Area

    CouchDB → curl and _security; Control Plane → Data pipelines → Apache CouchDB agent

    Applies to

    Gluesync 2.3

    Before you begin

    Make sure that:

    1. You have two Apache CouchDB 3.3, 3.4 or 3.5 servers that Core Hub can reach over HTTP, one for the source and one for the target. Single node or cluster both work. This guide was tested on CouchDB 3.5.1 with one single-node server for each role. The guide calls them couchdb-source and couchdb-target, on port 5984;

    2. You can run curl against both servers with a server administrator account. The examples write that account as admin:<admin password>;

    3. You can sign in to the Control Plane with a role that can create pipelines;

    4. You know that CouchDB replicates documents, not tables: one entity is one database. Every entity sits under the fixed schema _default.

    The curl commands in this guide are plain CouchDB HTTP calls. Run them from any machine that reaches the servers. For a cluster, send them to any node. The error texts quoted here were captured on CouchDB 3.5.1. A different CouchDB version can word them differently.

    Guide steps

    Optional: create the example source database

    CLI

    Skip this step if you already have a database to replicate. It creates a small database with an array, a nested object, a null, a decimal and a design document. The design document shows that CouchDB internals are not replicated.

    1. On the source server, create the database shop_orders and add four documents:

      curl -u admin:<admin password> -X PUT "http://couchdb-source:5984/shop_orders?q=2"

curl -u admin:<admin password> -X POST http://couchdb-source:5984/shop_orders/_bulk_docs \
  -H 'Content-Type: application/json' -d '{"docs":[
 {"_id":"order-1001","customer":"Acme Spa","total":1500.5,"items":3,"paid":true,
  "placed":"2026-10-28T10:00:00Z","tags":["gold","eu"],"shipping":{"city":"Rome","zip":"00100"}},
 {"_id":"order-1002","customer":"Globex","total":42,"items":1,"paid":false,
  "placed":"2026-10-29T09:30:00Z","tags":[],"shipping":{"city":"Berlin","zip":"10115"}},
 {"_id":"order-1003","customer":"Initech","total":99.99,"items":2,"paid":true,
  "placed":"2026-10-30T14:15:00Z","tags":["new"],"shipping":null},
 {"_id":"_design/app","views":{}}]}'
    Expected result

    CouchDB answers {"ok":true} for the database and an ok entry with a rev for each of the four documents.

    Create a read-only user on the source server

    CLI

    Gluesync reads the changes feed and the documents of a database. It does not need an administrator for that. It needs a user that is a member of every database to replicate.

    1. Create the user in the _users database:

      curl -u admin:<admin password> -X PUT \
  http://couchdb-source:5984/_users/org.couchdb.user:gluesync \
  -H 'Content-Type: application/json' \
  -d '{"name":"gluesync","password":"<a strong password>","roles":[],"type":"user"}'

    2. Make the user a member of the database. The request replaces the whole _security object, so add the user to the members that the database already has:

      curl -u admin:<admin password> -X PUT http://couchdb-source:5984/shop_orders/_security \
  -H 'Content-Type: application/json' \
  -d '{"admins":{"names":[],"roles":[]},"members":{"names":["gluesync"],"roles":[]}}'

      Repeat the second request for every database you want to replicate;

    3. Check that the user can read the database but cannot list the server:

      curl -u gluesync:<password> http://couchdb-source:5984/shop_orders/_all_docs
curl -u gluesync:<password> http://couchdb-source:5984/_all_dbs
    Expected result

    The first call lists the documents, design document included. The second call answers {"error":"unauthorized","reason":"You are not a server admin."}. This is normal. In CouchDB 3.x only a server administrator can list the databases, so in the next steps you type the database names in the connection instead.

    Prepare the target server

    CLI

    The target agent reads the current revision of the documents it updates and then writes, replaces and deletes them. The user needs read and write access to the database. Creating a database is a server administrator action, so you have two choices:

    • Create the database yourself, with the number of shards you want, and give Gluesync a user that is only a member of it. This guide does that;

    • Connect Gluesync as a server administrator and let the agent create the missing databases. The agent then uses the Shards (q) setting, 8 by default.

      1. Create the user, the database with eight shards, and the membership:

        curl -u admin:<admin password> -X PUT \
  http://couchdb-target:5984/_users/org.couchdb.user:gluesync \
  -H 'Content-Type: application/json' \
  -d '{"name":"gluesync","password":"<a strong password>","roles":[],"type":"user"}'

curl -u admin:<admin password> -X PUT "http://couchdb-target:5984/shop_orders?q=8"

curl -u admin:<admin password> -X PUT http://couchdb-target:5984/shop_orders/_security \
  -H 'Content-Type: application/json' \
  -d '{"admins":{"names":[],"roles":[]},"members":{"names":["gluesync"],"roles":[]}}'

      2. Read the database back:

        curl -u admin:<admin password> http://couchdb-target:5984/shop_orders
    Expected result

    The answer contains "doc_count":0 and "cluster":{"q":8,"n":1,…​}. The number of shards (q) is the main lever of the write speed, and it cannot be changed once the database exists.

    The name of the target database is the name of the entity, lowercased, with every character outside a-z, 0-9, _, $, (, ), +, / and - changed to _. A Database name prefix in the target settings and a Database name override on the entity change it. See Database names.
    Create the pipeline and the CouchDB source connection

    UI

    1. Open Data pipelines and select Create your first pipeline, or the + button when you already have pipelines. The Agents setup wizard opens on 1. Setup source;

    2. In the search field, type couch and select the Apache CouchDB agent;

    3. In Node address, enter the server without the scheme (couchdb-source, not http://couchdb-source). Keep the port 5984;

    4. In Databases (comma separated), enter shop_orders. The agent lists exactly these databases, because the user cannot call _all_dbs;

    5. In Username and Password, enter the user of step 2;

    6. Select Test and continue.

    Apache CouchDB connection form with node address couchdb-source
    Figure 1. The CouchDB source connection form: node address, port, the database list, user and password.
    Expected result

    The source card shows a green check and gluesync@couchdb-source:5984.

    TLS, a private certificate authority, more nodes of a cluster and a path prefix behind a reverse proxy are all options of the same form. See Setup via Web UI in the agent reference. Turn TLS on for any CouchDB that is not on a private network. The server certificate is always validated.
    Add the CouchDB target and finish the pipeline

    UI

    1. Select Next. On 2. Setup target(s), search for couch and select Apache CouchDB;

    2. Enter Node address couchdb-target, Databases (comma separated) shop_orders, and the user and password of the target user. On a target, the Databases field only fills the object browser, so it does not limit what the agent writes;

    3. Select Test and continue, then Next;

    4. On 3. Ready, enter CouchDB orders in Pipeline name and select Finish.

    Expected result

    The pipeline CouchDB orders opens with no entities. Its source and target icons are healthy.

    Add the entity and set _id as the key

    UI

    1. Select Entities. The page Listing database objects for shop_orders opens;

    2. In the Object browser, expand _default and tick shop_orders. The row shows Error in field mapping. Please review, because the entity has no key yet;

    3. In Selected entities, select the Field mapping icon of shop_orders. The dialog lists the fields that the agent inferred from a sample of the documents: _id and the seven fields of the example;

    4. On the source side, tick the key checkbox of _id. The target side follows;

    5. Select Confirm, then Confirm and continue.

    Field mapping dialog with the source database shop_orders and the target shop_orders
    Figure 2. The field mapping with _id as the key.
    Expected result

    The entity _default.shop_orders appears in the group Hold. Without a key the dialog shows the errors No key on the source and No key on the target, and Confirm stays disabled.

    Keep _id as the key. A delete in the changes feed carries only the _id of the document. On an entity whose key has more columns, the agent skips the delete and logs it, and the document stays on the target.
    Start the snapshot and CDC

    UI

    1. In the entity table, tick _default.shop_orders and select Start CDC in the blue bar. The dialog Sync changes (CDC) opens;

    2. In Start from, keep Full copy (Recommended for the first start);

    3. In Data already in the target, keep Keep and merge. Keep the default insert conflict strategy of the entity, Upsert: the snapshot and CDC overlap on purpose, and the upsert absorbs the overlap. See Insert conflict strategy;

    4. Select Start entity.

    Sync changes (CDC) dialog with the Full copy tile and the Keep and merge option selected
    Figure 3. The start dialog with Full copy and Keep and merge selected.
    Expected result

    After a few seconds the entity moves to the group Active and shows 3 in Last rows. The snapshot read the ids from the changes feed and the documents with _bulk_get requests. The design document was left out.

    For the snapshot and CDC start options in detail, see Start a snapshot and CDC replication from the Control Plane.

    Check the snapshot on the target

    CLI

    1. List the documents of the target database:

      curl -u admin:<admin password> http://couchdb-target:5984/shop_orders/_all_docs

    2. Read one document on both sides and compare them:

      curl -u admin:<admin password> http://couchdb-source:5984/shop_orders/order-1001
curl -u admin:<admin password> http://couchdb-target:5984/shop_orders/order-1001
    Expected result

    The target lists order-1001, order-1002 and order-1003, and no _design/app. The documents have the same _id, the same fields and the same value types: the array tags and the object shipping stay JSON, and shipping: null stays null.

    Check live changes: insert, update and delete

    CLI

    1. On the source, insert a document, update another one and delete a third:

      # insert
curl -u admin:<admin password> -X PUT http://couchdb-source:5984/shop_orders/order-1004 \
  -H 'Content-Type: application/json' \
  -d '{"customer":"Umbrella","total":10,"items":1,"paid":false,"placed":"2026-11-02T08:00:00Z","tags":["x"],"shipping":{"city":"Milan","zip":"20100"}}'

# update: send the current _rev of order-1002
REV=$(curl -s -u admin:<admin password> http://couchdb-source:5984/shop_orders/order-1002 | jq -r ._rev)
curl -u admin:<admin password> -X PUT http://couchdb-source:5984/shop_orders/order-1002 \
  -H 'Content-Type: application/json' \
  -d "{\"_rev\":\"$REV\",\"customer\":\"Globex\",\"total\":42,\"items\":1,\"paid\":true,\"placed\":\"2026-10-29T09:30:00Z\",\"tags\":[\"paid\"],\"shipping\":{\"city\":\"Berlin\",\"zip\":\"10115\"}}"

# delete: send the current _rev of order-1003
REV=$(curl -s -u admin:<admin password> http://couchdb-source:5984/shop_orders/order-1003 | jq -r ._rev)
curl -u admin:<admin password> -X DELETE "http://couchdb-source:5984/shop_orders/order-1003?rev=$REV"

    2. Wait a few seconds and read the target:

      curl -u admin:<admin password> http://couchdb-target:5984/shop_orders/_all_docs
curl -u admin:<admin password> http://couchdb-target:5984/shop_orders/order-1002
curl -u admin:<admin password> http://couchdb-target:5984/shop_orders/order-1003
    Expected result

    The target lists order-1001, order-1002 and order-1004. order-1002 has "paid":true and "tags":["paid"]. Reading order-1003 answers {"error":"not_found","reason":"deleted"}: the delete reached the target as a tombstone. In Last trans. the entity row shows the opaque position of the changes feed, which Core Hub keeps as the checkpoint.

    Choose the shards and the write settings of the target

    UI

    The target settings are properties of the agent connection. Open them when you let the agent create databases, or when you need more write speed.

    1. In the pipeline, select the gear icon above the entity table, then Edit target;

    2. Open Advanced settings and select Add property;

    3. In Key, select Shards (q). The field shows the default, 8. Enter the value you want, for example 16;

    4. Add more properties if you need them, for example Create database if missing, Write concurrency or Write batch size;

    5. Select Test and continue.

    Edit target page on the Advanced settings tab with the property Shards (q) selected and the value 8
    Figure 4. The Shards (q) property in the Advanced settings of the target.
    Expected result

    The pipeline page opens again. A database that the agent creates from now on gets the new number of shards. Databases that already exist keep theirs. In our test, an entity added after setting Shards (q) to 16, with the target connected as a server administrator, created a database with "q":16.

    The write throughput grows with the number of shards and with the requests in flight (Write concurrency). See Throughput and shards for the measurements and for how a snapshot upsert skips the revision lookup. The source has its own properties, for example Snapshot read concurrency, in Edit source → Advanced settings. See Snapshot.

    Data type notes

    CouchDB stores JSON. The source agent gives every field a type from the JSON kinds in a sample of its documents (Schema discovery sample size, 1000 by default), and the target writes the natural JSON form of each type.

    In the CouchDB document Gluesync type What to know

    String

    STRING

    Written back as a string. For a target column of type STRING you can choose json, which parses the text and writes the JSON value.

    Integer

    LONG

    A value that is not an exact 64-bit integer fails the row and names the field. Integers keep every digit.

    Decimal or exponent

    BIG_DECIMAL

    CouchDB stores a decimal as an IEEE double, so 12345678901234567890.123456789 is read back as 12345678901234567000.0. In the lab, the same document on the source and on the target held that value.

    true or false

    BOOLEAN

    Written as a JSON boolean.

    Object

    MAP

    Converted recursively. A target column of type MAP can also be written as a string with string.

    Array

    ARRAY

    Converted element by element, including null and mixed kinds. The target can write the JSON text with string.

    A field with two or more kinds, or only null

    STRING, as JSON text

    The type is json. Choose json for the target column to write it back as JSON.

    A field that a document lacks

    Absent

    The absent field policy of the entity decides what the target receives.

    Names that start with an underscore

    Not a field

    _id is the key. _rev and _attachments are never fields. Attachments are neither read nor written.

    A field that appears only in documents outside the sample is not discovered. Raise Schema discovery sample size in the source properties for a database whose documents have different shapes. The complete matrices are in Data types of the source and Data types of the target.

    Recover from the failures you are most likely to meet

    The target database does not exist and the user is not an administrator

    With Create database if missing on (the default), the first write creates a database that is not there. CouchDB reserves that to server administrators. A target user that is only a member gets this error on the entity:

    Authentication Failed
PUT /shop_customers failed with HTTP 401 (unauthorized): You are not a server admin.

    To fix it, create the database as an administrator, with the shards you want, and add the user as a member (step 3). You do not restart the entity: it keeps retrying, and in our test it wrote its documents to the new database within a minute. The other choice is to connect the target as a server administrator.

    A database that the agent creates gets CouchDB’s default security object, with the role _admin as its only member. A target user that is not an administrator cannot write to it. Set _security on the new database as in step 3.

    A technical field is refused on a CouchDB target

    The source property Technical fields adds columns such as _TRANSACTION_ID. CouchDB reserves the names that start with an underscore, and _bulk_docs rejects a whole request that carries one. The agent refuses the entity when you save it:

    Add entities page with the red banner The entity could not be saved
    Figure 5. The entity could not be saved because the target column starts with an underscore.
    The entity could not be saved.
Entity _default.shop_audit: CouchDB reserves field names that start with an underscore,
so '_TRANSACTION_ID' cannot be written. Rename the target column, for example
'TRANSACTION_ID'. Only '_id' may keep its name: it becomes the id of the document.

    To fix it, open the Field mapping of the entity, type TRANSACTION_ID in the Target (name) field of that row, and confirm. In our test the snapshot wrote "TRANSACTION_ID":null, because the snapshot has no change position, and every change read by CDC carried the position of the changes feed. See Technical fields.

    Troubleshooting

    Problem Possible cause What to do

    Host Credentials Update Failed or Failed to save: GET / failed with HTTP 401 (unauthorized): Name or password is incorrect

    The user name or the password is wrong. The connection is saved before the agent tests it

    Correct the user and the password in Edit source or Edit target, then select Test and continue again. Do not leave the wrong values saved

    None of the nodes http://couchdb-source:5984 could be reached: …​ Name or service not known

    The node address is wrong, has a scheme (http://), or the name does not resolve from Core Hub

    Enter the host name without the scheme, and test it from the machine or container that runs Core Hub. For a cluster, add the other nodes in Additional nodes

    The object browser stays empty and a Schemas Fetch notification says GET /<database> failed with HTTP 403 (forbidden): You are not allowed to access this db.

    The user is not a member of one of the databases in Databases (comma separated). One such name stops the whole list

    Add the user to the members of that database (step 2), or remove the name from the connection

    The object browser is empty and Databases is blank

    The user is not a server administrator, so _all_dbs is refused

    Type the database names in Databases (comma separated), or connect with a server administrator

    Error in field mapping. Please review, with No key on the source and No key on the target

    The entity has no key. CouchDB documents have no declared key

    Tick the key checkbox of _id in the Field mapping (step 6)

    Authentication Failed: PUT /<database> failed with HTTP 401 (unauthorized): You are not a server admin. on a target entity

    The target database does not exist and the user cannot create it

    Create the database as an administrator and add the user as a member. The entity recovers without a restart

    Permission Denied (HTTP 403) on a target entity

    The user is not a member of the target database, or the database was created by the agent with the default security

    Add the user to the members of the database with _security

    Database Not Found (HTTP 404)

    Create database if missing is off and the database does not exist, or the Path prefix is wrong

    Create the database with the name the agent computes (see the note in step 3), or turn the setting on with a server administrator

    CouchDB reserves field names that start with an underscore, so '_TRANSACTION_ID' cannot be written

    A technical field, or another column, starts with an underscore on a CouchDB target

    Rename the target column in the Field mapping, for example TRANSACTION_ID

    Deletes do not arrive on the target

    The entity key is not _id. A delete carries only the _id, and the agent skips it otherwise

    Make _id the only key of the entity

    CDC fails and asks for a new snapshot (HTTP 400 on the position)

    The changes feed no longer knows the position that the entity holds

    Start the entity again with Full copy

    Documents Rejected with ids and reasons

    A validate_doc_update function of the target database refuses the document, or a field name is not valid

    Read the reasons in the message, fix the document or the function. The documents that CouchDB accepted stay written

    Document Conflict or conflicts that keep changing after the retries

    Another writer updates the same documents while Gluesync writes them

    Stop the other writer, or raise Conflict retries in the target properties

    The snapshot or the writes are slow

    Few shards on the target database, few requests in flight, or a busy server

    Create the target databases with more shards (step 3), raise Write concurrency (step 10) and, on the source, Snapshot read concurrency

    A field that a document has is missing on the target

    The field appears only in documents outside the discovery sample

    Raise Schema discovery sample size in the source properties and map the entity again

    Related guides

    Contact MOLO17 Support if you need assistance with Apache CouchDB.