Gluesync SQL to NoSQL

Installation steps

Basic configuration example

This video tutorial will guide you through the basic configuration steps for setting up Gluesync for Microsoft SQL Server and Couchbase

The SQL to NoSQL module can be customized by using a configuration file, in JSON format. The file name to use must be specified as a parameter when launching the app, with the -f or --file tokens. The file must have the following structure:

{
  "sourceHost": "RDBMS_SERVER_ADDRESS_HERE",
  "sourcePort": "1433",
  "sourceName": "db-name",
  "sourceUsername": "db-user",
  "sourcePassword": "db-pwd",
  "sourceEntities": {
    // an example of virtual entity configured for Advanced data modeling
    "orders": {
      "dataModeling": [...],
      "type": "orders",
      "scope": "myscope"
    },
    // an example of virtual entity configured for Full table
    "DocumentType2": {
      "table": "CUSTOMERS",
      "schema": "MYSCHEMA",
      "type": "customers",
      "scope": "myscope"
    },
    // an example of virtual entity configured for Mapping
    "DocumentType3": {
      "mapping": {
        "id": "DRIVERID",
        "name": "DRIVERNAME",
        "surname": "DRIVERSURNAME"
      },
      "table": "DRIVERS",
      "schema": "MYSCHEMA",
      "type": "drivers",
      "scope": "myscope"
    },
    // an example of virtual entity configured for SQL query data modeling
    "locations": {
      "query": "SELECT ID, latitude, longiture FROM LOCATIONS INNER JOIN ... WHERE ...",
      "type": "orders",
      "scope": "myscope"
     }
  },

  "sourceChangeRetention": 5,

  "targetHost": "NOSQL_SERVER_ADDRESS_HERE",
  "targetPort": "8091",
  "targetName": "database-name-or-bucket-name",
  "targetUsername": "user",
  "targetPassword": "pwd",

  "copySourceEntitiesAtStartup": true,
  "maxTransactionCountPerIteration": 100,
  "maxItemsCountPerTransaction": 100,
  "maxMigrationItemsCountPerIteration": 1000
}

The parameters that refer to a source are the parameters that describe the connection to the SQL instance.

sourceHost: the host where the instance is reachable from;
sourcePort: the port which the instance is reachable from;
sourceName: the name of the source database in the SQL instance;
sourceUsername: username of a privileged user in the instance;
sourcePassword: password of such user;
sourceEntities: tables that Gluesync is going to replicate to the target database;
sourceChangeRetention (optional): defaults to 5. Number of retention days preserved into the source database before being cleared/purged either by the source database system or by Gluesync engine;

The parameters that refer to a target are the parameters that describe the connection to the NoSQL instance.

targetHost: the host where the instance is reachable from;
targetPort: the port which the instance is reachable from;
targetName: the name of the target database | bucket in the NoSQL instance;
targetUsername: username of a privileged user in the instance;
targetPassword: password of such user;

Other configuration parameters are the following:

copySourceEntitiesAtStartup: boolean which indicates whether Gluesync should copy the contents of the whole table when launched the first time;
maxTransactionCountPerIteration: the maximum number of transactions that should be processed when simultaneously;
maxItemsCountPerTransaction: the maximum number of rows that should be processed within a single transaction. This will involve a paged read of rows affected by the underlying transaction;
maxMigrationItemsCountPerIteration: defaults to 1000, it’s the size of each chunk of data being loaded by the initial snapshot process per each one of its threads. Change that value according to your specific use case needs;

Looking for data modeling features or other options?

For more detailed configuration options, including the ability to perform data modeling, please have a look at the dedicated Data modeling section when sourcing from an RDBMS.