Search docs.molo17.com

    Source Agent Advanced Settings

    This document provides detailed information about all advanced settings available for Gluesync source agents. These settings control change data capture (CDC) behavior, performance tuning, connection parameters, and other agent-specific configurations.

    Common Source Settings

    These settings are available across most source agents and control fundamental CDC behavior.

    Source change retention in hours

    Description: Controls how long change data is retained in the source agent’s local ArenaCache before being discarded.

    Explanation: This setting is the purge clock for ArenaCache, the disk-backed source cache that became the platform default in 2.2.11.2 (replacing Chronicle Queue). Changes older than this period are evicted from the agent’s local files. A higher value gives a longer window if Core Hub or the target is unavailable; a lower value keeps disk use down. The default of 24 hours (1 day) is suitable for most use cases. This setting does not move the CDC reader checkpoint — that lives in Core Hub for cache-based readers as of 2.2.11. See Source-side cache.

    Default: 24 (1 day)

    Applies to: Source

    For a step-by-step walkthrough, see the how-to guide Free disk space and manage cache and logs.

    IBM i (AS400) Source Advanced Settings

    Enable multi-member physical file support

    Description: Enables discovery and replication of data from all members within a multi-member physical file, treating each member as its own entity with independent CDC tracking.

    Explanation: IBM i physical files can contain multiple members (similar to tables within a table). When enabled, this setting allows Gluesync to discover and replicate data from all members within a multi-member physical file. Each member is treated as a separate entity with its own CDC tracking via journal APIs, snapshots, and bulk loads. This is particularly useful for IBM i systems that use multi-member files for data partitioning or organization.

    Default: false

    Applies to: Source

    Important: This setting is disabled by default. If you are already using the multi-member feature, you must enable this flag in the agent configuration under advanced settings.

    Journal reader polling interval in milliseconds

    Description: Controls how long the agent waits between journal API calls when the journal is completely drained (no new records available).

    Explanation: This setting manages the polling frequency for journal reading. When the IBM i journal returns no new records (indicating it’s been completely drained), the agent will wait for the specified number of milliseconds before making the next API call. If the journal does return records, the agent ignores this interval and immediately makes the next call to catch up with the journal head. Higher values reduce CPU usage on the IBM i system during idle periods but may increase latency for capturing new changes.

    Default: 10000 (10 seconds)

    Applies to: Source

    Journal reader API buffer size in KB (default 16384)

    Description: Sets the size of the receiver buffer used for each journal API call, controlling how much data can be retrieved in a single round trip.

    Explanation: Each call to the IBM i journal API writes into a contiguous receiver buffer whose size is governed by this parameter. The buffer size affects how many journal entries can be returned in a single API call. Regardless of the configured value, each individual call is hard-capped at 10 MB (the maximum receiver size Gluesync requests from the IBM i API in one round trip). When the buffer fills before the journal is drained, the API flags that more entries are available and Gluesync immediately issues a continuation call.

    Default: 16384 (16 MB)

    Applies to: Source

    Enable continuous journal reading (skip delay to next iteration)

    Description: When enabled, skips the delay between journal reading iterations for continuous streaming.

    Explanation: By default, the agent may introduce delays between journal reading iterations to manage resource utilization. When this setting is enabled, the agent skips these delays and performs continuous journal reading, which can reduce latency but may increase CPU usage on the IBM i system.

    Default: false

    Applies to: Source

    Journal reader page size (0 means disabled)

    Description: Controls the page size for journal reader operations (0 means disabled).

    Explanation: This setting controls the pagination of journal reader operations. When set to a non-zero value, it defines the number of journal entries processed per page. A value of 0 disables pagination, meaning all available entries are processed in a single operation.

    Default: 1000

    Applies to: Source

    Journal reader fetch size (0 means default)

    Description: Controls the fetch size for journal reader operations (0 means default).

    Explanation: This setting controls how many journal entries are fetched in each operation. A value of 0 uses the system default fetch size. Adjusting this can affect memory usage and performance of journal reading operations.

    Default: 1000

    Applies to: Source

    Legacy mode journal reader (use DSPJRN command)

    Description: When enabled, uses the legacy DSPJRN command instead of native Journal APIs for journal reading.

    Explanation: This setting enables the legacy journal reading mode that uses the DSPJRN CL command instead of the modern native Journal APIs (QjoRetrieveJournalEntries). The legacy mode may be required for compatibility with older IBM i systems or specific configurations, but generally the native APIs provide better performance and functionality.

    Default: false

    Applies to: Source

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Metadata setup concurrency (parallel QDBRTVFD calls)

    Description: Controls the number of parallel QDBRTVFD calls during metadata setup for table and column information.

    Explanation: This setting controls how many parallel calls are made to the QDBRTVFD API during the metadata setup phase when discovering table and column information. Higher values can speed up the initial setup process for entities with many tables but may increase load on the IBM i system during setup.

    Default: 10

    Applies to: Source

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to parse the user metadata block of incoming journal entries and silently discard any operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Max Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent operations (0 means no limitation).

    Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

    Default: 0 (no limitation)

    Applies to: Source

    Max Snapshot Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

    Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

    Default: 0 (no limitation)

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Disable journal reader

    Description: When enabled, disables the journal reader component for CDC operations.

    Explanation: This setting completely disables the journal reader, preventing the agent from performing CDC operations. This is typically used for snapshot-only configurations or when troubleshooting CDC issues.

    Default: false

    Applies to: Source

    Disable display journal files filter

    Description: When enabled, disables filtering of journal files in display operations.

    Explanation: This setting controls whether journal files are filtered in display operations. Disabling the filter may show more journal files but can impact performance and readability of journal information.

    Default: false

    Applies to: Source

    Display journal rows limit

    Description: Controls the maximum number of journal rows displayed in operations.

    Explanation: This setting limits the number of journal rows that are displayed in various operations and UI components. This helps prevent performance issues and overwhelming output when dealing with high-volume journal data.

    Default: 10000

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include RRN (Relative Record Number), receiver library, receiver name, sequence number, transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: _RRN, _RECEIVER_LIBRARY, _RECEIVER_NAME, _SEQUENCE_NUMBER, _TRANSACTION_ID, _TRANSACTION_TIMESTAMP, _TRANSACTION_OPERATION, _TRANSACTION_USER

    Use Dedicated Journal Reader

    Description: When enabled, uses a dedicated journal reader for this entity instead of the shared journal reader.

    Explanation: This setting controls whether the entity uses a dedicated journal reader instead of participating in the shared journal reader pool. Dedicated readers provide isolated, high-throughput pipelines for specific entities but consume more resources on the IBM i system. This is typically used for high-priority or high-volume entities that require dedicated resources.

    Default: false

    Applies to: Source

    For a step-by-step walkthrough, see the how-to guide Troubleshoot IBM i journals and receivers.

    Oracle LogMiner Source Advanced Settings

    LogMiner polling interval (seconds)

    Description: Controls how often LogMiner polls for new changes in the Oracle redo logs.

    Explanation: This setting determines the frequency at which LogMiner checks the Oracle redo logs for new changes. A lower value means more frequent polling and lower latency for change capture, but may increase CPU usage on the Oracle database. A higher value reduces CPU load but may increase the time it takes for changes to be captured.

    Default: 1 second

    Applies to: Source

    LogMiner start delay (seconds)

    Description: Controls the initial delay before LogMiner begins polling after startup.

    Explanation: This setting introduces a delay before LogMiner starts polling for changes after the agent starts up. This can be useful for allowing the Oracle database to stabilize after startup or for coordinating with other processes.

    Default: 1 second

    Applies to: Source

    LogMiner max error delay (seconds)

    Description: Controls the maximum delay between polling attempts when errors occur.

    Explanation: When LogMiner encounters errors during polling, it implements a backoff strategy. This setting controls the maximum delay between retry attempts. The actual delay increases exponentially based on the backoff factor up to this maximum.

    Default: 60 seconds

    Applies to: Source

    LogMiner backoff factor

    Description: Controls the exponential backoff factor for retry attempts when errors occur.

    Explanation: When LogMiner encounters errors, it increases the delay between retry attempts exponentially. This setting controls the multiplier for this exponential backoff. For example, with a factor of 2, delays would be 1s, 2s, 4s, 8s, etc. up to the maximum error delay.

    Default: 2

    Applies to: Source

    LogMiner batch size

    Description: Controls the number of redo log records processed in each batch.

    Explanation: This setting determines how many redo log records LogMiner processes in each batch. Larger batch sizes can improve throughput but may increase memory usage and latency for individual change processing.

    Default: 1000

    Applies to: Source

    LogMiner restart delay (seconds)

    Description: Controls the delay before restarting LogMiner after a failure or shutdown.

    Explanation: When LogMiner needs to be restarted due to a failure or planned shutdown, this setting controls how long the agent waits before attempting to restart the LogMiner process. This can help prevent rapid restart loops and allow the Oracle database to recover.

    Default: 5 seconds

    Applies to: Source

    Snapshot JDBC row prefetch size

    Description: Controls the number of rows prefetched during snapshot operations via JDBC.

    Explanation: This setting determines how many rows the JDBC driver prefetches during snapshot operations. Higher values can improve snapshot performance by reducing round trips to the database, but may increase memory usage.

    Default: 10000

    Applies to: Source

    Enable JDBC row prefetch for snapshots

    Description: When enabled, uses JDBC row prefetching during snapshot operations.

    Explanation: This setting controls whether JDBC row prefetching is enabled during snapshot operations. Prefetching can significantly improve snapshot performance by reducing the number of round trips to the database.

    Default: true

    Applies to: Source

    Snapshot batch processing multiplier

    Description: Multiplier for batch processing during snapshot operations.

    Explanation: This setting acts as a multiplier for batch processing during snapshot operations. It affects how many rows are processed in each batch during the snapshot phase, impacting both performance and memory usage.

    Default: 10

    Applies to: Source

    Enable parallel LogMiner parsing

    Description: When enabled, uses parallel processing for LogMiner parsing operations.

    Explanation: This setting controls whether LogMiner parsing operations are performed in parallel. Parallel parsing can improve performance on multi-core systems but may increase resource utilization.

    Default: true

    Applies to: Source

    LogMiner parsing parallelism

    Description: Controls the degree of parallelism for LogMiner parsing operations.

    Explanation: This setting determines how many parallel threads are used for LogMiner parsing operations when parallel parsing is enabled. Higher values can improve performance on multi-core systems but may increase CPU and memory usage.

    Default: 4

    Applies to: Source

    CSF buffer size

    Description: Controls the buffer size for the Committed Sparse File (CSF) used for transaction spill.

    Explanation: This setting controls the size of the buffer for the Committed Sparse File, which is used for spilling uncommitted transactions. Larger buffers can handle larger transactions but may increase disk space usage.

    Default: 1000

    Applies to: Source

    Enable Flow-based LogMiner parsing

    Description: When enabled, uses flow-based parsing for LogMiner operations.

    Explanation: This setting controls whether flow-based parsing is used for LogMiner operations. Flow-based parsing can improve performance and resource utilization for certain workloads.

    Default: true

    Applies to: Source

    Skip transactions with LogMiner parse errors

    Description: When enabled, skips transactions that encounter parse errors instead of failing.

    Explanation: This setting controls how the agent handles transactions that encounter parse errors during LogMiner processing. When enabled, transactions with parse errors are skipped and logged, allowing the agent to continue processing other transactions. When disabled, parse errors cause the agent to fail.

    Default: true

    Applies to: Source

    Reconstruct full row on sparse UPDATE

    Description: When enabled, reconstructs the full row on sparse UPDATE operations when minimal supplemental logging is used.

    Explanation: When LogMiner runs under minimal supplemental logging, the SQL_REDO of an UPDATE only contains the changed columns (plus the primary key). Unchanged columns are therefore absent and would be replicated as NULL, corrupting the target row. When enabled, the converter detects such sparse UPDATE images and re-reads the complete row from the source by ROWID so the full after-image is replicated. Disable only when the source tables already have full supplemental logging (ALTER TABLE …​ ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS), in which case the extra per-update SELECT is unnecessary.

    Default: true

    Applies to: Source

    Use Oracle RAC

    Description: When enabled, configures the agent for Oracle Real Application Clusters (RAC) environments.

    Explanation: This setting enables Oracle RAC-specific configurations and behaviors. In RAC environments, additional considerations are needed for handling multiple instances, redo logs, and coordination between nodes.

    Default: false

    Applies to: Source

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include ROWID, transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: ROWID, TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    LogMiner username

    Description: Specifies a dedicated username for the LogMiner connection, separate from the main Oracle connection.

    Explanation: When the LogMiner process uses a different database user than the main Oracle connection, this setting provides that username. This is useful when the LogMiner user has different privileges (e.g., LOGMINING role) than the replication user. Leave empty to use the same credentials as the main connection.

    Default: (empty)

    Applies to: Source

    LogMiner password

    Description: Specifies the password for the dedicated LogMiner username.

    Explanation: This setting provides the password for the LogMiner-specific connection when a dedicated LogMiner username is configured. Leave empty to use the same credentials as the main connection.

    Default: (empty)

    Applies to: Source

    LogMiner database

    Description: Specifies a dedicated database (service name) for the LogMiner connection.

    Explanation: When the LogMiner process connects to a different database or service than the main Oracle connection, this setting provides that database name. Leave empty to use the same database as the main connection.

    Default: (empty)

    Applies to: Source

    LogMiner connection timeout

    Description: Controls the connection timeout for the LogMiner connection in milliseconds.

    Explanation: This setting determines how long the agent waits when establishing the LogMiner-specific connection before timing out. The default of 10000 ms (10 seconds) is suitable for most network conditions.

    Default: 10000 ms (10 seconds)

    Applies to: Source

    LogMiner read timeout (ms)

    Description: Controls the read timeout for LogMiner operations in milliseconds.

    Explanation: This setting determines how long the agent waits for LogMiner read operations to complete before timing out. The default of 60000 ms (60 seconds) provides ample time for reading redo log data.

    Default: 60000 ms (60 seconds)

    Applies to: Source

    For a step-by-step walkthrough, see the how-to guide Set up Oracle LogMiner CDC end to end.

    PostgreSQL Source Advanced Settings

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    WAL sender timeout in seconds (supported from PostgreSQL 12+)

    Description: Controls the timeout for WAL sender operations in seconds.

    Explanation: This setting determines how long the agent waits for WAL sender operations to complete before timing out. This setting is only supported from PostgreSQL version 12 and above. A higher value provides more time for complex operations but may delay error detection.

    Default: 60 seconds

    Applies to: Source

    MySQL Source Advanced Settings

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    SQL Server Source Advanced Settings

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Max Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent operations (0 means no limitation).

    Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

    Default: 0 (no limitation)

    Applies to: Source

    Max Snapshot Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

    Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

    Default: 0 (no limitation)

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Enable automatic Change Data Capture / Change Tracking activation

    Description: When enabled, the agent enables Change Data Capture or Change Tracking on the source for you.

    Explanation: The SQL Server CDC and Change Tracking agents enable the change capture on the database, and on the tables, when you save the connection. Change Data Capture needs the sysadmin server role for the database and the db_owner role for the tables. Change Tracking needs the ALTER permission. With the setting off, the agent posts a warning that lists the statements to run by hand. If a failure happens with the setting on, the setup fails and the error lists the statements. The key is enableAutomaticChangeCaptureActivation.

    Default: false

    Applies to: Source

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Metadata setup concurrency (parallel QDBRTVFD calls)

    Description: Controls the number of parallel QDBRTVFD calls during metadata setup for table and column information.

    Explanation: This setting controls how many parallel calls are made to the QDBRTVFD API during the metadata setup phase when discovering table and column information. Higher values can speed up the initial setup process for entities with many tables but may increase load on the IBM i system during setup.

    Default: 10

    Applies to: Source

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to parse the user metadata block of incoming journal entries and silently discard any operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Max Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent operations (0 means no limitation).

    Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

    Default: 0 (no limitation)

    Applies to: Source

    Max Snapshot Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

    Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

    Default: 0 (no limitation)

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Disable journal reader

    Description: When enabled, disables the journal reader component for CDC operations.

    Explanation: This setting completely disables the journal reader, preventing the agent from performing CDC operations. This is typically used for snapshot-only configurations or when troubleshooting CDC issues.

    Default: false

    Applies to: Source

    Disable display journal files filter

    Description: When enabled, disables filtering of journal files in display operations.

    Explanation: This setting controls whether journal files are filtered in display operations. Disabling the filter may show more journal files but can impact performance and readability of journal information.

    Default: false

    Applies to: Source

    Display journal rows limit

    Description: Controls the maximum number of journal rows displayed in operations.

    Explanation: This setting limits the number of journal rows that are displayed in various operations and UI components. This helps prevent performance issues and overwhelming output when dealing with high-volume journal data.

    Default: 10000

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include RRN (Relative Record Number), receiver library, receiver name, sequence number, transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: _RRN, _RECEIVER_LIBRARY, _RECEIVER_NAME, _SEQUENCE_NUMBER, _TRANSACTION_ID, _TRANSACTION_TIMESTAMP, _TRANSACTION_OPERATION, _TRANSACTION_USER

    Use Dedicated Journal Reader

    Description: When enabled, uses a dedicated journal reader for this entity instead of the shared journal reader.

    Explanation: This setting controls whether the entity uses a dedicated journal reader instead of participating in the shared journal reader pool. Dedicated readers provide isolated, high-throughput pipelines for specific entities but consume more resources on the IBM i system. This is typically used for high-priority or high-volume entities that require dedicated resources.

    Default: false

    Applies to: Source

    For a step-by-step walkthrough, see the how-to guide Troubleshoot IBM i journals and receivers.

    Oracle LogMiner Source Advanced Settings

    LogMiner polling interval (seconds)

    Description: Controls how often LogMiner polls for new changes in the Oracle redo logs.

    Explanation: This setting determines the frequency at which LogMiner checks the Oracle redo logs for new changes. A lower value means more frequent polling and lower latency for change capture, but may increase CPU usage on the Oracle database. A higher value reduces CPU load but may increase the time it takes for changes to be captured.

    Default: 1 second

    Applies to: Source

    LogMiner start delay (seconds)

    Description: Controls the initial delay before LogMiner begins polling after startup.

    Explanation: This setting introduces a delay before LogMiner starts polling for changes after the agent starts up. This can be useful for allowing the Oracle database to stabilize after startup or for coordinating with other processes.

    Default: 1 second

    Applies to: Source

    LogMiner max error delay (seconds)

    Description: Controls the maximum delay between polling attempts when errors occur.

    Explanation: When LogMiner encounters errors during polling, it implements a backoff strategy. This setting controls the maximum delay between retry attempts. The actual delay increases exponentially based on the backoff factor up to this maximum.

    Default: 60 seconds

    Applies to: Source

    LogMiner backoff factor

    Description: Controls the exponential backoff factor for retry attempts when errors occur.

    Explanation: When LogMiner encounters errors, it increases the delay between retry attempts exponentially. This setting controls the multiplier for this exponential backoff. For example, with a factor of 2, delays would be 1s, 2s, 4s, 8s, etc. up to the maximum error delay.

    Default: 2

    Applies to: Source

    LogMiner batch size

    Description: Controls the number of redo log records processed in each batch.

    Explanation: This setting determines how many redo log records LogMiner processes in each batch. Larger batch sizes can improve throughput but may increase memory usage and latency for individual change processing.

    Default: 1000

    Applies to: Source

    LogMiner restart delay (seconds)

    Description: Controls the delay before restarting LogMiner after a failure or shutdown.

    Explanation: When LogMiner needs to be restarted due to a failure or planned shutdown, this setting controls how long the agent waits before attempting to restart the LogMiner process. This can help prevent rapid restart loops and allow the Oracle database to recover.

    Default: 5 seconds

    Applies to: Source

    Snapshot JDBC row prefetch size

    Description: Controls the number of rows prefetched during snapshot operations via JDBC.

    Explanation: This setting determines how many rows the JDBC driver prefetches during snapshot operations. Higher values can improve snapshot performance by reducing round trips to the database, but may increase memory usage.

    Default: 10000

    Applies to: Source

    Enable JDBC row prefetch for snapshots

    Description: When enabled, uses JDBC row prefetching during snapshot operations.

    Explanation: This setting controls whether JDBC row prefetching is enabled during snapshot operations. Prefetching can significantly improve snapshot performance by reducing the number of round trips to the database.

    Default: true

    Applies to: Source

    Snapshot batch processing multiplier

    Description: Multiplier for batch processing during snapshot operations.

    Explanation: This setting acts as a multiplier for batch processing during snapshot operations. It affects how many rows are processed in each batch during the snapshot phase, impacting both performance and memory usage.

    Default: 10

    Applies to: Source

    Enable parallel LogMiner parsing

    Description: When enabled, uses parallel processing for LogMiner parsing operations.

    Explanation: This setting controls whether LogMiner parsing operations are performed in parallel. Parallel parsing can improve performance on multi-core systems but may increase resource utilization.

    Default: true

    Applies to: Source

    LogMiner parsing parallelism

    Description: Controls the degree of parallelism for LogMiner parsing operations.

    Explanation: This setting determines how many parallel threads are used for LogMiner parsing operations when parallel parsing is enabled. Higher values can improve performance on multi-core systems but may increase CPU and memory usage.

    Default: 4

    Applies to: Source

    CSF buffer size

    Description: Controls the buffer size for the Committed Sparse File (CSF) used for transaction spill.

    Explanation: This setting controls the size of the buffer for the Committed Sparse File, which is used for spilling uncommitted transactions. Larger buffers can handle larger transactions but may increase disk space usage.

    Default: 1000

    Applies to: Source

    Enable Flow-based LogMiner parsing

    Description: When enabled, uses flow-based parsing for LogMiner operations.

    Explanation: This setting controls whether flow-based parsing is used for LogMiner operations. Flow-based parsing can improve performance and resource utilization for certain workloads.

    Default: true

    Applies to: Source

    Skip transactions with LogMiner parse errors

    Description: When enabled, skips transactions that encounter parse errors instead of failing.

    Explanation: This setting controls how the agent handles transactions that encounter parse errors during LogMiner processing. When enabled, transactions with parse errors are skipped and logged, allowing the agent to continue processing other transactions. When disabled, parse errors cause the agent to fail.

    Default: true

    Applies to: Source

    Reconstruct full row on sparse UPDATE

    Description: When enabled, reconstructs the full row on sparse UPDATE operations when minimal supplemental logging is used.

    Explanation: When LogMiner runs under minimal supplemental logging, the SQL_REDO of an UPDATE only contains the changed columns (plus the primary key). Unchanged columns are therefore absent and would be replicated as NULL, corrupting the target row. When enabled, the converter detects such sparse UPDATE images and re-reads the complete row from the source by ROWID so the full after-image is replicated. Disable only when the source tables already have full supplemental logging (ALTER TABLE …​ ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS), in which case the extra per-update SELECT is unnecessary.

    Default: true

    Applies to: Source

    Use Oracle RAC

    Description: When enabled, configures the agent for Oracle Real Application Clusters (RAC) environments.

    Explanation: This setting enables Oracle RAC-specific configurations and behaviors. In RAC environments, additional considerations are needed for handling multiple instances, redo logs, and coordination between nodes.

    Default: false

    Applies to: Source

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include ROWID, transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: ROWID, TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    LogMiner username

    Description: Specifies a dedicated username for the LogMiner connection, separate from the main Oracle connection.

    Explanation: When the LogMiner process uses a different database user than the main Oracle connection, this setting provides that username. This is useful when the LogMiner user has different privileges (e.g., LOGMINING role) than the replication user. Leave empty to use the same credentials as the main connection.

    Default: (empty)

    Applies to: Source

    LogMiner password

    Description: Specifies the password for the dedicated LogMiner username.

    Explanation: This setting provides the password for the LogMiner-specific connection when a dedicated LogMiner username is configured. Leave empty to use the same credentials as the main connection.

    Default: (empty)

    Applies to: Source

    LogMiner database

    Description: Specifies a dedicated database (service name) for the LogMiner connection.

    Explanation: When the LogMiner process connects to a different database or service than the main Oracle connection, this setting provides that database name. Leave empty to use the same database as the main connection.

    Default: (empty)

    Applies to: Source

    LogMiner connection timeout

    Description: Controls the connection timeout for the LogMiner connection in milliseconds.

    Explanation: This setting determines how long the agent waits when establishing the LogMiner-specific connection before timing out. The default of 10000 ms (10 seconds) is suitable for most network conditions.

    Default: 10000 ms (10 seconds)

    Applies to: Source

    LogMiner read timeout (ms)

    Description: Controls the read timeout for LogMiner operations in milliseconds.

    Explanation: This setting determines how long the agent waits for LogMiner read operations to complete before timing out. The default of 60000 ms (60 seconds) provides ample time for reading redo log data.

    Default: 60000 ms (60 seconds)

    Applies to: Source

    For a step-by-step walkthrough, see the how-to guide Set up Oracle LogMiner CDC end to end.

    PostgreSQL Source Advanced Settings

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    WAL sender timeout in seconds (supported from PostgreSQL 12+)

    Description: Controls the timeout for WAL sender operations in seconds.

    Explanation: This setting determines how long the agent waits for WAL sender operations to complete before timing out. This setting is only supported from PostgreSQL version 12 and above. A higher value provides more time for complex operations but may delay error detection.

    Default: 60 seconds

    Applies to: Source

    MySQL Source Advanced Settings

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    SQL Server Source Advanced Settings

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Max Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent operations (0 means no limitation).

    Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

    Default: 0 (no limitation)

    Applies to: Source

    Max Snapshot Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

    Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

    Default: 0 (no limitation)

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Disable automatic database change data capture activation

    Description: When enabled, prevents the agent from automatically enabling CDC on the source database.

    Explanation: By default, the agent can automatically enable Change Data Capture on the source database if it’s not already enabled. This setting disables that automatic activation, requiring CDC to be manually configured.

    Default: false

    Applies to: Source

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION, TRANSACTION_USER

    MariaDB Source Advanced Settings

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    DB2 LUW Source Advanced Settings

    The DB2 LUW agent has two variants: CDC (using the GS_SP_LOGMINER stored procedure) and Triggers (using database triggers). The settings below marked with (CDC only) are available only in the CDC variant. The Triggers variant supports a subset of these settings (Enable Recursion Protection, Trim text values, Technical fields) plus its own connection settings described in the target settings document.

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Stored procedure polling interval in seconds (CDC only)

    Description: Controls how often the agent polls the stored procedure for change data capture.

    Explanation: This setting determines the frequency at which the agent calls the DB2 stored procedure for CDC operations. A lower value means more frequent polling and lower latency for change capture, but may increase CPU usage on the DB2 server.

    Default: 10 seconds

    Applies to: Source

    Auto-create GS_SP_LOGMINER stored procedure if not found (CDC only)

    Description: When enabled, automatically creates the GS_SP_LOGMINER stored procedure if it doesn’t exist.

    Explanation: This setting controls whether the agent automatically creates the required GS_SP_LOGMINER stored procedure on the DB2 server if it doesn’t already exist. This simplifies setup but requires appropriate permissions.

    Default: true

    Applies to: Source

    Stored procedure library path (CDC only)

    Description: Specifies the library path for the stored procedure used for CDC operations.

    Explanation: This setting specifies the file system path to the library containing the stored procedure used for CDC operations. The default path is typically appropriate for most installations.

    Default: /function/libdb2.so

    Applies to: Source

    Max buffered events per source transaction (0 = library default) (CDC only)

    Description: Controls the maximum number of DML events buffered by GS_SP_LOGMINER for a single source transaction before it spills to staging tables.

    Explanation: This setting controls the maximum number of DML events that the GS_SP_LOGMINER stored procedure buffers for a single source transaction before spilling to staging tables. A value of 0 keeps the library’s built-in default. When set to a value greater than 0, it is passed through the extended P_MAX_EVENTS syntax ("<max_events>;max_tx_events=<n>"), which requires a log-reader library recent enough to understand it.

    Default: 0 (library default)

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    Oracle XStream Source Advanced Settings

    Xstream server name

    Description: Specifies the XStream outbound server name to connect to on the Oracle database.

    Explanation: This setting defines the name of the XStream outbound server that the agent connects to for receiving change data. The outbound server must be configured on the Oracle database. The default GSXOUT is the standard Gluesync XStream server name, but you can change it to match a custom server name if your Oracle database uses a different configuration.

    Default: GSXOUT

    Applies to: Source

    Xstream Low Watermark Retention in Hours

    Description: Controls how long the XStream low watermark is retained in hours.

    Explanation: This setting determines the retention period for the XStream low watermark, which is used for tracking the position in the change stream. A longer retention period provides better recovery capabilities but may require more storage.

    Default: 24 hours

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION, TRANSACTION_USER

    Oracle Triggers Source Advanced Settings

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Enable JDBC row prefetch for snapshots

    Description: When enabled, uses JDBC row prefetching during snapshot operations.

    Explanation: This setting controls whether JDBC row prefetching is enabled during snapshot operations. Prefetching can significantly improve snapshot performance by reducing the number of round trips to the database.

    Default: true

    Applies to: Source

    Snapshot batch processing multiplier

    Description: Multiplier for batch processing during snapshot operations.

    Explanation: This setting acts as a multiplier for batch processing during snapshot operations. It affects how many rows are processed in each batch during the snapshot phase, impacting both performance and memory usage.

    Default: 10

    Applies to: Source

    CDC batch processing multiplier

    Description: Multiplier for batch processing during CDC operations.

    Explanation: This setting acts as a multiplier for batch processing during CDC operations. It affects how many changes are processed in each batch during the CDC phase, impacting both performance and memory usage.

    Default: 20

    Applies to: Source

    Enable CDC direct-path reads

    Description: When enabled, uses direct-path reads for CDC table operations.

    Explanation: This setting controls whether direct-path reads are used for reading from the CDC table. Direct-path reads can improve performance for large data reads but may have different consistency characteristics.

    Default: true

    Applies to: Source

    Use partitions for CDC table

    Description: When enabled, uses partitioning for the CDC table.

    Explanation: This setting controls whether the CDC table uses partitioning. Partitioning can improve performance and manageability for large CDC tables but may add complexity to the setup.

    Default: false

    Applies to: Source

    Read CDC table data fetch size

    Description: Controls the fetch size for reading data from the CDC table.

    Explanation: This setting determines how many rows are fetched in each operation when reading from the CDC table. Higher values can improve performance but may increase memory usage.

    Default: 10000

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION, TRANSACTION_USER

    Informix Source Advanced Settings

    Max Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent operations (0 means no limitation).

    Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

    Default: 0 (no limitation)

    Applies to: Source

    Max Snapshot Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

    Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

    Default: 0 (no limitation)

    Applies to: Source

    CDC client timeout

    Description: Controls the timeout for CDC client operations in seconds.

    Explanation: This setting determines how long the CDC client waits for operations to complete before timing out. A higher value provides more time for complex operations but may delay error detection.

    Default: 10 seconds

    Applies to: Source

    Logical storage space name (sbspace)

    Description: Specifies the logical storage space name (sbspace) for CDC operations.

    Explanation: This setting specifies the name of the logical storage space (sbspace) used for CDC operations. The sbspace is a special storage area in Informix used for CDC data.

    Default: sbspace

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    Sybase Source Advanced Settings

    Max Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent operations (0 means no limitation).

    Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

    Default: 0 (no limitation)

    Applies to: Source

    Max Snapshot Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

    Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

    Default: 0 (no limitation)

    Applies to: Source

    Trim text values

    Description: When enabled, automatically trims whitespace from text values during data replication.

    Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

    Default: false

    Applies to: Source

    Disable automatic triggers creation

    Description: When enabled, prevents the agent from automatically creating triggers on the source database.

    Explanation: By default, the agent can automatically create the necessary triggers on the source database for CDC operations. This setting disables that automatic creation, requiring triggers to be manually configured.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION, TRANSACTION_USER

    SAP SQL Anywhere Source Advanced Settings

    Log polling interval (logPollingIntervalMilliseconds)

    Description: How often the transaction log is checked for new commits, in milliseconds.

    Explanation: A check that finds no new commit is a cheap query, and the log is translated with dbtran only when there is something to read. Lower values reduce the latency of a change; the minimum is 100. Each run of dbtran reads the whole active log, so keep the active log small. See Transaction log size and renames.

    Default: 1000

    Applies to: Source

    Transaction log translation timeout (dbtranTimeoutSeconds)

    Description: Time, in seconds, allowed to one run of dbtran.

    Explanation: A run that takes longer is stopped and reported as an error, and the agent tries again with a growing delay. Raise it when the active log is very large.

    Default: 900

    Applies to: Source

    Log rename gap tolerance (logRenameGapToleranceBytes)

    Description: Bytes of the transaction log that a rename may skip without the agent reporting a gap.

    Explanation: After BACKUP …​ TRANSACTION LOG RENAME the new log starts a few bytes above where a caught-up agent stopped. Up to this distance is taken as no loss. 0 is strict. A larger distance, with the renamed logs unreadable, is a gap that every CDC entity reports.

    Default: 64

    Applies to: Source

    Excluded users (excludedUsers)

    Description: Comma-separated list of database users whose changes are not replicated.

    Explanation: Passed to dbtran, which leaves out every transaction of those users. It is the loop prevention for bidirectional setups: the user that another Gluesync pipeline writes with is listed here.

    Default: (empty)

    Applies to: Source

    Max Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent operations (0 means no limitation).

    Default: 0 (no limitation)

    Applies to: Source

    Max Snapshot Concurrency (0 means no limitation)

    Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

    Default: 0 (no limitation)

    Applies to: Source

    Trim text values

    Description: When enabled, trims whitespace from CHAR, VARCHAR, NCHAR and NVARCHAR values.

    Default: false

    Applies to: Source

    MongoDB Source Advanced Settings

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    Redis Source Advanced Settings

    Topic Prefix

    Description: Specifies a prefix for topic names used in Redis CDC operations.

    Explanation: This setting allows you to specify a prefix that will be applied to topic names used during Redis CDC operations. This can help organize and distinguish topics in complex environments.

    Default: (empty)

    Applies to: Source

    Use Transactions

    Description: When enabled, uses transactions for Redis operations.

    Explanation: This setting controls whether Redis operations are performed within transactions. Using transactions can improve data consistency but may impact performance.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Cassandra Source Advanced Settings

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Snowflake Source Advanced Settings

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Couchbase Source Advanced Settings

    Use Scope and Collections

    Description: When enabled, uses Couchbase scopes and collections for data organization.

    Explanation: This setting controls whether the agent uses Couchbase’s scopes and collections feature for organizing data. Scopes and collections provide a way to logically group data within a bucket.

    Default: true

    Applies to: Source

    Index Replica Count

    Description: Controls the number of replicas for indexes created by the agent.

    Explanation: This setting determines how many replicas are created for indexes that the agent creates on Couchbase. More replicas provide better fault tolerance but require more storage and resources.

    Default: 0

    Applies to: Source

    Indexes Build Timeout Seconds

    Description: Controls the timeout for index build operations in seconds.

    Explanation: This setting determines how long the agent waits for index build operations to complete before timing out. Higher values provide more time for complex index builds but may delay error detection.

    Default: 30 seconds

    Applies to: Source

    Eventing Workers Number

    Description: Controls the number of workers for Couchbase eventing operations.

    Explanation: This setting determines how many worker threads are used for Couchbase eventing operations. More workers can improve throughput but may increase resource usage.

    Default: 3

    Applies to: Source

    Eventing Script Timeout Seconds

    Description: Controls the timeout for eventing script execution in seconds.

    Explanation: This setting determines how long eventing scripts are allowed to run before timing out. Higher values provide more time for complex scripts but may delay error detection.

    Default: 60 seconds

    Applies to: Source

    Eventing Background Polling Interval Seconds

    Description: Controls the polling interval for background eventing operations in seconds.

    Explanation: This setting determines how often the agent polls for background eventing operations. Lower values provide more responsive eventing but may increase resource usage.

    Default: 30 seconds

    Applies to: Source

    Eventing Language Compatibility

    Description: Controls the language compatibility version for Couchbase eventing.

    Explanation: This setting specifies the language compatibility version for Couchbase eventing scripts. Different versions may have different features and behaviors.

    Default: 6.6.2

    Applies to: Source

    Eventing Disable Automatic Function Configuration

    Description: When enabled, disables automatic configuration of Couchbase eventing functions.

    Explanation: This setting controls whether the agent automatically configures Couchbase eventing functions. When disabled, eventing functions must be manually configured.

    Default: false

    Applies to: Source

    Delete Events On Sync Confirmed

    Description: When enabled, deletes events after they have been confirmed as synchronized.

    Explanation: This setting controls whether events are automatically deleted after they have been successfully synchronized to the target. This can help manage storage usage but may impact debugging and recovery capabilities.

    Default: true

    Applies to: Source

    Enable Recursion Protection

    Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

    Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

    Default: false

    Applies to: Source

    Custom 'type' field name

    Description: Specifies a custom field name for the 'type' field used in Couchbase document type identification.

    Explanation: This setting allows you to specify a custom field name to be used for document type identification instead of the default 'type' field. This can be useful when the default field name conflicts with existing document structure.

    Default: (empty)

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    Kafka Source Advanced Settings

    Topic Prefix

    Description: Specifies a prefix for topic names used in Kafka CDC operations.

    Explanation: This setting allows you to specify a prefix that will be applied to topic names used during Kafka CDC operations. This can help organize and distinguish topics in complex environments.

    Default: (empty)

    Applies to: Source

    Use Transactions

    Description: When enabled, uses transactions for Kafka operations.

    Explanation: This setting controls whether Kafka operations are performed within transactions. Using transactions can improve data consistency but may impact performance.

    Default: false

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    YugabyteDB Source Advanced Settings

    Topology keys (ex. cloud1.datacenter1.rack1:1,cloud1.datacenter1.rack2:2)

    Description: Specifies topology keys for data placement in YugabyteDB clusters.

    Explanation: This setting allows you to specify topology keys that control data placement across the YugabyteDB cluster. The format is typically "cloud1.datacenter1.rack1:1,cloud1.datacenter1.rack2:2" where the numbers represent replica counts. This is useful for controlling data distribution and fault tolerance in distributed deployments.

    Default: (empty)

    Applies to: Source

    Query parallelism

    Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

    Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

    Default: 4

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    WAL sender timeout in seconds (supported from db version >= 12)

    Description: Controls the timeout for WAL sender operations in seconds.

    Explanation: This setting determines how long the agent waits for WAL sender operations to complete before timing out. This setting is only supported from YugabyteDB version 12 and above. A higher value provides more time for complex operations but may delay error detection.

    Default: 60 seconds

    Applies to: Source

    Vertica Source Advanced Settings

    Snapshot read parallelism (snapshotReadParallelism)

    Description: Number of parallel buckets, and connections, a table is read with during the snapshot (1 disables the parallel read).

    Explanation: Each bucket reads the rows where MOD(HASH(keys), N) = i at the epoch the snapshot is pinned to. Every bucket needs a session of the database: keep MaxClientSessions above the sessions all the pipelines use. See Sessions.

    Default: 4

    Applies to: Source

    JDBC result buffer size in bytes (resultBufferSizeBytes)

    Description: Size of the result buffer of the Vertica JDBC driver (ResultBufferSize).

    Explanation: A larger buffer reads faster at the cost of memory for each connection.

    Default: 8388608 (8 MiB)

    Applies to: Source

    CDC poll interval in milliseconds (cdcPollIntervalMilliseconds)

    Description: Minimum time between two CDC polls of an entity.

    Explanation: Each poll runs a few historical queries whose cost grows with the size of the table. Raise it for very large tables or when the latency of a change is not critical. See Limitations.

    Default: 1000

    Applies to: Source

    Delete detection bucket size (deleteDetectionBucketSize)

    Description: Number of keys held in memory at once when the agent works out which rows were deleted.

    Explanation: A window that deletes more rows is split in as many buckets as needed, and every bucket is an extra read of the table.

    Default: 1000000

    Applies to: Source

    Reload the table when its history is gone (resyncOnHistoryLoss)

    Description: What the agent does when the history it needs is gone.

    Explanation: A TRUNCATE, DROP_PARTITIONS, a purge, or an Ancient History Mark that moved past the position of the pipeline removes the history CDC compares. When enabled, the agent reloads the table on the target (a TRUNCATE and every row as an insert); when disabled, the entity fails. See When the history is lost.

    Default: true

    Applies to: Source

    Minimum history retention in seconds (minHistoryRetentionSeconds)

    Description: The HistoryRetentionTime that the setup check of a CDC entity requires.

    Explanation: The setup fails when HistoryRetentionTime of the database is lower than this value (the default of 0 included), with the ALTER DATABASE statement that fixes it. See Setup checks.

    Default: 3600

    Applies to: Source

    Technical fields

    Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

    Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. All the rows of a poll window carry the same transaction ID and timestamp.

    Default: (empty)

    Applies to: Source

    Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

    Apache CouchDB Source Advanced Settings

    Schema discovery sample size (schemaDiscoverySampleSize)

    Description: Number of documents sampled to infer the fields of a database.

    Explanation: Half of the sample comes from the start of _all_docs and half from its end, with the design documents left out. A field that only appears in documents outside the sample is not discovered, so raise it for a database whose documents have different shapes. See Databases, entities and fields.

    Default: 1000

    Applies to: Source

    Snapshot read concurrency (snapshotReadConcurrency)

    Description: Parallel _bulk_get requests of a snapshot.

    Explanation: The main lever of the snapshot speed: 22,000 documents per second with 1 request at a time and 94,000 with 16, measured on a single node. The number of shards of the source database sets the ceiling. A snapshot needs about 3 x Snapshot read concurrency x Snapshot fetch batch size documents in memory.

    Default: 8

    Applies to: Source

    Snapshot fetch batch size (snapshotFetchBatchSize)

    Description: Number of document ids in one _bulk_get request.

    Explanation: Lower it for very large documents, to keep the memory of a snapshot down.

    Default: 1000

    Applies to: Source

    Snapshot page size (snapshotPageSize)

    Description: Number of rows of one _changes page that lists the ids of a snapshot.

    Explanation: The ids are listed from the changes feed, ids only, with seq_interval set to the page size.

    Default: 10000

    Applies to: Source

    Changes batch size (changesBatchSize)

    Description: Number of rows of one _changes page of the change feed.

    Explanation: A poll keeps reading while the pages come back full, up to ten pages, so a backlog is not read at one page per polling interval.

    Default: 5000

    Applies to: Source

    Changes long poll timeout (changesLongPollTimeoutMilliseconds)

    Description: How long a poll of the change feed waits for a change before it answers empty, in milliseconds.

    Explanation: A change that arrives during the wait is delivered at once, so this is not a latency floor.

    Default: 1000

    Applies to: Source

    Connect timeout and Request timeout (connectTimeoutMilliseconds, requestTimeoutMilliseconds)

    Description: How long a node may take to accept a connection, and how long a request may run, in milliseconds.

    Explanation: Custom host credentials of the connection. A node that does not accept a connection in time is skipped for the next one. The request timeout covers long polls and big requests.

    Default: 10000 and 300000

    Applies to: Source and Target