Source Agent Advanced Settings

This document provides detailed information about all advanced settings available for Gluesync source agents. These settings control change data capture (CDC) behavior, performance tuning, connection parameters, and other agent-specific configurations.

Common Source Settings

These settings are available across most source agents and control fundamental CDC behavior.

Source change retention in hours

Description: Controls how long change data is retained in the source agent’s local ArenaCache before being discarded.

Explanation: This setting is the purge clock for ArenaCache, the disk-backed source cache that became the platform default in 2.2.11.2 (replacing Chronicle Queue). Changes older than this period are evicted from the agent’s local files. A higher value gives a longer window if Core Hub or the target is unavailable; a lower value keeps disk use down. The default of 24 hours (1 day) is suitable for most use cases. This setting does not move the CDC reader checkpoint — that lives in Core Hub for cache-based readers as of 2.2.11. See Source-side cache.

Default: 24 (1 day)

Applies to: Source

IBM i (AS400) Source Advanced Settings

Enable multi-member physical file support

Description: Enables discovery and replication of data from all members within a multi-member physical file, treating each member as its own entity with independent CDC tracking.

Explanation: IBM i physical files can contain multiple members (similar to tables within a table). When enabled, this setting allows Gluesync to discover and replicate data from all members within a multi-member physical file. Each member is treated as a separate entity with its own CDC tracking via journal APIs, snapshots, and bulk loads. This is particularly useful for IBM i systems that use multi-member files for data partitioning or organization.

Default: false

Applies to: Source

Important: This setting is disabled by default. If you are already using the multi-member feature, you must enable this flag in the agent configuration under advanced settings.

Journal reader polling interval in milliseconds

Description: Controls how long the agent waits between journal API calls when the journal is completely drained (no new records available).

Explanation: This setting manages the polling frequency for journal reading. When the IBM i journal returns no new records (indicating it’s been completely drained), the agent will wait for the specified number of milliseconds before making the next API call. If the journal does return records, the agent ignores this interval and immediately makes the next call to catch up with the journal head. Higher values reduce CPU usage on the IBM i system during idle periods but may increase latency for capturing new changes.

Default: 10000 (10 seconds)

Applies to: Source

Journal reader API buffer size in KB (default 16384)

Description: Sets the size of the receiver buffer used for each journal API call, controlling how much data can be retrieved in a single round trip.

Explanation: Each call to the IBM i journal API writes into a contiguous receiver buffer whose size is governed by this parameter. The buffer size affects how many journal entries can be returned in a single API call. Regardless of the configured value, each individual call is hard-capped at 10 MB (the maximum receiver size Gluesync requests from the IBM i API in one round trip). When the buffer fills before the journal is drained, the API flags that more entries are available and Gluesync immediately issues a continuation call.

Default: 16384 (16 MB)

Applies to: Source

Enable continuous journal reading (skip delay to next iteration)

Description: When enabled, skips the delay between journal reading iterations for continuous streaming.

Explanation: By default, the agent may introduce delays between journal reading iterations to manage resource utilization. When this setting is enabled, the agent skips these delays and performs continuous journal reading, which can reduce latency but may increase CPU usage on the IBM i system.

Default: false

Applies to: Source

Journal reader page size (0 means disabled)

Description: Controls the page size for journal reader operations (0 means disabled).

Explanation: This setting controls the pagination of journal reader operations. When set to a non-zero value, it defines the number of journal entries processed per page. A value of 0 disables pagination, meaning all available entries are processed in a single operation.

Default: 1000

Applies to: Source

Journal reader fetch size (0 means default)

Description: Controls the fetch size for journal reader operations (0 means default).

Explanation: This setting controls how many journal entries are fetched in each operation. A value of 0 uses the system default fetch size. Adjusting this can affect memory usage and performance of journal reading operations.

Default: 1000

Applies to: Source

Legacy mode journal reader (use DSPJRN command)

Description: When enabled, uses the legacy DSPJRN command instead of native Journal APIs for journal reading.

Explanation: This setting enables the legacy journal reading mode that uses the DSPJRN CL command instead of the modern native Journal APIs (QjoRetrieveJournalEntries). The legacy mode may be required for compatibility with older IBM i systems or specific configurations, but generally the native APIs provide better performance and functionality.

Default: false

Applies to: Source

Query parallelism

Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

Default: 4

Applies to: Source

Metadata setup concurrency (parallel QDBRTVFD calls)

Description: Controls the number of parallel QDBRTVFD calls during metadata setup for table and column information.

Explanation: This setting controls how many parallel calls are made to the QDBRTVFD API during the metadata setup phase when discovering table and column information. Higher values can speed up the initial setup process for entities with many tables but may increase load on the IBM i system during setup.

Default: 10

Applies to: Source

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to parse the user metadata block of incoming journal entries and silently discard any operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Max Concurrency (0 means no limitation)

Description: Controls the maximum number of concurrent operations (0 means no limitation).

Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

Default: 0 (no limitation)

Applies to: Source

Max Snapshot Concurrency (0 means no limitation)

Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

Default: 0 (no limitation)

Applies to: Source

Trim text values

Description: When enabled, automatically trims whitespace from text values during data replication.

Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

Default: false

Applies to: Source

Disable journal reader

Description: When enabled, disables the journal reader component for CDC operations.

Explanation: This setting completely disables the journal reader, preventing the agent from performing CDC operations. This is typically used for snapshot-only configurations or when troubleshooting CDC issues.

Default: false

Applies to: Source

Disable display journal files filter

Description: When enabled, disables filtering of journal files in display operations.

Explanation: This setting controls whether journal files are filtered in display operations. Disabling the filter may show more journal files but can impact performance and readability of journal information.

Default: false

Applies to: Source

Display journal rows limit

Description: Controls the maximum number of journal rows displayed in operations.

Explanation: This setting limits the number of journal rows that are displayed in various operations and UI components. This helps prevent performance issues and overwhelming output when dealing with high-volume journal data.

Default: 10000

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include RRN (Relative Record Number), receiver library, receiver name, sequence number, transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: _RRN, _RECEIVER_LIBRARY, _RECEIVER_NAME, _SEQUENCE_NUMBER, _TRANSACTION_ID, _TRANSACTION_TIMESTAMP, _TRANSACTION_OPERATION, _TRANSACTION_USER

Use Dedicated Journal Reader

Description: When enabled, uses a dedicated journal reader for this entity instead of the shared journal reader.

Explanation: This setting controls whether the entity uses a dedicated journal reader instead of participating in the shared journal reader pool. Dedicated readers provide isolated, high-throughput pipelines for specific entities but consume more resources on the IBM i system. This is typically used for high-priority or high-volume entities that require dedicated resources.

Default: false

Applies to: Source

Oracle LogMiner Source Advanced Settings

LogMiner polling interval (seconds)

Description: Controls how often LogMiner polls for new changes in the Oracle redo logs.

Explanation: This setting determines the frequency at which LogMiner checks the Oracle redo logs for new changes. A lower value means more frequent polling and lower latency for change capture, but may increase CPU usage on the Oracle database. A higher value reduces CPU load but may increase the time it takes for changes to be captured.

Default: 1 second

Applies to: Source

LogMiner start delay (seconds)

Description: Controls the initial delay before LogMiner begins polling after startup.

Explanation: This setting introduces a delay before LogMiner starts polling for changes after the agent starts up. This can be useful for allowing the Oracle database to stabilize after startup or for coordinating with other processes.

Default: 1 second

Applies to: Source

LogMiner max error delay (seconds)

Description: Controls the maximum delay between polling attempts when errors occur.

Explanation: When LogMiner encounters errors during polling, it implements a backoff strategy. This setting controls the maximum delay between retry attempts. The actual delay increases exponentially based on the backoff factor up to this maximum.

Default: 60 seconds

Applies to: Source

LogMiner backoff factor

Description: Controls the exponential backoff factor for retry attempts when errors occur.

Explanation: When LogMiner encounters errors, it increases the delay between retry attempts exponentially. This setting controls the multiplier for this exponential backoff. For example, with a factor of 2, delays would be 1s, 2s, 4s, 8s, etc. up to the maximum error delay.

Default: 2

Applies to: Source

LogMiner batch size

Description: Controls the number of redo log records processed in each batch.

Explanation: This setting determines how many redo log records LogMiner processes in each batch. Larger batch sizes can improve throughput but may increase memory usage and latency for individual change processing.

Default: 1000

Applies to: Source

LogMiner restart delay (seconds)

Description: Controls the delay before restarting LogMiner after a failure or shutdown.

Explanation: When LogMiner needs to be restarted due to a failure or planned shutdown, this setting controls how long the agent waits before attempting to restart the LogMiner process. This can help prevent rapid restart loops and allow the Oracle database to recover.

Default: 5 seconds

Applies to: Source

Snapshot JDBC row prefetch size

Description: Controls the number of rows prefetched during snapshot operations via JDBC.

Explanation: This setting determines how many rows the JDBC driver prefetches during snapshot operations. Higher values can improve snapshot performance by reducing round trips to the database, but may increase memory usage.

Default: 10000

Applies to: Source

Enable JDBC row prefetch for snapshots

Description: When enabled, uses JDBC row prefetching during snapshot operations.

Explanation: This setting controls whether JDBC row prefetching is enabled during snapshot operations. Prefetching can significantly improve snapshot performance by reducing the number of round trips to the database.

Default: true

Applies to: Source

Snapshot batch processing multiplier

Description: Multiplier for batch processing during snapshot operations.

Explanation: This setting acts as a multiplier for batch processing during snapshot operations. It affects how many rows are processed in each batch during the snapshot phase, impacting both performance and memory usage.

Default: 10

Applies to: Source

Enable parallel LogMiner parsing

Description: When enabled, uses parallel processing for LogMiner parsing operations.

Explanation: This setting controls whether LogMiner parsing operations are performed in parallel. Parallel parsing can improve performance on multi-core systems but may increase resource utilization.

Default: true

Applies to: Source

LogMiner parsing parallelism

Description: Controls the degree of parallelism for LogMiner parsing operations.

Explanation: This setting determines how many parallel threads are used for LogMiner parsing operations when parallel parsing is enabled. Higher values can improve performance on multi-core systems but may increase CPU and memory usage.

Default: 4

Applies to: Source

CSF buffer size

Description: Controls the buffer size for the Committed Sparse File (CSF) used for transaction spill.

Explanation: This setting controls the size of the buffer for the Committed Sparse File, which is used for spilling uncommitted transactions. Larger buffers can handle larger transactions but may increase disk space usage.

Default: 1000

Applies to: Source

Enable Flow-based LogMiner parsing

Description: When enabled, uses flow-based parsing for LogMiner operations.

Explanation: This setting controls whether flow-based parsing is used for LogMiner operations. Flow-based parsing can improve performance and resource utilization for certain workloads.

Default: true

Applies to: Source

Skip transactions with LogMiner parse errors

Description: When enabled, skips transactions that encounter parse errors instead of failing.

Explanation: This setting controls how the agent handles transactions that encounter parse errors during LogMiner processing. When enabled, transactions with parse errors are skipped and logged, allowing the agent to continue processing other transactions. When disabled, parse errors cause the agent to fail.

Default: true

Applies to: Source

Reconstruct full row on sparse UPDATE

Description: When enabled, reconstructs the full row on sparse UPDATE operations when minimal supplemental logging is used.

Explanation: When LogMiner runs under minimal supplemental logging, the SQL_REDO of an UPDATE only contains the changed columns (plus the primary key). Unchanged columns are therefore absent and would be replicated as NULL, corrupting the target row. When enabled, the converter detects such sparse UPDATE images and re-reads the complete row from the source by ROWID so the full after-image is replicated. Disable only when the source tables already have full supplemental logging (ALTER TABLE …​ ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS), in which case the extra per-update SELECT is unnecessary.

Default: true

Applies to: Source

Use Oracle RAC

Description: When enabled, configures the agent for Oracle Real Application Clusters (RAC) environments.

Explanation: This setting enables Oracle RAC-specific configurations and behaviors. In RAC environments, additional considerations are needed for handling multiple instances, redo logs, and coordination between nodes.

Default: false

Applies to: Source

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Query parallelism

Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

Default: 4

Applies to: Source

Trim text values

Description: When enabled, automatically trims whitespace from text values during data replication.

Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include ROWID, transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: ROWID, TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

LogMiner username

Description: Specifies a dedicated username for the LogMiner connection, separate from the main Oracle connection.

Explanation: When the LogMiner process uses a different database user than the main Oracle connection, this setting provides that username. This is useful when the LogMiner user has different privileges (e.g., LOGMINING role) than the replication user. Leave empty to use the same credentials as the main connection.

Default: (empty)

Applies to: Source

LogMiner password

Description: Specifies the password for the dedicated LogMiner username.

Explanation: This setting provides the password for the LogMiner-specific connection when a dedicated LogMiner username is configured. Leave empty to use the same credentials as the main connection.

Default: (empty)

Applies to: Source

LogMiner database

Description: Specifies a dedicated database (service name) for the LogMiner connection.

Explanation: When the LogMiner process connects to a different database or service than the main Oracle connection, this setting provides that database name. Leave empty to use the same database as the main connection.

Default: (empty)

Applies to: Source

LogMiner connection timeout

Description: Controls the connection timeout for the LogMiner connection in milliseconds.

Explanation: This setting determines how long the agent waits when establishing the LogMiner-specific connection before timing out. The default of 10000 ms (10 seconds) is suitable for most network conditions.

Default: 10000 ms (10 seconds)

Applies to: Source

LogMiner read timeout (ms)

Description: Controls the read timeout for LogMiner operations in milliseconds.

Explanation: This setting determines how long the agent waits for LogMiner read operations to complete before timing out. The default of 60000 ms (60 seconds) provides ample time for reading redo log data.

Default: 60000 ms (60 seconds)

Applies to: Source

PostgreSQL Source Advanced Settings

Query parallelism

Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

Default: 4

Applies to: Source

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

WAL sender timeout in seconds (supported from PostgreSQL 12+)

Description: Controls the timeout for WAL sender operations in seconds.

Explanation: This setting determines how long the agent waits for WAL sender operations to complete before timing out. This setting is only supported from PostgreSQL version 12 and above. A higher value provides more time for complex operations but may delay error detection.

Default: 60 seconds

Applies to: Source

MySQL Source Advanced Settings

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

SQL Server Source Advanced Settings

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Max Concurrency (0 means no limitation)

Description: Controls the maximum number of concurrent operations (0 means no limitation).

Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

Default: 0 (no limitation)

Applies to: Source

Max Snapshot Concurrency (0 means no limitation)

Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

Default: 0 (no limitation)

Applies to: Source

Trim text values

Description: When enabled, automatically trims whitespace from text values during data replication.

Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

Default: false

Applies to: Source

Disable automatic database change data capture activation

Description: When enabled, prevents the agent from automatically enabling CDC on the source database.

Explanation: By default, the agent can automatically enable Change Data Capture on the source database if it’s not already enabled. This setting disables that automatic activation, requiring CDC to be manually configured.

Default: false

Applies to: Source

Query parallelism

Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

Default: 4

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION, TRANSACTION_USER

MariaDB Source Advanced Settings

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

DB2 LUW Source Advanced Settings

The DB2 LUW agent has two variants: CDC (using the GS_SP_LOGMINER stored procedure) and Triggers (using database triggers). The settings below marked with (CDC only) are available only in the CDC variant. The Triggers variant supports a subset of these settings (Enable Recursion Protection, Trim text values, Technical fields) plus its own connection settings described in the target settings document.

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Stored procedure polling interval in seconds (CDC only)

Description: Controls how often the agent polls the stored procedure for change data capture.

Explanation: This setting determines the frequency at which the agent calls the DB2 stored procedure for CDC operations. A lower value means more frequent polling and lower latency for change capture, but may increase CPU usage on the DB2 server.

Default: 10 seconds

Applies to: Source

Auto-create GS_SP_LOGMINER stored procedure if not found (CDC only)

Description: When enabled, automatically creates the GS_SP_LOGMINER stored procedure if it doesn’t exist.

Explanation: This setting controls whether the agent automatically creates the required GS_SP_LOGMINER stored procedure on the DB2 server if it doesn’t already exist. This simplifies setup but requires appropriate permissions.

Default: true

Applies to: Source

Stored procedure library path (CDC only)

Description: Specifies the library path for the stored procedure used for CDC operations.

Explanation: This setting specifies the file system path to the library containing the stored procedure used for CDC operations. The default path is typically appropriate for most installations.

Default: /function/libdb2.so

Applies to: Source

Max buffered events per source transaction (0 = library default) (CDC only)

Description: Controls the maximum number of DML events buffered by GS_SP_LOGMINER for a single source transaction before it spills to staging tables.

Explanation: This setting controls the maximum number of DML events that the GS_SP_LOGMINER stored procedure buffers for a single source transaction before spilling to staging tables. A value of 0 keeps the library’s built-in default. When set to a value greater than 0, it is passed through the extended P_MAX_EVENTS syntax ("<max_events>;max_tx_events=<n>"), which requires a log-reader library recent enough to understand it.

Default: 0 (library default)

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

Oracle XStream Source Advanced Settings

Xstream server name

Description: Specifies the XStream outbound server name to connect to on the Oracle database.

Explanation: This setting defines the name of the XStream outbound server that the agent connects to for receiving change data. The outbound server must be configured on the Oracle database. The default GSXOUT is the standard Gluesync XStream server name, but you can change it to match a custom server name if your Oracle database uses a different configuration.

Default: GSXOUT

Applies to: Source

Xstream Low Watermark Retention in Hours

Description: Controls how long the XStream low watermark is retained in hours.

Explanation: This setting determines the retention period for the XStream low watermark, which is used for tracking the position in the change stream. A longer retention period provides better recovery capabilities but may require more storage.

Default: 24 hours

Applies to: Source

Trim text values

Description: When enabled, automatically trims whitespace from text values during data replication.

Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION, TRANSACTION_USER

Oracle Triggers Source Advanced Settings

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Query parallelism

Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

Default: 4

Applies to: Source

Enable JDBC row prefetch for snapshots

Description: When enabled, uses JDBC row prefetching during snapshot operations.

Explanation: This setting controls whether JDBC row prefetching is enabled during snapshot operations. Prefetching can significantly improve snapshot performance by reducing the number of round trips to the database.

Default: true

Applies to: Source

Snapshot batch processing multiplier

Description: Multiplier for batch processing during snapshot operations.

Explanation: This setting acts as a multiplier for batch processing during snapshot operations. It affects how many rows are processed in each batch during the snapshot phase, impacting both performance and memory usage.

Default: 10

Applies to: Source

CDC batch processing multiplier

Description: Multiplier for batch processing during CDC operations.

Explanation: This setting acts as a multiplier for batch processing during CDC operations. It affects how many changes are processed in each batch during the CDC phase, impacting both performance and memory usage.

Default: 20

Applies to: Source

Enable CDC direct-path reads

Description: When enabled, uses direct-path reads for CDC table operations.

Explanation: This setting controls whether direct-path reads are used for reading from the CDC table. Direct-path reads can improve performance for large data reads but may have different consistency characteristics.

Default: true

Applies to: Source

Use partitions for CDC table

Description: When enabled, uses partitioning for the CDC table.

Explanation: This setting controls whether the CDC table uses partitioning. Partitioning can improve performance and manageability for large CDC tables but may add complexity to the setup.

Default: false

Applies to: Source

Read CDC table data fetch size

Description: Controls the fetch size for reading data from the CDC table.

Explanation: This setting determines how many rows are fetched in each operation when reading from the CDC table. Higher values can improve performance but may increase memory usage.

Default: 10000

Applies to: Source

Trim text values

Description: When enabled, automatically trims whitespace from text values during data replication.

Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION, TRANSACTION_USER

Informix Source Advanced Settings

Max Concurrency (0 means no limitation)

Description: Controls the maximum number of concurrent operations (0 means no limitation).

Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

Default: 0 (no limitation)

Applies to: Source

Max Snapshot Concurrency (0 means no limitation)

Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

Default: 0 (no limitation)

Applies to: Source

CDC client timeout

Description: Controls the timeout for CDC client operations in seconds.

Explanation: This setting determines how long the CDC client waits for operations to complete before timing out. A higher value provides more time for complex operations but may delay error detection.

Default: 10 seconds

Applies to: Source

Logical storage space name (sbspace)

Description: Specifies the logical storage space name (sbspace) for CDC operations.

Explanation: This setting specifies the name of the logical storage space (sbspace) used for CDC operations. The sbspace is a special storage area in Informix used for CDC data.

Default: sbspace

Applies to: Source

Trim text values

Description: When enabled, automatically trims whitespace from text values during data replication.

Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

Default: false

Applies to: Source

Query parallelism

Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

Default: 4

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

Sybase Source Advanced Settings

Max Concurrency (0 means no limitation)

Description: Controls the maximum number of concurrent operations (0 means no limitation).

Explanation: This setting limits the maximum number of concurrent operations that the agent can perform. A value of 0 means there is no limitation. This can be used to control resource utilization and prevent overwhelming the source database system.

Default: 0 (no limitation)

Applies to: Source

Max Snapshot Concurrency (0 means no limitation)

Description: Controls the maximum number of concurrent snapshot operations (0 means no limitation).

Explanation: This setting specifically limits the maximum number of snapshot operations that can run concurrently. A value of 0 means there is no limitation. This is useful for controlling resource usage during initial data loads.

Default: 0 (no limitation)

Applies to: Source

Trim text values

Description: When enabled, automatically trims whitespace from text values during data replication.

Explanation: This setting controls whether text values are automatically trimmed of leading and trailing whitespace during replication. This can be useful for ensuring data consistency, especially when the source and target systems handle whitespace differently.

Default: false

Applies to: Source

Disable automatic triggers creation

Description: When enabled, prevents the agent from automatically creating triggers on the source database.

Explanation: By default, the agent can automatically create the necessary triggers on the source database for CDC operations. This setting disables that automatic creation, requiring triggers to be manually configured.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, operation type, and transaction user. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION, TRANSACTION_USER

MongoDB Source Advanced Settings

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

Redis Source Advanced Settings

Topic Prefix

Description: Specifies a prefix for topic names used in Redis CDC operations.

Explanation: This setting allows you to specify a prefix that will be applied to topic names used during Redis CDC operations. This can help organize and distinguish topics in complex environments.

Default: (empty)

Applies to: Source

Use Transactions

Description: When enabled, uses transactions for Redis operations.

Explanation: This setting controls whether Redis operations are performed within transactions. Using transactions can improve data consistency but may impact performance.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Cassandra Source Advanced Settings

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Snowflake Source Advanced Settings

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Couchbase Source Advanced Settings

Use Scope and Collections

Description: When enabled, uses Couchbase scopes and collections for data organization.

Explanation: This setting controls whether the agent uses Couchbase’s scopes and collections feature for organizing data. Scopes and collections provide a way to logically group data within a bucket.

Default: true

Applies to: Source

Index Replica Count

Description: Controls the number of replicas for indexes created by the agent.

Explanation: This setting determines how many replicas are created for indexes that the agent creates on Couchbase. More replicas provide better fault tolerance but require more storage and resources.

Default: 0

Applies to: Source

Indexes Build Timeout Seconds

Description: Controls the timeout for index build operations in seconds.

Explanation: This setting determines how long the agent waits for index build operations to complete before timing out. Higher values provide more time for complex index builds but may delay error detection.

Default: 30 seconds

Applies to: Source

Eventing Workers Number

Description: Controls the number of workers for Couchbase eventing operations.

Explanation: This setting determines how many worker threads are used for Couchbase eventing operations. More workers can improve throughput but may increase resource usage.

Default: 3

Applies to: Source

Eventing Script Timeout Seconds

Description: Controls the timeout for eventing script execution in seconds.

Explanation: This setting determines how long eventing scripts are allowed to run before timing out. Higher values provide more time for complex scripts but may delay error detection.

Default: 60 seconds

Applies to: Source

Eventing Background Polling Interval Seconds

Description: Controls the polling interval for background eventing operations in seconds.

Explanation: This setting determines how often the agent polls for background eventing operations. Lower values provide more responsive eventing but may increase resource usage.

Default: 30 seconds

Applies to: Source

Eventing Language Compatibility

Description: Controls the language compatibility version for Couchbase eventing.

Explanation: This setting specifies the language compatibility version for Couchbase eventing scripts. Different versions may have different features and behaviors.

Default: 6.6.2

Applies to: Source

Eventing Disable Automatic Function Configuration

Description: When enabled, disables automatic configuration of Couchbase eventing functions.

Explanation: This setting controls whether the agent automatically configures Couchbase eventing functions. When disabled, eventing functions must be manually configured.

Default: false

Applies to: Source

Delete Events On Sync Confirmed

Description: When enabled, deletes events after they have been confirmed as synchronized.

Explanation: This setting controls whether events are automatically deleted after they have been successfully synchronized to the target. This can help manage storage usage but may impact debugging and recovery capabilities.

Default: true

Applies to: Source

Enable Recursion Protection

Description: When enabled, prevents the agent from processing changes that were made by its own connection username to avoid infinite loops in bi-directional sync arrangements.

Explanation: In bi-directional synchronization scenarios, changes made by the agent itself could be captured and sent back to the source, creating infinite loops. When enabled, this setting instructs Gluesync to filter out operations executed by the agent’s own connection username.

Default: false

Applies to: Source

Custom 'type' field name

Description: Specifies a custom field name for the 'type' field used in Couchbase document type identification.

Explanation: This setting allows you to specify a custom field name to be used for document type identification instead of the default 'type' field. This can be useful when the default field name conflicts with existing document structure.

Default: (empty)

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

Kafka Source Advanced Settings

Topic Prefix

Description: Specifies a prefix for topic names used in Kafka CDC operations.

Explanation: This setting allows you to specify a prefix that will be applied to topic names used during Kafka CDC operations. This can help organize and distinguish topics in complex environments.

Default: (empty)

Applies to: Source

Use Transactions

Description: When enabled, uses transactions for Kafka operations.

Explanation: This setting controls whether Kafka operations are performed within transactions. Using transactions can improve data consistency but may impact performance.

Default: false

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

YugabyteDB Source Advanced Settings

Topology keys (ex. cloud1.datacenter1.rack1:1,cloud1.datacenter1.rack2:2)

Description: Specifies topology keys for data placement in YugabyteDB clusters.

Explanation: This setting allows you to specify topology keys that control data placement across the YugabyteDB cluster. The format is typically "cloud1.datacenter1.rack1:1,cloud1.datacenter1.rack2:2" where the numbers represent replica counts. This is useful for controlling data distribution and fault tolerance in distributed deployments.

Default: (empty)

Applies to: Source

Query parallelism

Description: Controls the degree of parallelism for query operations during snapshot and data retrieval.

Explanation: This setting determines how many parallel threads are used for executing queries during snapshot operations and data retrieval. Higher values can improve performance for large data sets but may increase load on the source database system.

Default: 4

Applies to: Source

Technical fields

Description: Additional technical fields to include in the replicated data for tracking and debugging purposes.

Explanation: This setting allows you to specify additional technical fields that should be included in the replicated data. These fields can include transaction ID, timestamp, and operation type. These fields are useful for auditing, debugging, and tracking the origin of changes.

Default: (empty)

Applies to: Source

Options: TRANSACTION_ID, TRANSACTION_TIMESTAMP, TRANSACTION_OPERATION

WAL sender timeout in seconds (supported from db version >= 12)

Description: Controls the timeout for WAL sender operations in seconds.

Explanation: This setting determines how long the agent waits for WAL sender operations to complete before timing out. This setting is only supported from YugabyteDB version 12 and above. A higher value provides more time for complex operations but may delay error detection.

Default: 60 seconds

Applies to: Source