Search docs.molo17.com

    Vertica Agent for Gluesync: Source and Target Integration

    Vertica logo screenshot

    Core principles

    This agent establishes a connection to Vertica via the Vertica JDBC driver, as a source and as a target.

    Change data capture

    As a source, the agent reads a snapshot of your tables and then captures their changes from the epochs of Vertica. Vertica has no log that a client can read and no row triggers, but every committed row carries the epoch of its commit, and AT EPOCH n SELECT …​ reads the database as it was at epoch n. The agent compares two epochs to deliver the net change of each poll window, so it creates no triggers, no shadow tables and no objects of its own, and never writes to the source.

    The snapshot reads the tables at one pinned epoch, in parallel hash buckets, so it is consistent while applications keep writing and CDC resumes from the same epoch without losing or repeating a change.

    To read more, see How it works, History retention and Limitations.

    Supported version

    As a source, Gluesync is tested against Vertica 24.1.

    As a target, you can write to any Vertica deployment, whether it is under Amazon AWS, Google Cloud or Azure.

    Requirements

    • For a source, HistoryRetentionTime of the database set to at least one hour, because changes are read from the history of the database. With the default of 0, the history lasts a few minutes. See History retention;

    • For a source, a database user with SELECT on the tracked tables and USAGE on their schemas and on the public schema. See Privileges;

    • For a source, a key on every entity that runs CDC: the primary key of the table, or the keys chosen on the entity.

    Platform supported features

    Feature Support Notes

    Target role

    ✅

    Writes to Vertica via the official JDBC driver with TLS, pooling, and load-balanced connections.

    Source role

    ✅

    Reads from Vertica via the official JDBC driver, with TLS and pooling. The agent only runs SELECT statements.

    Snapshots

    ✅

    Snapshot jobs read every table at one pinned epoch, in parallel hash buckets, and resume per bucket after a restart.

    CDC

    ✅

    The net change of each poll window: inserts, updates and deletes, with a key change delivered as a delete and an insert. See Limitations.

    CDC type

    Epochs

    Changes are read as the difference between two epochs (AT EPOCH). No log, triggers or tables are created in the database.

    Bulk load

    ✅

    Supports bulk load for high-throughput snapshot operations.

    Database pooling

    ✅

    Supports connection pooling for efficient resource utilization across multiple pipelines.

    Source persisted internal cache

    N/A

    Nothing is cached on the Gluesync side: the changes are read from the history of the database, and the position of each entity (an epoch and a row count) is kept by Core Hub.

    Target persisted internal cache

    N/A

    Writes flow directly to Vertica without local disk buffers.

    Driver / connector

    Vertica JDBC driver

    Vendor driver bundled with Gluesync for secure connectivity.

    Chains

    N/A

    Not supported.

    Truncate operation support (reading)

    ✅

    TRUNCATE removes the history of a table, so the agent reloads the table on the target (resyncOnHistoryLoss). See When the history is lost.

    Truncate operation support (writing)

    ✅

    Supports the Core Hub “TRUNCATE before snapshot” option when enabled.