Greenplum Agent for Gluesync: Target Integration

Index

Core principles

Greenplum speaks the PostgreSQL wire protocol. This agent connects to the coordinator through the built-in PostgreSQL JDBC driver and shares the PostgreSQL target’s type mapping, discovery and batch write path, narrowed to the types Greenplum has. What differs is what Greenplum needs to run well as a massively parallel (MPP) target:

Two write paths. The batch path writes through JDBC batches. The bulk path, for snapshots and for CDC, writes the rows to a spool file in Core Hub and loads it either through COPY on the coordinator, or through an embedded gpfdist server that every segment reads in parallel.
Distribution and storage. The tables the agent creates get a distribution policy, and optionally append-optimized storage with compression.
No ON CONFLICT and no MERGE. Greenplum 6 has no INSERT … ON CONFLICT and neither version has MERGE, so the agent applies an upsert as an UPDATE followed by an INSERT of the missing keys.

The agent is a target only. It does not call any Greenplum management API: it needs a database connection and, for the gpfdist load, network access from the segment hosts to Core Hub.

Supported versions

Greenplum Database 6.x and 7.x.

The integration test suite runs against Greenplum Database 6.27 and 7.1, each with two primary segments. A PostgreSQL server is rejected when the agent connects, with a message that points to the PostgreSQL agent.

Platform supported features

Feature Support Notes

Feature	Support	Notes
Target role	✅	Writes Greenplum tables through the bundled PostgreSQL JDBC driver, with TLS and pooling support.
Source role	❌	Greenplum acts only as a target within Gluesync.
Snapshots	✅	Snapshot rows are written by the batch path or by the bulk path.
CDC	❌	Target-only connector.
CDC type	N/A	Target-only connector.
User defined data types	✅	Enum types, arrays and `hstore` (with the extension installed) are written.
Bulk load	✅	Snapshots and CDC. Rows are spooled to a file and loaded into a staging table with `COPY` or through `gpfdist`, then merged into the target table; a snapshot of a table with an enforced unique key is loaded straight into the table. See Bulk load.
Table distribution and storage	✅	Tables created by the agent are distributed by their primary key (randomly when they have none) and can use append-optimized row or column storage with `ZSTD` or `ZLIB` compression.
Database pooling	✅	Supports connection pooling for efficient resource utilization across multiple pipelines.
Source persisted internal cache	N/A	Target-only connector.
Target persisted internal cache	N/A	Batch writes go straight to Greenplum. The bulk path keeps rows in a spool file only until the load commits.
Driver / connector	Built-in JDBC driver	The PostgreSQL JDBC driver, over the PostgreSQL wire protocol.
Chains	N/A	Target-only connector.
Truncate operation support (reading)	N/A	Target-only connector.
Truncate operation support (writing)	✅	Supports Core Hub’s "TRUNCATE before snapshot" option when acting as a target.

Target role

✅

Writes Greenplum tables through the bundled PostgreSQL JDBC driver, with TLS and pooling support.

Source role

❌

Greenplum acts only as a target within Gluesync.

Snapshots

✅

Snapshot rows are written by the batch path or by the bulk path.

CDC

❌

Target-only connector.

CDC type

N/A

Target-only connector.

User defined data types

✅

Enum types, arrays and hstore (with the extension installed) are written.

Bulk load

✅

Snapshots and CDC. Rows are spooled to a file and loaded into a staging table with COPY or through gpfdist, then merged into the target table; a snapshot of a table with an enforced unique key is loaded straight into the table. See Bulk load.

Table distribution and storage

✅

Tables created by the agent are distributed by their primary key (randomly when they have none) and can use append-optimized row or column storage with ZSTD or ZLIB compression.

Database pooling

✅

Supports connection pooling for efficient resource utilization across multiple pipelines.

Source persisted internal cache

N/A

Target-only connector.

Target persisted internal cache

N/A

Batch writes go straight to Greenplum. The bulk path keeps rows in a spool file only until the load commits.

Driver / connector

Built-in JDBC driver

The PostgreSQL JDBC driver, over the PostgreSQL wire protocol.

Chains

N/A

Target-only connector.

Truncate operation support (reading)

N/A

Target-only connector.

Truncate operation support (writing)

✅

Supports Core Hub’s "TRUNCATE before snapshot" option when acting as a target.