Gluesync architecture

Gluesync is an agent-based platform with a single orchestrator. Agents sit next to your systems and capture or write data; Core Hub holds configuration, transforms records in flight, and serves the Control Plane. Nothing in the path is a vendor-owned data store.

This page is the component and data-path reference. For the conceptual overview, start at What is Gluesync?.

The pieces

Gluesync data path Three lanes of data flow left to right. Source systems (Oracle or SQL Server, MongoDB or Couchbase, and IBM i or Db2) feed source agents that capture changes and buffer them in a local durable queue. The agents forward events to Core Hub, which maps, filters, transforms, and routes each record. Core Hub passes the result to target agents, which write to a warehouse, a relational database, and a streaming platform. The Control Plane UI, REST API, and Prometheus metrics sit above Core Hub. Conductor, Chronos, Automator, Query Studio, and AI Studio attach below it. Control plane Web UI REST API Metrics Source systems Source agents Target agents Target systems Core Hub map · filter · transform · route RDBMS Oracle · SQL Server Source agent Transaction-log CDC Target agent Bulk load · upsert Warehouse Snowflake · BigQuery NoSQL MongoDB · Couchbase Source agent Change streams Target agent JDBC · transactions RDBMS PostgreSQL · MySQL IBM i · Db2 Journal receivers Source agent Journal CDC Target agent Publish · partition Streaming Kafka · Solace Modules Conductor Chronos Automator Query Studio AI Studio
Piece Role

Source agents

Connect to one source technology, capture an initial snapshot plus ongoing change data capture (CDC), and buffer events locally before forwarding them.

Core Hub

Orchestrates pipelines, applies mapping and transformation, coordinates snapshots and writes, hosts the Control Plane UI and REST API, and exposes Prometheus metrics.

Target agents

Connect to one target technology and apply writes using the strategy you configured — insert, upsert, merge, bulk load, or message publish.

Pipelines

The configuration that binds a source agent, a target agent, and the entities (tables or collections) flowing between them.

Modules

Optional processes that talk to Core Hub for scheduling, agent deployment, backup, load testing, query, and AI. See Modules.

SDKs

Open-source handshake libraries so you can build your own modules or agents. See SDKs.

Agents are per-technology because that is where vendor drivers and SDKs live. Core Hub does not embed those drivers. Credentials and data paths stay with the agent that owns them.

Data path

A running entity moves through four stages. Snapshot and CDC share the same transform and deliver path.

Stage Where it happens

Snapshot

The source agent reads the current state of an entity and sends it through Core Hub so the target can be initialized or re-aligned. See Snapshot tasks.

Capture

The source agent reads committed changes from the system’s native mechanism — transaction logs, journals, change streams, or a CDC API. Supported source agents write those events to ArenaCache first (see [source-cache]).

Transform

Core Hub applies field mapping, filters, denormalization, computed values, field functions, and user-defined functions while the record is in flight.

Deliver

The target agent writes the result using write strategies and, where the target supports it, bulk loading.

One source can fan out to several targets. Pipelines can run in groups or chains, or bidirectionally with recursion protection so a change does not loop.

Agent-to-Hub channel

From Gluesync 2.2, agents connect to Core Hub through a plugin-based connector channel. The connection stays resident inside Core Hub rather than as a per-agent WebSocket session. That is what makes pooling, higher pipeline throughput, and Conductor-managed upgrades possible.

Modules that need interactive Control Plane access — Chronos, Bootstrapper, and similar — still use WebSockets.

If you are coming from 2.1, Conductor decommissions the legacy agent once the plugin is live. See What’s new in 2.3 for the plugin-system summary.

Source-side cache (ArenaCache)

From Gluesync 2.1.9, supported source agents include an embedded, persistent cache on local disk. From 2.2.11.2 that cache is ArenaCache, Gluesync’s own technology. It replaces Chronicle Queue, which filled the same role from 2.1.9 through 2.2.11.1.

ArenaCache ran in production on the Oracle LogMiner source agent for about six months before it became the default for every source agent that persists changes locally.

The cache sits inside the source agent process:

  1. The agent ingests changes from the database (journal, transaction log, CDC API) and appends them to ArenaCache — one file per entity.

  2. An internal worker reads those files in order and forwards events to Core Hub over the plugin channel.

  3. If Core Hub or the network is unavailable, the agent keeps caching. When the path is restored, it drains the backlog.

Compared with Chronicle Queue, ArenaCache uses less disk and IOPS, isolates a busy or paused table onto its own file, recovers from a truncated or corrupted file instead of stalling the agent, and avoids the Windows metadata-lock issues of the previous store.

That design has four consequences:

  • The source database is decoupled from Core Hub. Long-lived cursors and extra source-side artifacts stay off the critical path.

  • A Core Hub restart or a short outage does not force a resynchronization.

  • Retention is configurable via "Source change retention in hours" (default 24), so you trade disk for a longer replay window.

  • Core Hub stays stateless with respect to source buffering. As of 2.2.11, cache-based readers persist the CDC cursor in Core Hub; ArenaCache holds the in-flight backlog, not the reader position. Horizontal scaling of Hub processing is a separate concern — it is not declared GA here.

Agents on ArenaCache in 2.2.11.2: IBM i (standard journal reader), Informix, MariaDB, MySQL, Oracle Triggers, and Oracle LogMiner (already on ArenaCache; 2.2.11.2 splits one file per entity). Agents that never persisted a local source cache are unchanged. IBM i dedicated journal reader mode still bypasses the local cache — see IBM i journal data capture architecture.

ArenaCache is not a file-format compatible replacement for Chronicle Queue. After 2.2.11.2 the agent opens new cache files and does not read existing Chronicle Queue files. Upgrade when pipelines are caught up. For cache-based readers that persist position in Core Hub (IBM i, MySQL, MariaDB, Informix, Oracle LogMiner), the source cursor survives; the agent can re-read from that checkpoint if a local backlog cannot be carried over.

Gluesync does not treat this cache as a substitute for a backup. It is an edge buffer, not a system of record.

What Core Hub actually does

Core Hub is not a database you query for replicated rows. It is the control and in-flight processing plane.

  • Configuration and identity — pipelines, entities, users, API keys, licenses, module registration.

  • Orchestration — start, stop, pause, snapshot, redo, maintenance mode, group and chain execution.

  • In-flight processing — mapping, filtering, aggregation, denormalization, UDFs, field functions.

  • Interfaces — Control Plane on port 1717 (often fronted on 443 in production), REST API, Prometheus metrics, and the embedded MCP server used by AI surfaces.

  • Security — authentication (local accounts or OIDC), RBAC, TLS, optional node-to-node encryption under the Advanced Security license.

A redundant Core Hub pair and automatic failover are not part of current GA behavior. Size and deploy a single Hub per environment unless you have an explicit HA agreement. See Core Hub 101 and System requirements.

Where the platform runs

All components ship as container images (Kotlin on the JVM):

  • Linux on Docker or Podman, Docker Compose, or Kubernetes.

  • Windows Server with native nanoserver images — no WSL.

  • macOS for development.

Topology is yours: fully on-premises, fully in cloud, or hybrid with Core Hub in one place and agents next to the systems they serve. Lifecycle — install, OTA updates via Conductor, agent deploy — is driven from the Control Plane. See Deploy and run.

Prometheus and Grafana ship with the platform for pipeline, latency, and volume metrics. See Monitoring.

Modules

Modules extend Core Hub. They are separate processes (or, in Automator’s case, a standalone executable) that authenticate to Hub and add a capability. You deploy only what you use.

Module What it adds

Conductor

Deploys and updates agents from the Control Plane, including plugin-agent cutover.

Chronos

Time-based and event-based scheduling of syncs, snapshots, and Query Studio actions.

Bootstrapper

Scripted creation of pipelines and entities in bulk.

Automator

Standalone web UI over Bootstrapper, and the supported path for configuration backup, restore, and duplication.

Whisperer

Load generation against connected databases, for tests and benchmarks.

Query Studio

SQL workbench in the Control Plane against connected agents, with PII labels and result masking. See PII discovery and masking.

Query Forge

Federated SQL gateway for JDBC clients, with pushdown across SQL-capable agents.

Core Hub MCP server

Embedded MCP tools for external AI clients and for in-product agents.

AI Studio

Control Plane workspace for bring-your-own LLM agents.

DBMoto converter

Turns Syniti metadata.xml into Bootstrapper templates.

SDKs

The handshake SDKs implement authentication and the Core Hub protocol so you can build a custom module or agent without reverse-engineering the wire format.

SDK Status

Python

Available — GitLab

Node.js

Available — GitLab

Java and Kotlin

Coming soon — see SDKs overview

Security in the architecture

  • TLS on the Control Plane and agent APIs. See HTTPS and WSS.

  • Optional node-to-node encryption (AES-256) under Enterprise with Advanced Security.

  • Local users or OIDC, with RBAC (Super admin, Manager, Viewer).

  • Credentials stored with the agent that uses them, not in a shared data plane.

Recoverability

Automator exports Core Hub configuration — pipelines, agents, UDFs, mappings — so you can clone an environment or rebuild after a failure.

Those exports are configuration, not live CDC offsets. Pair them with source/target checkpoints (and, if you exported secrets, the same Core Hub storage key) for a complete recovery. See Backup and restore.

Related