Gluesync Architecture

Gluesync’s architecture is built on a flexible, agent-based system centered around the Core Hub, with extensibility through SDKs and modules. This design enables real-time data integration across diverse platforms while ensuring scalability, reliability, and security.

Architecture Overview

Gluesync 2 Architecture

The architecture consists of five main components:

  • Source Agents: Handle data extraction and change capture

  • Core Hub: Orchestrates data flow and system operations

  • Target Agents: Manage data loading and transformation

  • SDKs: Enable developer integration with Core Hub

  • Modules: Extend platform functionality through SDK-based components

Core Components

Core Hub

The Core Hub is the central orchestrator, featuring:

  • High Availability

    • Core Hub #1 and #2 for redundancy

    • Automatic failover support

    • Load balancing capabilities

  • Data Processing

    • Aggregation engine

    • Denormalization support

    • Transaction coordination

  • Management

    • Administration interface

    • Monitoring dashboard

    • Security controls

Source Agents

Agent Type Capabilities

NoSQL CDC

* Natively integrated NoSQL CDC * Real-time change capture * Event-based replication

RDBMS

* Transaction-log based CDC * SQL database support * Real-time monitoring

Data Lakes

* AWS S3 & S3-like support * Dell ECS integration * Azure blob storage support

Target Agents

Agent Type Features

NoSQL

* Native integration * Load balancing * Multi-target support

RDBMS

* JDBC-oriented connections * Transaction pool management * Bulk loading capabilities

Data Lakes & Streaming

* S3 and blob storage support * Kafka integration * Solace messaging support

Communication Flow

Data Flow Diagram

diagram

  • WebSocket connections enable real-time bidirectional communication

  • Core Hub processes and transforms data in-flight

  • Multi-target support allows for parallel data distribution

Advanced Features

Data Processing

  • Aggregation Engine

    • Real-time data aggregation

    • Custom aggregation rules

    • Performance optimization

Denormalization

  • Smart Denormalization

    • Configurable strategies

    • Automatic schema mapping

    • Performance tuning

Multi-Source Support

  • Event-based Integration

    • Multiple source connections

    • Parallel processing

    • Consistent ordering

Deployment Options

Flexible Deployment Models

Model Description Best For

On-Premises

Full deployment within your infrastructure

High security requirements

Cloud

Deployment on AWS, Azure, or GCP

Scalability and flexibility

Hybrid

Mix of on-premises and cloud components

Balanced approach

Security Architecture

Security Measures

  • Network Security

    • TLS encryption

    • Secure WebSocket connections

    • Network isolation options

  • Authentication

    • API key authentication

    • Role-based access control

    • Session management

  • Data Protection

    • In-transit encryption

    • Secure credential storage

    • Audit logging

SDKs and Developer Integration

Open Source SDKs

Gluesync provides open source SDKs that enable developers to build custom integrations and extensions:

SDK Features

Python SDK

* Core Hub handshake protocol implementation * Authentication and authorization * Event handling and processing * Available at GitLab

Node.js SDK

* JavaScript-based Core Hub integration * Real-time event processing * Promise-based API design * Available at GitLab

Coming Soon

* Java SDK * Kotlin SDK

Developer Benefits

  • Open Platform

    • Build custom agents and modules

    • Extend Core Hub functionality

    • Access Gluesync APIs securely

  • Integration Support

    • Comprehensive documentation

    • Reference implementations

    • Community-supported examples

Modules

Platform Extensions

Modules are platform extensions built on top of Gluesync SDKs that enhance system capabilities:

Module Functionality

Chronos

* Advanced scheduling capabilities * Time-based job orchestration * Recurring task management * Available at GitLab

Bootstrapper

* System initialization * Configuration management * Deployment automation * Available at GitLab

Module Architecture

diagram

  • Modules connect to Core Hub through SDK interfaces

  • Custom modules can extend platform capabilities

  • Open architecture enables third-party development

Monitoring & Administration

Built-in Monitoring

  • Real-time metrics collection

  • Performance monitoring

  • Resource utilization tracking

  • Alert management

Administration Tools

  • Web-based admin interface

  • REST API access

  • Configuration management

  • System health monitoring

For detailed deployment instructions, see our Deployment Guide for Docker Compose and Kubernetes.