Gluesync Conductor Static Repository

Gluesync Conductor is a container management platform that provides a comprehensive RESTful API for managing Docker containers, with specialized integration for Gluesync agent orchestration and lifecycle management within containerized environments.

Open Source This project is Open Source

Overview

Gluesync Conductor serves as the orchestration layer for Gluesync deployments by providing a centralized API for container management. It enables seamless agent deployment, configuration, and monitoring through integration with the Gluesync Core Hub, offering a streamlined experience for managing complex data pipeline infrastructures in containerized environments.

Built with Node.js and designed for flexibility, Conductor provides consistent RESTful API responses and supports advanced container operations including version management, persistence tracking, and automated updates.

Key Features

  • Container Management API

    • Complete RESTful API for Docker container operations

    • Container listing with detailed configuration and status information

    • Add, update, and remove containers programmatically

    • Container restart and image pulling capabilities

  • Agent Orchestration

    • Seamless integration with Gluesync Core Hub Web UI

    • Automated agent deployment from pipeline wizard

    • Dynamic agent provisioning based on pipeline requirements

    • Version checking and update management for containers

  • Configuration Management

    • Docker label-based container identification and tracking

    • Persistent container configuration through Docker Compose

    • Environment variable management and port mapping

    • Volume mounting and network configuration

  • Gluesync SDK Integration

    • Direct connection to CoreHub for advanced orchestration

    • License and security configuration management

    • Automatic service discovery and connection handling

    • UDP port range configuration for CoreHub communication

  • Monitoring & Operations

    • Real-time container status and health monitoring

    • Detailed logging and error handling

    • Container lifecycle management and persistence tracking

    • Support for multiple container runtimes (Docker, Podman)

Installation & prerequisites

System requirements

  • Node.js: Version 16 or higher

  • Container Runtime: Docker and Docker Compose (or Podman)

  • Package Manager: Yarn

Quick start

# Install dependencies
cd packages/gluesync-conductor
yarn install

# Start the service
PORT=50015 yarn dev

Environment configuration

For Gluesync SDK integration, configure the following environment variables:

  • GLUESYNC_LICENSE_FILE: Path to license file (default: /opt/gluesync/data/gs-license.dat)

  • GLUESYNC_SECURITY_CONFIG: Path to security config (default: /opt/gluesync/data/security-config.json)

  • COREHUB_UDP_PORT_RANGE: Port range for discovery (default: 1718-1727, recommended: 1717-1727)

API endpoints

Conductor provides a comprehensive REST API for container management:

Container operations

  • GET /containers: List all containers with detailed information

  • GET /containers/id: Get specific container configuration

  • POST /containers: Add new containers to the system

  • PUT /containers/id: Update existing container configuration

  • POST /containers/id/restart: Restart a container

  • POST /containers/id/pull: Pull latest container image

Version management

  • GET /containers/id/version: Check for available container updates

  • Integration with MOLO17 backoffice API for version information

Docker labels & identification

Conductor uses Docker labels to identify and track containers:

Container labels

  • com.molo17.conductor.unique_id: Stores container nickname as unique identifier

  • com.molo17.conductor.versiontag: Tracks container version tag

  • com.molo17.conductor.type: Identifies container as Gluesync agent

Persistence tracking

Containers added through the API are automatically persisted to the Docker Compose configuration file, ensuring they survive system restarts and can be managed as part of the infrastructure.

Integration with Core Hub

Conductor integrates deeply with the Gluesync Core Hub to provide:

  • Web UI Integration: Direct agent deployment from pipeline creation wizard

  • Status Synchronization: Real-time agent status updates in Core Hub interface

  • Configuration Management: Centralized agent configuration through Web UI

  • Security Coordination: SSL certificate management and validation

  • Service Discovery: Automatic agent registration and discovery

Container platform support

Docker & Docker Compose

Primary support for Docker environments:

  • Direct container management through Docker API

  • Docker Compose stack integration

  • Network configuration and service linking

  • Volume management and persistent storage

Alternative runtimes

  • Podman: Support for Podman container runtime

  • Kubernetes: Integration capabilities for K8s environments

  • Container runtime abstraction for flexibility

Best practices

  • Version Management: Regularly check for and apply container updates

  • Resource Allocation: Configure appropriate CPU and memory limits

  • Persistence: Ensure critical containers are properly persisted

  • Monitoring: Implement logging and monitoring for container health

  • Security: Use SSL/TLS for API communications in production

  • Backup: Maintain backups of Docker Compose configurations

Troubleshooting

Common issues

  • Connection Failures: Verify Docker daemon accessibility and API connectivity

  • Version Conflicts: Check container image versions and compatibility

  • Resource Constraints: Monitor system resources and container limits

  • Network Issues: Validate network configuration and port mappings

  • Persistence Problems: Ensure Docker Compose files are properly maintained

Diagnostic tools

  • Container logs: docker logs <container_id>

  • API health checks: Test endpoints with curl or API client

  • Docker Compose status: docker-compose ps

  • Network diagnostics: Container networking verification

Development

Conductor includes development tools for code quality and testing:

# Code quality checks
yarn lint
yarn type-check

# Testing
yarn test

# Documentation
yarn docs

Related topics

Release Notes

0.4.0

Released: November 24, 2025

  • Auto adopt previously hand held services;

  • Removed depends_on in adopted services (even "adopted by hand");

  • Handled update of Conductor by doContainersAction update api;

  • Auto heal Conductor conf if some configuration of Conductor is wrong it will self heal (for now only logs file);

  • If Auto healed auto update Conductor version, no more latest in docker-file;

  • Edited get version api to notify when modules have mandatory updates to be applied.

0.3.1

Released: November 14, 2025

  • Handled read of docker file configuration both in List Style and Map Style;

  • Handled updates only for services handled by Conductor;

0.3.0

Released: November 12, 2025

  • Update with an empty array update all must accept the releaseChannel parameter;

  • With update all: update only containers with core-hub, agent, and module (no conductor);

  • List all containers except third party containers (grafana, prometheus, portainer, or traefik);

  • Fix current version in get latest;

  • Handled null version tag in get latest (which does not fires an actual update);

0.2.1

Released: November 10, 2025

  • Updated Node.js SDK to the latest version;

  • Re-send license headers at every reconnection to the Core Hub;

  • Filter out containers without Id.

0.2.0

Released: October 30, 2025

  • Added support for Collect logs API.

0.1.1

Released: October 26, 2025

  • Added support for Agents TAG environment variable;

  • Fixed missing agent’s data folder.

0.1.0

Released: October 24, 2025

  • Initial release